about
GLIDE: Photorealistic Image Generation, Editing w/ Text-Guided Diffusion Models (arxiv.org)
26 points by minimaxir on Dec 21, 2021 | hide | past | pdf | 6 comments on HN

In plain words: It turns a caption into an image by refining random noise, training the model to follow text without a separate checker. Human judges found its images more realistic and better matched to captions than the alternative, and preferred them over DALL-E's best-picked samples.

Abstract · GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models

Diffusion models have recently been shown to generate high-quality synthetic images, especially when paired with a guidance technique to trade off diversity for fidelity. We explore diffusion models for the problem of text-conditional image synthesis and compare two different guidance strategies: CLIP guidance and classifier-free guidance. We find that the latter is preferred by human evaluators for both photorealism and caption similarity, and often produces photorealistic samples. Samples from a 3.5 billion parameter text-conditional diffusion model using classifier-free guidance are favored by human evaluators to those from DALL-E, even when the latter uses expensive CLIP reranking. Additionally, we find that our models can be fine-tuned to perform image inpainting, enabling powerful text-driven image editing. We train a smaller model on a filtered dataset and release the code and weights at https://github.com/openai/glide-text2im.

Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, Mark Chen
arXiv:2112.10741 · cs.CV, cs.GR, cs.LG · submitted Dec 20, 2021 · updated Mar 8, 2022
abstract · pdf · html · 20 pages, 18 figures

add comment on HN

I don't understand how this didn't make the front-page, it's absolutely mind-blowing. Or am I missing something and it's not that much of a revolution?
It's a bigger, better version of DALL-E from earlier this year: https://openai.com/blog/dall-e/

It's good, important work, but not so new that I'd expect to see hundreds of upvotes. Also, mobile users generally won't read PDF submissions.

GLIDE is NOT Dall-E. Dall-E is a transformer (basically GPT-3), while GLIDE is a diffusion model. While they share some similarities, the major difference is that transformers generate image sequentially from top to bottom, pixel-by-pixel (technically, token-by-token), so one can condition them only by the text and the top of the image. At the same time, diffusion models predict all pixels at the same time, so one can naturally trade compute for result quality (do more inference iterations) and, beside sampling, do other image manipulation tasks, like text-prompted inpainting.
It's easily an impressive enough leap to get hundreds of upvotes. It's about that fact that it takes two clicks (including waiting for arxiv's absurdly slow PDF download speeds), and then some scrolling to get the results.

As someone else said, the blog post that they'll likely release will hit front page.

Surprisingly OpenAI did not release a blog post with this paper: I suspect that will be the one that makes the front page.