Gary Marcus here again to throw cold water on anything deep learning. The negative tone when not succeeding in tasks it wasn't designed or advertised for continues to be disappointing.

> To the extent that the goal is to develop artificial intelligence that can be trusted in safety-critical applications (Marcus & Davis, 2019), a much higher standard must be applied.

I have seen no mention of that being the intended use of DALL-E (2). In fact, the most common use case I've seen described is in replacing Fiverr-type tasks: quick graphic design.

That said, the results are actually encouraging given that it _wasn't_ designed to succeed here:

> Nevertheless, for 5 out of the 14 prompts, at least one of the ten images fully satisfied our requests.

Some of the authors' interpretations could be argued against, as well. For instance, in example 10, "An old man is talking to his parents":

> In none of these images did DALL-E successfully infer that image should show an old man with two even older people

Several of the images appear to show exactly that? How is the author judging "even older"?

As annoying as the constant goalpost moving is in AI, it's also true that how these models fail is much more interesting than how they succeed. That's where all the information is. Dalle-2 is better at some things than others. Why? What's a limitation of the model, and what's an accident of architecture? I'm reminded of GPT-3's difficulty rhyming and doing math, which seems to be a result of the tokens it was trained on, compared with trailing off and inconsistency which may be more foundational.

I completely agree. However, here are some things a more useful study would cover:

- Describe the task creation process, why they are appropriate for measuring X, etc. Preferably drawing from similar studies in people instead of ad hoc. Lots of work has already gone into measuring these things. They may have to be modified for DALL-E but it would be a better starting point.

- Show several variations of the same task and outputs. For example #11, maybe the issue is that DALL-E has a poor understanding of milk sizes as expressed but not size relationships in general. Example #13 with pizza sizes is ambiguous by the authors own interpretation yet they deem it a failure. It would be trivial to construct dozens of similar examples to give a more holistic understanding.

- Narrow the scope of the prompts if you want to see how the model understands a particular relationship. Many of the tasks include multiple ancillary statements.

- Discuss prompt engineering in more depth. We already know that these models are sensitive to the formulation of the prompt. What did the authors try / not try?

- Replace the authors' individual opinions of the outputs with crowdsourced opinions from mturk or even Twitter. As I noted above, example #10 is not as clear cut as the authors suggest.

- Measure how well people do at the same task as a baseline: draw something that aligns with a given text prompt and compare to DALL-E, maybe with a crowdsourced opinion of which is a more accurate interpretation. Even if they're just stick figures this would be interesting to see.

As-is, this article doesn't really add anything substatial about the model's capabilities to the conversation.

> Why? What's a limitation of the model, and what's an accident of architecture?

The limitation is caused by the CLIP model they used to encode text and images. It's a separate model only generating an embedding, it's not using attention and pairwise interactions on the whole sequence. This causes Dalle2 to be bad at handling multiple objects with multiple attributes. There is no reason the complete prompt could not be related to the generated image instead of an embed, thus correctly stacking the coloured cubes and assigning the right age to each person mentioned in the prompt.

I was actually gonna say something like that, but I believe that the full prompt is also made available to the GLIDE portion. Even still, GLIDE isn't a language model it simply can't understand the full relational content of the caption. With changes running from minor tweaks to major upsizes that's probably fixable.

what goalposts changed? i pointed out same limits in 2001, 2012, 2018, 2019, 2020.

please actually read my work and please don’t make stuff up.

I think you'll find if you reread my comment I was defending your paper here! We agree on little, but your insights are appreciated.

I didn’t get a negative tone from the paper, it seemed mostly positive to me , just calling out a handful of areas ( like relative positioning ) where Dall-e fails .

Many of the perceived failures of DALL-E 2 are subjective and ambiguous (by the author's own interpretation in some cases!) and speak to the author's negative biases which are well known outside of this publication.

To be clear, I'm not defending DALL-E 2. I'm criticizing a poorly written paper that was published to Arxiv to lend further credibility for a Twitter audience to substantiate a claim that the DALL-E 2 authors have not made but that Gary Marcus has a vested interest in perpetuating:

> How much does DALL-E have to do with AGI? Maybe not so much, after all… A lesson in caveat emptor: - https://twitter.com/GaryMarcus/status/1521120022298464256

This should have been a blog post or Twitter thread like the dozen or so other experimentations people have done with the system.

Yikes, that Twitter feed: "Thoughts and prayers for the deep learning fanboys," as if ones admiration for DALL-E 2's achievement is something to be mocked... I feel like this sentiment is the way we're going to get I Have no Mouth, and I Must Scream. There's just something in there about negative AI minimalists or AI alarmists (same coin different side).

> Many of the perceived failures of DALL-E 2 are subjective and ambiguous

Could you expand on this a bit? DALL-E provides graphical, interpretive, output meant for humans. Isn't that necessarily subjective? Any qualitative metric of DALL-E 2 will need to involve some aggregate of humans being subjective. The papers title is "A very preliminary analysis of DALL-E 2", so low data points/opinions/speculation/further questions should be expected.

Heh, so many of these people's reaction to 'we have not created AGI' seems to be mocking and pessimism that we ever will.

My response to 'we have not created AGI' is

"Thank goodness".

I don't think we're ready for that yet.

Marcus is only one of three. From what we know if Aaronson, I doubt that this was a Marcus-driven paper with the others just along for the ride. But I agree some of the intro has a Marcus ring to it.

For me, I read the introduction and how they discuss how impressive DALL-E 2 is, and then they show very simple mistakes it makes and I want to be protective of DALL-E 2. I want to say "you did a good job!"

That the paper provides "a clearer picture of what remains to be done" is very hard to accept, as all it does is show edge cases which are subjective at best. If anything the picture is less clear as they don't even try to form a hypothesis why DALL-E makes mistakes like these. One thing in particular I have noticed is that DALL-E has trouble producing action images. It may be because it has no sense of temporality and in those cases it could serve to run the parameters a bit longer using the same scene. But I am not an AI scientist so what do I know.

I'd be curious to compare the results of Dall-E 2's output vs a group of human artists each individually given the exact same text prompt (with no follow up clarification allowed) and asked to produce 3-5 drafts.

> ... it is reasonable to question whether DALL-E 2 constitutes progress toward solving the deep challenges of commonsense reasoning, comprehension, reliability, and so forth that would be needed for a truly general-purpose AI ...

I like how they pose this question as a bait to the abstract yet they do not even attempt to answer it. Instead they focus on general shortcomings of the model.

Moreover, there is no proper conclusion which would discuss the findings.

Given how outspoken Gary Marcus is on Twitter - criticising current advances in DL I would expected him to do a much better job publishing a document about it.

I think you are being disingenuous by omitting the preceding claim.

> Whether results of this kind should be considered as successes for the program – what is the proper measure to use in evaluating success – depends on the intended use of the program. If the goal is to generate candidate images that a graphic artist will choose from, or choose from and edit, then the system can reasonably be measured in terms of the quality of the best result out of ten or out of one hundred.

They basically admit your Fiverr use case is valid. But say, that it should not be used "in safety-critical applications" which is neither a grand claim nor controversial. It is probably the most blasé claim because, as you point out, no one is expecting this to be used in safety-critical applications. From an economics point of view, the Fiverr use case seems pretty strained to me. If you've ever watched street art, some dazzling things can be done in under 10 minutes. Unless the DALL-E gets it correct on the first shot +99% of the time, someone sifting through images is probably just as costly as paying for Fiverr. What this paper elucidates to me is that even historical figures are off-limits which, in my expectation, is a non-trivial use case.

> someone sifting through images is probably just as costly as paying for Fiverr

The difference being that anyone can sift through images and identify good ones, whereas few people can create good images. The amount of time it takes to complete the task may be the same, but the number of people who can do it greatly increases.

If someone says, "Cars may be great at transportation to and from work but we should be cautious to use them as a counterbalance for a trebuchet." I don't think it's disingenuous to take issue with the second part of the claim and to not give the author credit for the first part of the claim.

Right, unless the creator of the car's mission is to use it as a counterbalance for a trebuchet. From [1]:

OpenAI is an AI research and deployment company. Our mission is to ensure that artificial general intelligence benefits all of humanity.

[1] - https://openai.com/about/

I think that's an unfair perspective.

> Our mission is to ensure that artificial general intelligence benefits all of humanity.

Saying that DALL-E isn't achieving their mission of beneficial AGI is like getting mad at NASA for pointing their test rockets sideways when trying to go to the moon.

All technology known, biological or man made, was created with incremental improvements, with the early iterations almost alway being unhelpful, and sometimes creating a burden. For example, the construction/invention of the first plow was completely unhelpful, and nothing but a distraction, from that years crop. The incremental improvements after now have it where the majority of humanity would perish within a year if you were to remove modern plows from existence.

There's no instantaneous way to achieve their mission. The incremental steps to achieve their mission don't have to individually achieve that mission. That's not realistic or a rational expectation.

Disregard the above if your perspective is that all technology is bad for man, which I've seen rationally discussed.

I don't see what's unfair. It's a pretty uncontroversial take and I feel like the people replying are being unfair by using tangentially related analogies. In your analogy, no one is getting mad at NASA here. It's like it is WWII and Germany is hitting London with rockets. The claim here is like "these things are great if we want to launch explosives from this side of the channel and we don't care what we hit in the city. But if we're aiming for the moon, we still have a long way to go." Like, it is just a such an indifferent claim that I'm honestly shocked people are taking issue. For the past month, all I've been hearing is how amazing DALLE-2. If you were a layman, you might believe that we're at AGI. This is simply a sobering reminder not to get caught up in the PR machine. The authors have 10x the text espousing the impressive result, what is wrong with a single sentence?

How quickly they forget. 45 minutes after the DALL-E n release, Sam Altmann tweeted “AGI is gonna get wild”; Greg Brockman posted multiple tweets around the idea of AGI and walls broken. That invites a critical analysis.

Ad hominem against me won’t remedy the limitations that we and others have observed.

This doesn't seem like an ad hominem attack at all. It's a criticism of criteria you're using to analyze DALL-E in this paper. You're talking about safety-critical applications, but nobody is expecting DALL-E to be used for those. Even your examples are just people tweeting generally about the future of AGI (the obvious context being that this is a big demonstrable step forward, not that DALL-E is an AGI that society is going to rely on for critical tasks).

Sure, if you analyze it based on the criteria you've set out, it fails. The point is that nobody else cares if it fails based on that criteria, because nobody else thinks that criteria is relevant.

This is not very substantial work. They used the API, gave it more challenging questions on 14 prompts, and judged whether it met expectations. I would have expected something much more intelligent from Aaronson, although this kind of light-weight analysis is par for the course with Marcus, who has been fighting his rearguard action against large parametric models for many years, and is not above motivated reasoning. It's like one of those open letters that people get roped into, only hosted on arXiv.

I don't see much of a downside to a light, quick exploration of a new tool and a writeup shared publicly. While this could have been a (very long) blog post, it seems like arXiv is a good place for this "report of findings".

I agree. The title is "A very preliminary analysis of DALL-E 2". There is really nothing offensive about this publication.

In this case the only thing "light" is the paper itself and manner in which the authors scrutinized the work.

The prompts themselves were very complex and thought through, probably a result of a lot of cherrypicking to find weaknesses in the model.

Still, it is impressive how DALL-E dealt with them even in cases where it got it wrong.

It's nice to see some examples that haven't been filtered by the creators as being the best ones, to show how capable it really is. This paper (at least to me) makes me even more impressed by DALL-E 2, it hasn't parsed the input exactly but it's still incredibly impressive imo.

I wouldn't describe these as challenging, the prompts are fairly simple if you're capable of understanding pretty basic concepts.

> A pear cut into seven pieces arranged in a ring.

> A couple in formal evening wear going home get caught in a heavy downpour with no umbrellas

The work successfully demonstrates that the API lacks basic conceptual understanding by example.

To quote the generally incredible Scott Aaronson with a somewhat more optimistic perspective:

Judged against where AI was 20-25 years ago, when I was a student, a dog is now holding meaningful conversations in English. And people are complaining that the dog isn’t a very eloquent orator, that it often makes grammatical errors and has to start again, that it took heroic effort to train it, and that it’s unclear how much the dog really understands.

This is a very serious paper about a very serious topic, AI and AGI and how the...

Who am I fooling? They're clearly just messing around and having fun. I don't begrudge them, though.

It would be an interesting experiment to see where along the path some of these details are lost. For example, in the experiment where they ask for a pear cut into 7 slices, and Dall-E often gives the wrong number. First check if you can train a classifier on CLIP embeddings of the caption to recover the specified number. If so, check if you can do the same for the image embedding the prior produces. Then check if you can do it for unCLIP's internal feature maps, and so on. Similar experiments could be conducted for other lost attributes like object relationships (on top, behind, etc.) or specific identities (Lincoln, Washington, etc.) .

Very good points, but I would start with the obvious one - the contrastive objective of CLIP forces the prompt representation to a single embedding vector. So it acts as a bottleneck, limiting the number of object-attribute pairs it can simultaneously encode. And of course the solution is to use an autoregressive setup like Dalle-1, but then they can't have the same diffusion model decoder for image generation.

I bet they consciously prioritised the artistic applications over exact semantics. The CLIP embedding space has nice properties, it's tempting to use it. From my experience semantic similarity based on embedding dot products is much easier to do than exact semantic matching. Three is similar to four and red similar to blue in embedding space.

Yes, DALL-E2 clearly gets confused when dealing with multiple objects, where each object is given specific attributes.

However, it does answer SOME of these types of prompts correctly: It often correctly handles the simplest case of two objects with one specific property each.

My prediction is that since it can handle SOME of these cases already, it means this problem has shown itself to be tractable, and future ML researchers will be able to chip away at the deficiencies with additional effort.

I'm confident the next versions of these AI systems will handle the prompts given in this paper with ease.

I bet a lot of the issue is just the CLIP encoding. It's doing a ton of work getting text prompts into vector form, and it's incredible it does so well as is, but past a point you're squeezing blood from a stone. It's just not a full language model! Bump up the size, or figure something out about the space it's working in, I bet you'd see huge improvements.

From what I've read elsewhere DALL-E2 can't handle overly long/complex requests. Pretty much everybody who has tried it has discovered the same thing. Keep the requests simple.

I really liked this paper. On the one hand, all they did was come up with some prompts and show the results. On the other hand, it does a really good job of being clear about exactly what they queried and why and how much selection they are applying (eg. they mention accidentally discarding a result once; this kind of thing was always very unclear in people's twitter posts of DALLE2 images). Also the paper highlights things that the model is clearly still struggling with, which researchers can then focus on improving.

But I expected more clever reasoning and insight into what specific properties of prompts cause failures, in particular by trying variants of prompts. And more discussion of the weird style issue that causes "old man talking to his parents" to always be in a specific web-comic style.

They look a lot like the art from the copyright-free Japanese clip art site irasutoya.

https://latelyjapanese.com/culture/20201214/do-you-know-abou...

> Caption: a red ball on top of a blue pyramid with the pyramid behind a car that is above a toaster.

Have you ever heard humans talking like that? I bet you each person would also draw it differently. I consider myself human and yet I don't grasp a concept of a "car that is above toaster". What do they mean? Like hovering? Or just standing on the top? Cars aren't UFOs, they don't hover.

The whole paper seems like they tried to break the system with tasks even humans would have problems grasping. What's the point??

It’s like an IQ test. I think it’s a good way to probe the “cognition” (if you will) of the model.

Sure, humans would probably differ in how they draw that picture. IQ 150 humans would almost all get the positional relationships correct though, even if the precise interpretation of “above” might differ. The fact that none of the images were correct under any interpretation of the relationships shows something meaningful, namely that this model can’t understand long chains of relationships.

The paper complains that some of the answers are not perfect but half the humans I know would have gotten them wrong… (thinking about the octopus one). I just find this technology fascinating. Imagine once you can extend this to video (“Star wars but the Jedis are the bad guys”, “breaking bad but an 80s sitcom” … Netflix should be investing in this…)

The original paper examines some likely causes for failure to encode relational details between subjects, e.g. the corgi with a red hat and green necktie or vice versa chosen seemingly at random. The model seems to learn about contrasting subjects but not track particularly which attributes belong to which subjects. This is, as far as I can tell from the original paper, inherent in the training and would require different data sets or more parameters to learn a greater number of subject-attribute associations in each image.

https://arxiv.org/abs/2204.06125

Because many of the examples seem to show that the system easily misunderstands certain grammatical implications (prepositional relations, etc), I wonder if it would benefit from of different sort of input system in which the relationships of the objects in the desired image are more concretely defined, something more like a tree graph perhaps? Or perhaps this would be very difficult to implement?

At the very least, seems it would be useful for users to edit a given output so that it can be continually tweaked, keeping what they like about an image (e.g. object layout, proportions, color scheme) while editing other aspects.

The criticism that autoregressive language models with billions of parameters don’t acquire basic logics (e.g., propositional, quantificational) is valid and, in my opinion, extremely interesting. Unfortunately, the subjectivity of their evaluation undermines that critique. The subjectivity is especially weird since these characteristics can be objectively evaluated (e.g., finding contradictions).

This paper notes that average output can be flawed...which is the case with all AI-generated content and why it's important to note the importance of cherry picking/selection bias whenever posting it publicly.

Fortunately, AI Image generation has helped visually communicate how AI content generation isn't quite sci-fi magic where you always get what you want with zero ambiguity. Yet.

It’s an interesting point - cherry-picking muddies the water in an academic context. However it’s how these models will actually be used in the real world (with a human “editor” in the loop) so in some sense you care more about “best of N” performance. Or maybe even “best performance from someone that knows how to AI-whisper” since these are the new content creators that will produce the best output with these models (or their descendants).

This is really a case where the real-world market will provide an assessment regardless of the sort of academic assessment of quality in the OP.

GPT-3 take on the paper abstract and title:

> Reviewer #2: The authors provide a complement to the original DALL-E paper, but the paper is light on empirical evaluations. It is not clear why, for some of the prompts, the generated images are so poor and for others, they are pretty good. Why not provide a quantitative evaluation of this?

(cherry picked from 10 attempts, temp=1)

Honestly try getting a bunch of PhDs in computer science to solve the same task and I think you will get dramatically worse results, not only artistically but also in terms of comprehension...

The toaster/pyramid/ball one makes no sense to me - "with the pyramid behind a car that is above a toaster"?

Sorry for the n00b question, but how do I see the actual paper?

Press the “Download PDF” button.

How do you try this out? Is there a free alternative you can try? I would like to make some backgrounds for a platformer game.

How do you try this out? Is there a free alternative you can try? I would like to make some backgrounds for a platformer game.

Can someone remind of the link that turns arxiv papers into vaguely readable HTML?

https://www.arxiv-vanity.com/ but… "This paper doesn't have LaTeX source code, so it can't be rendered as a web page."

Are these people just trying to pad their citation count?

It really looks like nothing more than an hour's work to write this 'paper'.

Some academics are experts in also being science influencers. In those cases I actually know their non-scientific work better than their actual scientific contributions. That is I know Scott Aaronson from Shtetl optimised not from any particularly memorable scientific accomplishment, which he might well have.

He's surely a leading light in complexity, but since it's not my field I also know him more from the blog.