It was already known that training on code improves performance on all reasoning tasks[1], so this is not really surprising.

[1] https://arxiv.org/abs/2210.07128

I don't think it can be dismissed that easily. You could imagine that most of the benefit from being trained on programming languages comes from learning any language at all, but a 15% gain from one language to another is pretty surprising I'd say.

This is true with natural language as well in LLMs, ASRs, and more.

At Rev we found that adding data from other languages to primarily-English ASR models helped reduce WER in the target language substantially (especially when accents were involved). We found similar benefits when training translation models.

I've just realized that the most ultimate "no good deed goes unpunished" is that the mountain of open source code released by programmers will be used by LLMs to put them out of jobs.

And the mountain of concept art created by digital artists and uploaded to the web is putting them out of jobs thanks to DALL·E and friends.

And the mountain of writing created by authors and uploaded to the web is putting them out of jobs.

And the mountain of music created by musicians is going to start putting them out of jobs.

Generative AI is an unprecedented transfer of wealth-generating capability from creative people to giant corporations.

And it will occur faster than any transition in history. Meatspace has natural limits to how fast things can do this.

But, contrary to at any such developments in history, the people who are losing out aren't the "poor working class". They already have cultural and political power that they stand to lose. This might still go a lot of ways.

Some of them sure, but you're painting with the broadest brush possible. Do you really think the average artist is not "poor working class", that they have cultural and political power?

Maybe not the average but the power is concentrated in a few and they are also affected. I'm expecting many who collectively do have power to participate in shaping how these things are used and perhaps oppose the big companies.

You imagine there won’t be other limits, but maybe we just don’t know what they are yet? It tends to be true that some things that are easy for people are hard for AI.

This sounds a bit more like a cope than any substantiated claim, what tasks would AI struggle with?

Neither claim is substantiated. We're just imagining different futures. You can imagine what you like.

It is a fair point and I hope you're right because I think humans being second best at everything will probably have bad outcomes for most of us but given the current trajectory, it does seem like a run away train for now.

in a world where all office workers are now permanently unemployable: what use are Microsoft's products?

AI kills 90% of demand for their products

I don't think roofers need Microsoft Teams

But Microsoft doesn't sell to office workers, they sell to executives... who would relish the idea of getting rid of those office workers. And guess who owns "Open" "AI"...

Shareholders will replace executives immediately, and I don't think it's far off.

If you think executives are safe from AI, well I think you're crazy.

Imagine saving hundreds of millions a year in executive salaries by hiring a robot that has 1000x better strategic capabilities than you, is 10x cheaper, has absolutely zero empathy, never sleeps, just optimizes for money and just makes the required cold hard decisions ?

Shareholders will go fucking nuts for this.

will the executives buy legions of Windows machines if they don't have any workers?

No, but Windows is a loss leader for Microsoft now. The real money is in AI and Azure.

Fortunately we’ll get revolutionary new products from it, such as calendar app (with AI integration), email app (with AI integration), resume builder app (with AI integration), and more!

oh no and a wave of patents with 'using ai' it will be 1999 all over again.

I didn’t touch LLMs yet, but was hardcore learning SD and satellite nets last two weeks in a full workday mode. If they are similar, then I assure you that the complexity of being able to create something detailed that you want is on par with bare full-stack of stacks style programming. My brain melts down at the end of every day. It will put people out of jobs. But the hiring pool for that new domain will consist of the exact same people and double the deficit. Random, seemingly effortless examples may wow-trick one into thinking that it’s over. Doesn’t feel like it once you try it yourself. But when you succeed, the results are indeed impressive.

I have two thoughts:

One is that I love being able to effortlessly program in scripts & source languages I don’t know.

The LLM quickly creates drafts from my requests and assists in my debugging or polishing the code, based on the results as I apply it.

Two, this is the beginning. LLM’s will never be this incompetent again.

Any judgement on them that doesn’t acknowledge that we now have a runaway improvement situation, a best used by date for all limitations, isn’t complete.

Three decades is the longest, most pessimistic, time frame I see for us to be critiquing “their” code in any way. After that, it’s an interesting question if they will be critiquing our code or just trash canning it all.

We are living through one of just a few transformational moments in our universe. So far. Hard to fathom.

> After that, it’s an interesting question if they will be critiquing our code or just trash canning it all.

I feel like with many replacement technologies, the human condition leads to nothing being fully replaced, even assuming your runaway theory to be true. Ok, so there is super seemingly good code coming out of LLM in a decade or two but no human does understand it anymore and it will not be fully trusted. You will end up having the option to choose unproven, mystical, machine-generated programs or pain-stakingly handcrafted programs produced by "programming artisans" with a good reputation for 10-100x the price.

Furthermore, I feel like with some packages I have built, the documentation, the packaging and publishing and everything else around it took more time than the actual programming. In the future, there might even be mandatory certifications of it if the EU gets even more controlling. If the LLM doesn't get good at all of these tasks, it will only ever replace the small part of pure programming and as with Amdahl's law, it will at some point only yield diminishing returns in efficiency increase.

Good points. Formal verification could close the trust gap.

What happens when we are no longer capable of understand many of the machines' activities will be another matter entirely.

You're assuming that non-technical people have the ability to describe exactly what it is that they want. Until they can do that, or until LLMs can generate and gather requirements, we're very safe.

In fact, that's exactly what programming is. Learning the syntax of a programming language is easy, telling a computer precisely what needs to be done is hard.

The reason we have programming languages is not because you can't use more intuitive metaphors, "no code" programming is about as old as code itself. There are graphical tools, tools that attempt to replicate natural languages, etc... And yet, most programming tasks use code, because code is the best way we found for expressing ideas in a way that is sufficiently detailed so that computers can execute them without supervision.

So using ChatGPT to write code effectively is also programming, and without the mindset of a programmer, you won't get far. It may act as a force multiplier, making programmers more productive, like compilers and other non-AI tools, but I don't expect proficient programmers to be out a a job.

What it’s actually going to do is provide a new library of primitives like “audio to text” and “image classification” that will make formerly hard problems simple library usage. Then, since any beginner can glue together these things, we’ll become more ambitious in the problems we try to solve. So, we’ll never run out of work for programmers because the problems to be solved will grow more complex to absorb any raising of the floor.

I think it's more of a problem for established businesses? Nothing will be valuable when you can just say, "Give me a Google Maps clone", "Uber", "AirBNB" ?

So the equivalent of transfer learning. Neat, but I'd like to see it forced into a new paradigm or syntax. How would it handle Haskell or Clojure?

They do show training on HTML is still leading to improvements across languages. It looks to me like training on C led to the biggest average improvement among the tested languages. I'm curious if the effect translates to humans as well. I know that for myself learning C taught me a lot about what the higher languages are abstracting away.

It's a common recommendation, right? I think this might be close to being actual evidence for it.

The risk for permanent brain damage is too high. Although knowing C helps understanding why some horrible design elements exist in “modern” programming languages.

is it the programming languages themselves, or the programmer comments within the training sets that improve output?

There are comments in the data, but 95% of the comments in the HTML one seem to be variations of /*CSS styles go here*/.

> More surprisingly, we found that CodeM-HTML 7B trained on the HTML corpus can improve Java by an absolute 15.24% pass@1.

(Tongue in cheek reaction:)

Any sufficiently advanced technology is indistinguishable from magic — Arthur C. Clarke

Compared to 17% when pre-trained on Python. Looks like the debate on whether or not HTML is a programming language is settled!

Was the HTML corpus it trained on only HTML and text, or was there JS code in the HTML?

There sure was. Here is a snippet from the very first example in the html.json file:

  <input type="text" id="searchQuery" name="searchQuery" placeholder="Enter name...">
        <button onclick="searchEmployees()">Search</button>
    </div>
    
    <script>
        function searchEmployees() {
            var input, filter, table, tr, td, i, txtValue;
            ...etc...

There's no code in the repo, but the paper says this about how the inputs were generated:

For all the selected languages except HTML, we adopt an in-depth evolution, by asking GPT-3.5 to rewrite the seed instruction (Python) into a more complicated version relevant to the target language (Python, JavaScript, TypeScript, C, C++, Java, or Go). However, for HTML, we adopt in-breadth evolution to produce a brand-new HTML-related instruction...

Not even, you just need to comprehend the syntax and understand the caveats of a particular language.

Once a person has the fundamentals almost all languages are their oyster.

Wow, this is the first contradiction to Betteridge's law that I've encountered...

https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...

Sort of. None of the programming languages mentioned in the paper were actually boosted in any way, an LLM was.

Betteridge's Law can be trivially confuted. For example one could make a blog post titled "Did the Sun Raise Yesterday?" I don't dare trying the law with a headline like that about tomorrow though.

Since this is on arxiv one presumes it's not been peer reviewed yet, right? Is there a way to leave comments? There are a couple of confusing language/grammar mistakes including some in the first sentence.

Even the first sentence of the summary is unclear:

> When human programmers have mastered a programming language, it would be easier when they learn a new programming language.

I think they're saying "When human programmers have mastered a programming language, it makes it easier for them to learn a new programming language", but... I'm not entirely sure.

Yeah, that's what they're saying. English is not an easy language.

"It would be easier to learn a programming language, when you have mastered any other programming language" is clear, IMHO, but flipping the clauses makes it less clear.

> English is not an easy language.

"ChatGPT, please translate my incoherent stream of consciousness into clear English"

then review its output like one would a rough draft, and repeat as needed.

As far as human languages go, yes English is at least B-tier difficulty. Not least of which due to having the largest working vocabulary on Earth. But for all of the problems that LLMs might have with object permanence and memory and such, I think it's pretty safe to say that GPT3+ has mastered both the commonly used vocabulary and even the most subtle turns of grammar in the English Language. Which is never going to not dumbfound me. O_O

Email the first author.

If there's anything I've learnt in the past year about LLMs, it's this:

there's so much hype, ungrounded research "findings", and next-big-thing libraries (remember langchain?)

LLMs, at least the ones companies use, are APIs all the way down. Stay away from anything or anyone who claims they've discovered something no one else knew about LLMs.

Maybe staying away from someone selling something while making that claim, but staying curious seems appropriate.

> LLMs, at least the ones companies use, are APIs all the way down.

Really does seem like it. So many interesting little projects that end up being nothing more than a hidden prompt and an api call to OpenAI