Why didn't we get GPT-2 in 2005?

also the internet wasn't as robust so less training data ofc

>There was no grand plan that led to LLMs being created. People just did a bunch of things step-by-step. With so much uncertainty, progress is more “little hops” rather than grand journeys—technology is more “evolved” and less “designed”.

This pretty much sums up the whole article, and it makes perfect sense.

So much was going to happen anyway that you really do have to look away from the technology to answer why.

I would say GPTs didn't come earlier because sama didn't have enough high-dollar contacts yet who would extend their confidence to him.

And then what you get is little hops done big because that's whats already familiar to the big money owners.

So LLMs are first out of the gate to reach impressive capabilities, but something completely unfamiliar that may be superior due to purposeful design including fundamental principles will have to lag behind until a very big change would become possible.