Reminder that the "innovation" in LLMs is really just the willingness of techbros to do labour exploitation, not an advancement in technology
"So why didn't LLMs happen sooner?"
Because for it to happen, two things needed to be true:
1. Someone needs to be sufficiently unscrupulous to exploit labour to this degree
2. They also need to have a big enough bag of money to actually make it happen
... and it takes a while for both of those conditions to occur together, and the result was "OpenAI"
@joepie91 I think at least two more things had to be in place: one is access to massively parallel matrix operations, so modern GPUs, which is something that happened independently. But there were also some relevant advances in NN architectures, like the self-attention mechanism.