Reminder that the "innovation" in LLMs is really just the willingness of techbros to do labour exploitation, not an advancement in technology
@joepie91 I think at least two more things had to be in place: one is access to massively parallel matrix operations, so modern GPUs, which is something that happened independently. But there were also some relevant advances in NN architectures, like the self-attention mechanism.
"So why didn't LLMs happen sooner?"
Because for it to happen, two things needed to be true:
1. Someone needs to be sufficiently unscrupulous to exploit labour to this degree
2. They also need to have a big enough bag of money to actually make it happen
... and it takes a while for both of those conditions to occur together, and the result was "OpenAI"