Sven Slootweg @joepie91@pixie.town

I genuinely believe that the "technological advancements" of modern AI, i.e. Advanced Interpolation, are not impressive. like, a lot of people will say that "you gotta give it to them" because the technology is impressive, but it's not, and here's why.

note that I'm going to be heavily simplifying the mathematics of AI in a way that is close enough for explanation, but not entirely accurate. don't @ me about it.

modern AI accumulates the sum total of all human knowledge into several machine models, using the most advanced interpolation methods mathematics has available, to create something that barely works.

think of it this way: the reason why GPT and co. are treated like search engines is because search engines are really the predecessor to these models. Google and co. have done an incredible amount of work representing data in a generic way such that it can be looked up. however, there are a few key differences between Advanced Interpolation and search engines:

AI has access to private data that search engines will not show you. these models are trained on the private information of thousands of people, given unknowingly to companies who have been scratching their heads trying to monetise it. they include texts that must be purchased, images and videos that were never given to the public, etc. the reason why they're able to make a pretty good impression of what a person looks like is because they have millions of photos of people that were never published online, but were scooped off of people's phones anyway
for information that is untagged, AI leverages exploited labour to tag it instead. for example, you might not say what's in your photo album, but an underpaid, exploited worker can easily tag it as a person, maybe even what kind of person, after a second's glance
AI does not keep a copy of the original work. what I mean by this is that AI is no longer trying to find the specific reason why your terms match the result, and claims to give a result to your query. it does this by combining existing information in the best ways it can

and I say that these are the "most advanced" interpolation methods, but this is giving it a lot of credit. the mathematics behind AI, although obscured behind weird metaphors for biology ("neural networks," etc.) isn't particularly new. the new part is the absolute scale of the data involved. humans can only visualise data in three dimensions, not even four, but data inside an AI model might be anywhere from thousands to millions of dimensions. even if you just draw a line between all the points on a graph, a million-dimensional line will still be incomprehensible to a human, and might look genuinely impressive. except it's not just a line, but a polynomial in several orders of magnitude, maybe ten to a hundred. the mathematics of that aren't anything new, and the only new thing is choosing to allocate so much energy and resources into something so ridiculous

like, in addition to it matching the acronym, I call AI Advanced Interpolation because it's only that: connecting between points instead of going beyond them. it's been demonstrated that existing AI models are absolutely horrendous when asked to stray even slightly beyond their input parameters, and something you might not realise is that their input parameters are poorly defined.

like, the entirety of the internet has just been treated as training data for these models. we can convert all the text, images, audio, and video on the internet into these million-dimensional points and draw a hundredth-order polynomial between them, and you'll get pretty good results if you ask about the definition of a word, or ask to draw a picture of something people take pictures of every day.

but the like all interpolation methods, these methods fail ridiculously poorly when they're set beyond their initial boundaries. one example (which is only partially mitigated by the way AI works) is Runge's phenomenon, where adding more dimensions to data might make the quality worse, and even if you improve the interpolation between points, extrapolation is still pretty bad.

and like, sure, we do a lot of standard things every day, but there are also a lot of non-standard things we do. as an example, rearranging a deck of 52 cards, 52-factorial ways, means that even if everyone on earth shuffled a deck every morning, it would still be effectively impossible for anyone to get the same shuffle before the universe ends.

so, take all combinations of words and account for some grammatical similarities, and it's likely that several things said every single day have never been said before, ever. there's also a reason why every piece of art is unique; sure, there are plenty of similarities, but many pieces of art are made every day that have never been made before. all of these can be interpolated between the things the model has seen before, but the moment you add just a few too many things that it hasn't seen very often, it messes up. it can't even draw hands, for fuck's sake.

like, and what did it take to get here? an amount of CPU and GPU time far greater than that spent on basically anything useful in the entire history of computers' existence, for sure. (CPU time meaning time multiplied by the number of CPUs doing things; having two computers do the same thing at once means double the CPU time per actual time)

like, I'm not impressed. you've collected the sum total of human knowledge, exploited a bunch of people, guzzled water and other precious resources at a time when the planet is in grave danger, and for what? something that barely works? fuck you. abandon your shitty project and do some actual research for once.

"Film projector"? Oh you mean a framethrower.

a mastodon instance for delightfully tooting about cats