@polychromata Mmm. There's a bit of important context there - that page is *very* old, and it was written under very different circumstances; when the main problem was shitty companies shutting down and deleting people's stuff with very little notice, frequently using robots.txt as a way to hoard data and prevent others from getting at it (much like companies are now doing with weird AI scraping company access deals).

So that page was written with malicious corporate robots.txt files in mind, not personal sites let alone stuff like fedi instances, which were pretty much entirely irrelevant to ArchiveTeam at the time of writing.

@polychromata (That's not to say that there aren't sometimes people who get involved in Archive Team with dubious ideas about morals, but that's basically unconnected to that particular article or the policy behind it)

@polychromata Or to phrase it differently: the motivation behind that policy was "fuck companies who keep individuals from saving their own stuff from mass deletion", not "fuck people who try to keep others from taking their stuff"