I regret to inform you, wider fediverse, that authorized fetch has defeated every scraper we've seen to date.

Follow

@t54r4n1 it really hasn't? there's been recent developments in scrapers using client api's on other servers, making them impossible to block without blockimg that entire server. Also Mastodon 4.x exposes a lot of the client api usable for scraping by default now, even with authorized fetch

@f0x oh, also blocking servers with bad actors. but like, that is standard,
but I hear you.

@f0x @t54r4n1 there's a second flag that disables most of the public 4.0 client API at the cost of breaking your logged-out web frontend with no end-user-facing explanation.

but the federated timeline streaming APIs that scrapers are targeting now long predates 4.0, is public by default, and the bigger servers are never going to turn it off because they like their little public tickers for showing off all the instances they're connected to.

private Fedi for grownups when 😩

Sign in to participate in the conversation
Pixietown

Small server part of the pixie.town infrastructure. Registration is closed.