Show newer

@angilly i appreciate you taking these concerns in stride. In the future though, I'd recommend having the public discussion *before* starting to collect any data

I have a separate instance that doesn't really federate, and it's been quite 'interesting' to monitor the logs for all these scrapers.

Also Friendica is very noisy for some reason? requesting a bunch of endpoints on discovery even tho you don't really federate with them

Show thread

@angilly @f0x text search that bypasses blocks (i.e. literally anything that scrapes fedi into a centralized data store) is an enabler for harassment and is therefore a massive red flag

just because the data is there and the use is permissible doesn't mean it's ethical

@angilly on my instances these requests get denied, yes. but just because data is public does not mean people consent to it being scraped, processed, and published elsewhere. If arbitrary full-text search was desireable on the fediverse, it would've been there.

It's not a technical challenge that needs solving, it's a social decision to do without it.

docs.joinmastodon.org/user/net
> It deliberately does not allow searching for arbitrary strings in the entire database, in order to reduce the risk of abuse by people searching for controversial terms to find people to dogpile.

blog.joinmastodon.org/2018/07/
> Mastodon deliberately does not support arbitrary search

Assorted list of Fediverse scrapers you might want to block on the network level (some more malicious than others)

git.pixie.town/f0x/nixos/src/b

another scraper by an (ex-?)twitter employee, running without recognizable user-agent (uses the typhoeus library with defaults)

creating some full-text search federation wide search... macaw.social/@angilly/10959740

Show thread

fediverse.network scrapes a whole range of endpoints, but the site it refers to is just a parked domain

Show thread

another scraper in my logs, "fedimapper" by hachyderm.io/@tedivm/109569119

not sure why it needs to scrape my domain_blocks with that stated purpose

@goat@hellsite.site going to the tavern to drown their sorrows

@dragon hey heavy metal deathtrap trying to avoid others is my band name, how did you know

@kim slacks off (it's a video meeting anyways) slack on

@dragon high on car-centric infrastructure and control of a metal deathmachine

maybe i should try writing happy poetry lol, but then it's easy to get sappy? can't win huh (and can't write, anyways)

Show thread

im completely sober but in a car ride so you all have to deal with my bad thoughts cause I have nothing better to do

Show older
Pixietown

Small server part of the pixie.town infrastructure. Registration is closed.