My search engine crawler so far seems to be perfectly fine with about 20MB of heap size (ie. "true" memory use), though it doesn't do the actual content indexing yet. Promising results so far though!
At first glance, I seem to be getting a roughly 4% robots.txt rejection rate on a pile of personal websites, which is honestly lower than I had expected
@arianvp Yeeeep. I've had much the same experience with Google's JS libraries.
idle thought maybe hot take
The whole thing with "official fan $thing" and the paradox therein, seems like it's really just a version of the "temporarily embarrassed millionaire" phenomenon, where someone is trying to appear grassroots and independent, but really their actual hope in doing so is to become the Big Guy, not for legitimate grassroots reasons
@arianvp That's one of the industries it happens for, yeah, the thing with the infosec industry is that they're extremely easy to buy goodwill from, you just hire a couple security folks who are well-liked in the community, have them write a couple really interesting and cool blogposts with knowledge that's hard to find in a concise format elsewhere, and bam, you've bought the market.
See also: Cloudflare, Tailscale, ...
(There's some other industries where this happens, DevOps is another one, but it's why this particular predatory model happens so often in specific industries)
Disentangling colonial thought patterns, whiteness is a death cult (reading links on extinction, genocide, colonialism)
When you wish extinction on *everyone*, you're wishing violence on the people we forced to be complicit in our sins; through slavery, through colonialism, through domination and subjugation, through abusively wielding global power.
The human race is not the problem. White people and white supremacy are the problem.
bitwarden, enshittification, etc., personal frustration (2)
So like, consider this your early warning for Tailscale and all the other currently-hip "open-source-friendly" things that are rapidly buying goodwill with neat technical blogposts. We'll see where they are in a year or five.
(And this one isn't exactly an 'early' warning anymore, but Microsoft "embracing open-source" is absolutely one of these as well.)
bitwarden, enshittification, etc., personal frustration
And once again, a "FOSS" thing that people *assured* me was legitimate FOSS when I refused to use it because of its commercial VC-y setup... is starting to lock down their project after increased VC funding.
How many times does this need to happen before people stop trying to argue about this shit with me and just accept that there is a very good reason I will not touch anything that looks VC-y, no matter how "we're FOSS, honest, promise" it claims to be?
When will people learn that corporate FOSS with an unclear business model *will* end up backstabbing them, and it's just a matter of 'when', not a matter of 'if'?
"But this one is different!" No it's fucking not, and I fucking *told* you that this would happen and why, but you didn't want to listen did you?
#AskFedi: What would the most sensible way to implement robots.txt for a search engine project? The intended behaviour is to be reasonably conservative; ideally it should tolerate some weirdness in the robots.txt and default to "do not crawl" in cases of doubt as to the author's intention, but still have good coverage of sites that clearly did not intend to block crawlers (and default to "crawl" if there is no robots.txt or equivalent at all).
(Asking because there's not one robots.txt standard, and perhaps people here have preferences on what variant is the best choice to support here)
@freakazoid Actually, a better example: the common implementation of "meritocracy" in FOSS is pretty much exactly what you describe, and it basically always results in a toxic environment. Likewise, meritocracy has a kernel of truth to it, but it can only work as part of a bigger change.
@freakazoid I feel like that's overly optimistic, honestly. It's not entirely *wrong* - removing money does remove an avenue of power accumulation. But... it is not the only one, and unless you also get rid of the underlying power-seeking culture, people will find another way to accumulate and misuse power, most likely reinventing money, or religion, or something like it.
Getting rid of money can definitely *help*, in that it removes a pillar of power, and so weakens the overall system of hierarchy, and makes it easier to knock it over. But you do still need to fight the concept of 'deserved' hierarchy itself to actually get there!
(You can see this happening at a small scale with the many local initiatives that try to exist 'without money', but then do not recognize or address power relations otherwise, and ultimately end up succumbing to some power struggle from out of left field)
In the process of moving to @joepie91. This account will stay active for the foreseeable future! But please also follow the other one.
Technical debt collector and general hype-hater. Early 30s, non-binary, ND, poly, relationship anarchist, generally queer.
- No alt text (request) = no boost.
- Boosts OK for all boostable posts.
- DMs are open.
- Flirting welcome, but be explicit if you want something out of it!
- The devil doesn't need an advocate; no combative arguing in my mentions.
Sometimes horny on main (behind CW), very much into kink (bondage, freeuse, CNC, and other stuff), and believe it or not, very much a submissive bottom :p
My spoons are limited, so I may not always have the energy to respond to messages.
Strong views about abolishing oppression, hierarchy, agency, and self-governance - but I also trust people by default and give them room to grow, unless they give me reason not to. That all also applies to technology and how it's built.