Forest @forestjohnson@pixie.town

@notplants they do both, they build software that doesn't silently fail, and they also work really hard to maximize their sender reputation.

I think those are two separate things, but its a lot easier to keep good rep if you can even know in the first place that your message was rejected!

@notplants yeah they definitely do a lot of that stuff. IMO that's a whole different problem/concern. I know Microsoft will never accept my emails because I'm not big enough to get on thier allow list... as George Carlin said, "it's a big club, and you aint in it."

I just want to be able to know if the email was immediately rejected or not. IMO its not too much to ask.

@notplants I believe things like this do exist, its just not "normalized" as a feature that all SMTP server implementations should have.

@notplants well I think email itself is practically sedimentary rock at this point, we can't change the protocols.

But I was proposing to just make a new thing on top, similar to how mailgun, sendgrid, etc did, just as a built-in feature of selfhostable SMTP servers in instead of a proprietary service only. Basically the same thing I already did except not based on tailng the logs :P

In my experience with SMTP for transactional email (logins, etc), email servers will reject the messages directly, they dont accept it and then send a bounce, or accept it and then black hole it. They might send it to the spam folder but there's not much we can do about that.

@technomancy @graydon I have soft spot in my heart for the NixOS and npm way, where each dependency gets to declare its own unique version of its own dependencies. So then you get like 36 different versions of the same dependency. Honestly, I would argue that saying you have to have only 1 single version of any given lib was a mistake :P

I work with JVM stuff at work a lot , and the way it's set up the libraries will automatically get upgraded quite often (version ranges). This has broken things a few times, every time its been version conflicts between two different deps that want different versions of some other library. I believe if you told my coworkers to pin to specific versions of every library they would tell you no. They would tell you "we don't have enough time to manually upgrade all those pinned versions every time there's a an automated CVE ticket". I guess a lot of businesses have found that it's easier to just update everything all the time than to hire people who can tell the difference between actual vulnerability and some bullshit CVE. Also, there's compliance rules that they have to abide by.

I think all this stuff is always going to be imperfect and messy. The more code you add, the worse it gets. I think that's kind of a universal truth.

@notplants I almost did this...

But I eventually kind of realized why they did what they did.

The problem is that SMTP submission ( as its implemented today) does not support Delivery Failures. The protocol simply doesn't have any place for them. So, if your email message gets rejected by an email server, then you will not be able to know that it happened.

That's why everyone started using a different protocol for submitting transactional email.

Especially for interactive systems like logins, it's crucial that the user can receive a warning when their email provider bounces the email.

For capsul we ended up implementing our own super janky version of this which was based on tailing the logs from SMTPD. https://git.cyberia.club/cyberia/smtpd-delivery-monitor

This is just another lump on the "email is fucking terrible and impossible to work with" Ball of mud. It's no surprise to me that a lot of companies have sprung up around trying to solve these issues and reduce the pain, damn the consequences and burn the old way of doing things.

It's also no surprise to me that the open source community generally has no interest in doing that.

in my opinion, we really should be talking about better email server software and better protocols for email submission. I think that's a prerequisite to Software like Ghost supporting non-commercial email providers.

@decentral1se Jes is on here !! @j3s

@Tak Yeah in the config, there's a bunch of rules that allow certain things thru, git clients being one of them.

@monsieuricon

Right now, I think that these bot deterrents are mostly just functioning similar to a "security thru obscurity" javascript blob.

I don't think the difficulty actually matters at all, you might as well set it to one because if scrapers ever try to solve the proof of work in the future, I think sha256 is categorically not going to work anymore since it's so easy to accelerate and so many accelerators for it already exist (bitcoin).

I actually created a proof of work bot deterrent before the LLM hype even existed. Back then I chose Scrypt as a memory-hard hash function because I wanted it to be as easy as possible for normal website visitors to solve, but as painful as possible for scrapers, even after they perceive it and react to it.

I don't have mine triggering on browser user agents. I just have it trigger all the time by default except for some tools that I allow list like git, npm, go, etc. I also explicitly allow home pages and repository home pages so that search indexers can still find things and display them.

You can see a demo of it here as well as the source code:

https://git.sequentialread.com/forest/pow-bot-deterrent-rp

@Tak I run gitea/forgejo aswell and I made my own, it works pretty well. The config isn't perfect yet but on the off chance you are using docker I do have a docker compose config example on there.

https://git.sequentialread.com/forest/pow-bot-deterrent-rp

Yeah its very cringe how the LLM scraper bots will try to download both zip and tarball of the repo at every single commit.