wish that someone would fork ghost to make it support sending e-mails not just through mailgun
or someone would fork keila and make a version that doesnt have any language about marketing campaigns
related great blog post: https://blog.kjamistan.com/building-a-privacy-first-newsletter.html
@notplants I almost did this...
But I eventually kind of realized why they did what they did.
The problem is that SMTP submission ( as its implemented today) does not support Delivery Failures. The protocol simply doesn't have any place for them. So, if your email message gets rejected by an email server, then you will not be able to know that it happened.
That's why everyone started using a different protocol for submitting transactional email.
Especially for interactive systems like logins, it's crucial that the user can receive a warning when their email provider bounces the email.
For capsul we ended up implementing our own super janky version of this which was based on tailing the logs from SMTPD. https://git.cyberia.club/cyberia/smtpd-delivery-monitor
This is just another lump on the "email is fucking terrible and impossible to work with" Ball of mud. It's no surprise to me that a lot of companies have sprung up around trying to solve these issues and reduce the pain, damn the consequences and burn the old way of doing things.
It's also no surprise to me that the open source community generally has no interest in doing that.
in my opinion, we really should be talking about better email server software and better protocols for email submission. I think that's a prerequisite to Software like Ghost supporting non-commercial email providers.
what you are describing about SMTP not supporting delivery failures seems like one annoying piece of the e-mail protocol but, like you said, also not the only one?
as im reading more about this, im seeing that different e-mail providers have different bounce messages that would need to be handled, outside of the smtp protocol
but even beyond handling all these bounce formats, and beyond trackable "delivery failures" theres also email silently being sent to spam
in this thread https://sunbeam.city/@dmd@mastodon.social/113669758874048161 , I also was having the feeling that actually a deeper problem in email is the approach to spam itself -- anyone being able to send anyone an email with zero cost to making a new server is actually not a viable system because it would be too easy to spam
so I guess all im proposing here, is that if were going to go about revising the email protocol, in addition to adding delivery confirmation, maybe we also need something about spam built into the protocol too?
@forestjohnson
as it stands, it feels to me like there would be no easy purely technical solution to email with the current protocols -- but I still wish ghost would support more different email sending providers than just mailgun
@notplants well I think email itself is practically sedimentary rock at this point, we can't change the protocols.
But I was proposing to just make a new thing on top, similar to how mailgun, sendgrid, etc did, just as a built-in feature of selfhostable SMTP servers in instead of a proprietary service only. Basically the same thing I already did except not based on tailng the logs :P
In my experience with SMTP for transactional email (logins, etc), email servers will reject the messages directly, they dont accept it and then send a bounce, or accept it and then black hole it. They might send it to the spam folder but there's not much we can do about that.
@notplants I believe things like this do exist, its just not "normalized" as a feature that all SMTP server implementations should have.
@notplants yeah they definitely do a lot of that stuff. IMO that's a whole different problem/concern. I know Microsoft will never accept my emails because I'm not big enough to get on thier allow list... as George Carlin said, "it's a big club, and you aint in it."
I just want to be able to know if the email was immediately rejected or not. IMO its not too much to ask.
@notplants they do both, they build software that doesn't silently fail, and they also work really hard to maximize their sender reputation.
I think those are two separate things, but its a lot easier to keep good rep if you can even know in the first place that your message was rejected!
@forestjohnson fair enough. yes would be much easier to keep good rep if you had more clarity over what arrived.
i want my emails to get there and not go to spam...
and short of re-writing the email protocol, I would be happy to pay an organization to handle the relations and reputation management which seem necessary to me
i just dont want to pay mailgun, would rather pay a big tech coop
e.g. here is @mayfirst sharing their email deliverability rates for the last week https://comment.mayfirst.org/t/e-mail-newsletter-via-mayfirst/2745/2?u=mfowler
i wonder what they have on the technical side for confirming delivery
@notplants @forestjohnson @mayfirst May First doesn't have any way of confirming email deliveries, but we send a bounce to the envelope from address if the message is not sent. Most bulk email programs support VERP style messages, so every message is sent with a unique envelope from address that encodes the actual recipient (e.g. bounces+jamie=jamiedomain.org@senderdomain.org). Your program can check this mailbox and keep track of bounces, and by inverse, successful deliveries.
I had considered that option, but I decided against it because I want to be able to tell the difference between time out and a success.
I'm blocking the ui thread until I know whether the email was accepted or rejected. So I can't just wait 30 seconds for a bounce notif and then assume success. No one would use this site if you have to wait 30s every time you log in
@forestjohnson I guess my questions is whether mailgun, sendgrid etc. just built a new thing on top
or are they actually also doing human relationship management with the big email providers, to stay in good standing, and keep their sender IPs whitelisted
this being necessary, as a result of the sedimentary rock that is there
maybe talking to someone who actually works at one of these companies would be needed to know?