Matrix homeserver development be like
* 10 minutes of writing code
* 2 hours of trying to work out a spec omission and asking people
* 5 minutes of writing code
* 30 minutes of filing spec issues
* 15 minutes of writing code
* two days of waiting for clarification on something ambiguous in the spec
I like the premise of the protocol, but sheesh, the spec has some serious quality issues, and it's making it almost impossible to get anything done on my homeserver project without losing focus, because the whole thing constantly blocks on spec holes
Can we get an experienced IETF RFC writer on this or something?
@joepie91 tbh most of the ietf rfc specs I read are the most unreadable things I have ever seen. However I fully agree that matrix spec is not readable. Imho I would go as far as saying that the new design made it somewhat worse too.
@MTRNord The IETF RFCs are often difficult to read, but there's generally one thing you can just assume: that it will contain the answer to your question. There's a reliable source you can fall back on.
The Matrix spec is written more like a tutorial, which is superficially more readable, but in practice ~all the information about the corner cases and procedures is missing.
@joepie91 @MTRNord I'm sorta in the camp that the tutorial text should also appear, and then be backed by a very clear strict spec. I think right now we have a mix of tutorial & spec, with a few holes around the place.
I generally feel the spec is uneven due to the approach if it being written over time by different authors, and there is value in actually going through and finding all the strange bits.
@halfy @MTRNord I would say there's rather more than "a few holes". Like, this isn't a small problem - the spec as it currently stands is essentially unimplementable, at least on the S2S side.
After reading spec for over an hour and talking to several others, the conclusion seems to be that the entire process of "how do you deal with events referencing a branch/event you don't know about" is just *entirely* undefined. Completely.
This is a crucial part of server-to-server semantics, and ensuring that different implementations remain interopable. And it's just... not there? At all?
Part of my frustration is precisely that the issues with the spec are often framed as "we just need to find the issues and fix them", but that isn't the problem at all, and the real scope is much bigger.
There are structural problems with the way the spec is written, regardless of writing styles; there is no IETF-style rigor, edgecases are constantly omitted, information is spread over different sections in strange ways, and so on.
These are not issues that you can fix piecemeal as you run across them (I've been doing so for years now and there are just too many - I would estimate thousands). They require a fundamentally different approach to writing a spec to fix.
And most importantly, if we ever want to end up with a spec that fixes issues faster than it creates new ones, we're gonna need real-time engagement between "people who know the spec" and "people who find the issues". It needs to take a minute to get a clarification, not an hour or a day.
Given that the spec is essentially still in the same place that it was 5 years ago on this point, I don't have much remaining hope that the current approach will ever solve the problem. It's clearly not working.
@halfy @MTRNord To further illustrate the point: I've been trying to work on a homeserver implementation for several years now.
I've gotten essentially nowhere, and this is almost *entirely* because of the problem described in the original toot, where I'm having to spend almost all of my time and energy on sorting out spec issues.
If it is essentially this impossible to write a compliant implementation, can it even still be credibly called an open protocol?
@MTRNord @halfy And to be clear, I don't actually even *like* reading IETF-style specs.
So if there's *that* much difference in time and energy requirements to get a working implementation, that is not a good sign for a supposedly 'open' protocol.
Is it really 'open' if you effectively need a well-funded engineering team to get anywhere within any reasonable timeframe?
@joepie91 @halfy Well. Others like conduit and construct were able to write a working matrix server which starts to make me wonder what they are doing different. I still fully agree with spec needing WAY more work and probably a rewrite which starts at "what do we have" and "what is missing" and ends at "did we tick all checkboxes before we replace current spec text" (because there is a habit of loosing information as well).