Linux distros just going and disabling critical security features like the Go Checksum Database seems like a regular occurrence. It’s unclear to me whether there’s any Linux community that I can identify with enough to run their packages.
I understand that point of view, sort of that google is "too big to fail" and so they would never dare do something like that... but IDK.. I think at least if they are gonna do that they shouldn't hide it and mislead people about it
@forestjohnson the point is that the team went out of their way to build a transparency log to make it impossible to hide misbehavior, which no other centralized language package ecosystem (all of them) did. Calling that telemetry, and a confusing error message misleading (when there’s plenty of docs about the whole thing) is… a choice.
@filippo I use the word "misleading" because pretty much every golang developer I run into (including myself before this issue thread) has no idea that this feature exists, and their mental model of what go get is doing is wrong.
@forestjohnson that person doesn’t read much wrong to me? Google is not in control of the namespace, the checksum database is an append only transparent log that anyone can audit. We could always write more docs, but why do you feel misled? By whom? For what purpose?
This is a really good doc that explains how it works,
https://go.dev/ref/mod#checksum-database
> The checksum database is served by sum.golang.org, which is run by Google.
TBH I didn't know how it worked until I read this page just now, its really clever how it's designed, it sounds like the `go get` command really does try to avoid just blindly trusting whatever the proxy says, and while it does trust whatever the sumdb says, since the sumdb is a merkle tree and all responses are signed by the server's key, its highly tamper-proof.
> the first step is to retrieve the record data through the /lookup endpoint. If the module version is not yet recorded in the log, the checksum database will try to fetch it from the origin server before replying. This /lookup data provides the sum for this module version as well as its position in the log, which informs the client of which tiles should be fetched to perform proofs. The go command performs “inclusion” proofs (that a specific record exists in the log) and “consistency” proofs (that the tree hasn’t been tampered with) before adding new go.sum lines to the main module’s go.sum file.
@filippo You are right that considering how much effort went into making this system hard to abuse, it's kinda insane that people like me will still complain about it, sr.ht will block it, and fedora will disable it by default.
I think a lot of that is simply because of
> "if it looks like a duck, if it quacks like a duck...."
No one has enough time in their day to actually research and understand this super-complicated system, they simply find out one day that `go get` is not connecting to the server that they thought it was connecting to, and they treat it just like anything else which has unexpected "seemingly secret" behaviour:
They have a knee-jerk reaction and block it.
IMO this whole misunderstanding could be avoided if the `go get` command would print a log message about its behaviour that includes a URL to an ELI5 version of the https://go.dev/ref/mod#checksum-database doc section.
I would be willing to write that doc if there was a chance it would be published that way.
