Linux distros just going and disabling critical security features like the Go Checksum Database seems like a regular occurrence. It’s unclear to me whether there’s any Linux community that I can identify with enough to run their packages.
I understand that point of view, sort of that google is "too big to fail" and so they would never dare do something like that... but IDK.. I think at least if they are gonna do that they shouldn't hide it and mislead people about it
@filippo I use the word "misleading" because pretty much every golang developer I run into (including myself before this issue thread) has no idea that this feature exists, and their mental model of what go get is doing is wrong.
@forestjohnson that person doesn’t read much wrong to me? Google is not in control of the namespace, the checksum database is an append only transparent log that anyone can audit. We could always write more docs, but why do you feel misled? By whom? For what purpose?
This is a really good doc that explains how it works,
https://go.dev/ref/mod#checksum-database
> The checksum database is served by sum.golang.org, which is run by Google.
TBH I didn't know how it worked until I read this page just now, its really clever how it's designed, it sounds like the `go get` command really does try to avoid just blindly trusting whatever the proxy says, and while it does trust whatever the sumdb says, since the sumdb is a merkle tree and all responses are signed by the server's key, its highly tamper-proof.
> the first step is to retrieve the record data through the /lookup endpoint. If the module version is not yet recorded in the log, the checksum database will try to fetch it from the origin server before replying. This /lookup data provides the sum for this module version as well as its position in the log, which informs the client of which tiles should be fetched to perform proofs. The go command performs “inclusion” proofs (that a specific record exists in the log) and “consistency” proofs (that the tree hasn’t been tampered with) before adding new go.sum lines to the main module’s go.sum file.
@filippo You are right that considering how much effort went into making this system hard to abuse, it's kinda insane that people like me will still complain about it, sr.ht will block it, and fedora will disable it by default.
I think a lot of that is simply because of
> "if it looks like a duck, if it quacks like a duck...."
No one has enough time in their day to actually research and understand this super-complicated system, they simply find out one day that `go get` is not connecting to the server that they thought it was connecting to, and they treat it just like anything else which has unexpected "seemingly secret" behaviour:
They have a knee-jerk reaction and block it.
IMO this whole misunderstanding could be avoided if the `go get` command would print a log message about its behaviour that includes a URL to an ELI5 version of the https://go.dev/ref/mod#checksum-database doc section.
I would be willing to write that doc if there was a chance it would be published that way.
> when there’s plenty of docs about the whole thing
Aka "RTFM", doesn't sound great here.
I _**know**_ it's misleading directly from my lived experience, and from hearing from 90+% of go developers who had exactly the same incorrect mental model of `go get`.
Whether it talks about this in some manual page somewhere or not doesn't really matter; no one is going to read that until after they discover that `go get` isn't doing what they expected.
The affordances and apparent behavior of the tool are the only way to "explain" this to users so they know what they're getting.
@filippo Like, even if it just printed a log by default
> Now I am connecting to the default go package proxy `proxy.golang.org`
> Now I am checking this hash `1a2b3c4d5e6f......` against the go sum db at `sum.golang.org`
That would address what I'm complaining about.
Even better, it would overwrite the lines in your `go.mod` , i.e.,
from
```
require (
git.sequentialread.com/forest/config-lite 164dc71bce04989dc5ffbbfd5769a689230f126a
)
```
to
```
require (
proxy.golang.org/git.sequentialread.com/forest/config-lite 164dc71bce04989dc5ffbbfd5769a689230f126a
)
```
> hearing from 90+% of go developers
Sorry, this was worded poorly, what I meant was, everyone I meet who uses go tends to fall into one of three camps:
25%: Knows about go proxy and go sumdb because they read about it on a social media posts like this one
70%: Thinks that `go get` directly connects to the server domain listed in `go.mod`
5%: Knows about go proxy and go sumdb because it broke their build
I have yet to meet anyone who learned about it by reading the documentation.
@forestjohnson the point is that the team went out of their way to build a transparency log to make it impossible to hide misbehavior, which no other centralized language package ecosystem (all of them) did. Calling that telemetry, and a confusing error message misleading (when there’s plenty of docs about the whole thing) is… a choice.