I need to write a blog post explaining how lockfiles are not actually needed for reproducibility and they were just added as a workaround to retrofit reproducibility on top of existing badly-behaved systems like rubygems, but then for some reason they hired the bundler guy to make the dependency manager for rust so he copied the same design despite it being an unnecessary complication in a greenfield system
@technomancy I thought lock files were also supposed to act as TOFU for dependencies so the file contents behind a version tag cant be modified after the fact
@forestjohnson you can use them for that too I guess; I'm only talking about how you resolve dependencies
I think there are potentially useful lockfile-like constructs you can layer on top of that once you have decided what dependencies to use; they just shouldn't be used to calculate the versions used
@forestjohnson @technomancy but they .. they aren't used to calculate the versions to use? I'm really genuinely curious what you think is wrong in this design and/or can be trivially avoided. like not "let's have a fight on the internet" curious, but "suppose I worked on both the system you're describing and will work again on subsequent ones and I really enjoy it when there's a hidden trivial answer to what I understand to be an actually hard and vexing problem, so could you please explain?" curious
@graydon @forestjohnson my experience working with Leiningen has led me to believe that specifying version ranges for dependencies are a mistake (Leiningen uses Maven and doesn't have to reinvent the wheel from scratch; technically speaking Maven's dependency logic does support ranges, but they are only used very rarely and are widely seen to be a mistake)
in my experience it's better to build out a whole tree where every node declares a dependency on one specific version of a library it needs, and then when different parts of the tree disagree about which version to use, the closest to the root wins; this is simpler and you get reproducibility for free
everyone who's used to lockfiles is like "doesn't that cause problems?" but I've been doing this since 2009 (and Java people have been doing it for even longer) and I am struggling to think of a single time it caused trouble
@technomancy @graydon I have soft spot in my heart for the NixOS and npm way, where each dependency gets to declare its own unique version of its own dependencies. So then you get like 36 different versions of the same dependency. Honestly, I would argue that saying you have to have only 1 single version of any given lib was a mistake :P
I work with JVM stuff at work a lot , and the way it's set up the libraries will automatically get upgraded quite often (version ranges). This has broken things a few times, every time its been version conflicts between two different deps that want different versions of some other library. I believe if you told my coworkers to pin to specific versions of every library they would tell you no. They would tell you "we don't have enough time to manually upgrade all those pinned versions every time there's a an automated CVE ticket". I guess a lot of businesses have found that it's easier to just update everything all the time than to hire people who can tell the difference between actual vulnerability and some bullshit CVE. Also, there's compliance rules that they have to abide by.
I think all this stuff is always going to be imperfect and messy. The more code you add, the worse it gets. I think that's kind of a universal truth.
