I hate seeing package download graphs like this because it tells me that people aren't fucking caching their CI runs:

when they were hiring for a rust infra position a few months ago one of their questions was how can we scale crates.io to support more downloads, since it keeps increasing every day, and my answer was you should try and push more people to cache downloads on their CI systems instead of just downloading every hecking time they need to test a commit

my answer was labelled "strong," instead of "great," so, I obviously was not as competent as anyone else who applied and didn't get the position

(I'm still salty about the "bias-free" hiring process which was very, very far from bias-free. although I'm sure the person who did get hired is actually doing a good job, I think that any claim of "bias-free" hiring is bullshit)

like I am certain that, while it doesn't come close to the wasted resources for crypto/ML bullshit, a very large percentage of wasted resources in the tech sector come from CI checks

like, some of them are things that are more difficult to optimize, like, actually making sure your code is power-efficient and optimized

but other ones are just because companies are more willing to pay for a higher CI tier than pay their employees to actually cache shit

you know what the best way to save resources is? not downloading every fucking thing every fucking time

@clarfonthey I only have data from npm but yeah, the vast, vast majority of npm downloads is from CI installations and I have repeatedly heard people talk about how caching on CI is 'difficult'.

@joepie91
@clarfonthey

Sometimes I take what we get in #JVM land for granted. We have 3 levels of caching: repository layer (jfrog/artifactory), build dependency cache (local m2 / .gradle caches), and build task caching (#gradle build cache).

But it goes to show: these are solved problems. Sometimes it feels like other ecosystems are too busy looking down on #Java instead of learning a thing or two.

@wakingrufus @clarfonthey I don't think this is really much to do with the 'ecosystems', to be honest; most package managers have caching functionality, for example, but that's not going to save you if the CI environment deliberately wipes the entire system on every run.