Forest @forestjohnson@pixie.town

@nolan

So, I have never used Sidekiq before but Queues of various kinds, distributed systems and their performance issues used to be my bread and butter at my old job.

From the aral post, I think the most important part is to listen to what the server admin is saying about how the problem occurs:

> (you have 23k followers, let’s assume 3k different servers), as soon as you create the post 3k Sidekiq jobs are created. At your current plan you have 12 Sidekiq threads, so to process 3k jobs it will take a while because it can only deal with 12 at a time.

Right away I am already seeing architectural red flags here. If 12 different other servers that I am trying to push updates to are all currently overwhelmed and timing out, it sounds like the entire process will grind to a halt. But there's reason for that; it's not the server who has a lot of push work to do can't do it all in time!!

The problem is that the logic that the server uses to push updates is highly flawed. First of all, an arbitrary limit of 12 in-flight requests max is pretty damn low. I would adjust that up to 1000 or higher. A raspberry pi can handle 1000s of concurrent http requests no problem. HOWEVER I suspect that "worker count" of 12 or whatever it is, is ridiculously low for some reason -- It sounds like each worker is its own OS thread, so spawning 1000s of them could do terrible things to the poor CPU, cause it to spend way too much time context-switching between OS threads.

This sounds like oldschool 1-thread-per-request concurrency. Still works fine in 2022 as long as the # of things happening at once is close to the # of processor cores you have. But when you start having (or wanting to have, for performance reasons) 1000s of things going on at once, you start looking at implementing asynchronous IO properly.

couroutines, goroutines, greenlets, async tasks, lightweight threads, fibers, actors, event loop, epoll, async io, they go by many different names depending on the language du jour. But they are all ways of having 1 single OS thread doing multiple things at once without triggering OS-level context switches.

Key takeaways so far:

--------------------------------------

The problem is that the code is using 90s concurrency technology

Buying a faster computer wont really fix the problem, although it might ameliorate it a bit

Properly fixing the problem involves refactoring the code so that it can do 1000s of things at once.

Note that doing 1000 things at once is totally normal for modern applications and it does not require a fast computer.

@Literally @joepie91 my point is, it only takes 1 person who is capable of giving up their free time in order to get yelled at.
People do the darndest things.

Honestly, I wish more people would yell at **me** lol

@ironiridis @reese 2/2

I've never really liked code using abbrevs or single letter names outside of tight loops even before I took that class.

And back when I used to work as a developer I would often run into comments that had become confusing or misleading as the code changed but folks forgot to update the comments

@ironiridis @reese 1/2 yeah I totally forgot about the part where he's dangerously obsessed with minimizing function size... Mostly I just remembered the naming stuff. Even if this guy is the worst person ever it doesn't disqualify him from having 1 or 2 good ideas. I had no idea about any of this, i just remembered the long names / explain with names instead of with comments concepts. I think they hold up 🤷

@reese @ironiridis

You answered it yourself, ETA is the word, right ?

You also said it has a unit, and the unit is minutes.

So I'd say the correct variable name should include the unit as well as what the unit represents. Not just what it is in terms of types of things (duration, timedelta, etc) but what it represents specifically:

etaMinutes
minutesUntilETA minutesUntilArrival
minutesUntilNextBus

At least that's what I was taught in school, that longer variable/function names like this are generally better. Textbook was https://picopublish.sequentialread.com/files/cleancode.pdf

I think I largely agree with that book to this day 🤓

@reese This is probably at least partially wrong information but I think it speaks a little bit to what monads are good for / why anyone would give a shit about them

@reese

I was introduced to monads thru C#.. C#'s iterator interface IEnumerable<T> is a monad. Maybe<T> is like an iterator that only iterates once, so instead of zero or more values, it can have zero or one value.

Just like how one might write

mysequence.map(x => x+1)

to add one to every number in a sequence (without having to know how many there are or having to load all of them in RAM before starting)

One can also write

maybething.map(x => x+1)

to add one to a number without having to know if it exists or not.

The "without having to know if it exists or not" part is the special magic... Not only does it help prevent null reference exceptions, it also allows languages like C# to build in declarative (SQL-like) features directly into the language. That's what LINQ (Language-Integrated Query) in C#.

Declarative meaning we don't tell it **how** to process the data, we just tell it what we want and it figures out the most efficient way to get there.

@reese

i dont know much about monads other than, other than my mom got one put some `Maybe<T>` and `Result<T, TError>` on there an I play em