👏 more 👏 developers 👏 need 👏 to 👏 hear 👏 this
I can count on one hand the number of my clients over the past couple of years who haven't either over-architected for scale or were unnecessarily concerned about it (prior to coming to me for strategic advice, of course 😉).
You don't need to understand Distributional Little's Law to figure this out, it's obvious with primary school level math.
(screenshot excerpt from https://tailscale.com/blog/new-internet)
commentary/addition on blogpost
@ryantownsend I think their answer to "how did we get here" is missing a crucial bit of nuance: the influence of tech company blogs.
I've noticed a pretty clear pattern over the years where some tech company has to solve a problem at scale, blogs about how they've done it, and then there's a horde of developers who excitedly reads that blogpost and declares it the new best practice, elevating it to the level of "you are not a serious developer if you don't do this", often egged on by the tech company in question.
When prodded as to why, given that they are not a big tech company themselves, the answer has always been the same: a *belief*, not merely a hope, that they will be soon. Many of them outright reasoning along the lines of "if I do the same thing as this big tech company, I *will* grow as big as them and get rich, because clearly it worked for them".
This is, IMO, a much stronger effect than that article implies; this kind of technical writing actually drives entire hype cycles and steers entire developer communities in specific directions (as opposed to individual developers just being a bit optimistic), and I've seen it happen first-hand many times.
commentary/addition on blogpost
@joepie91 @ryantownsend on the other hand I've talked with an admin who very much prefers to deploy to kubernetes rather than bare metal, because k8s makes deploying and managing things and keeping services running much easier and more uniform. Yes, even if it's a single python script running on a single low powered machine.
commentary/addition on blogpost
@viq @ryantownsend I've seen this motivation quite a few times, and I strongly suspect it's similar to the whole thing with microservices and how it supposedly allows for decoupling code, when that benefit actually comes from code modularization and that's only a tangential side effect of microservices. It's not that microservices don't *also* have this benefit, it's that you could have had that same benefit without all the complexity of microservices.
Same thing with deploying to kubernetes; the actual benefit here is a consistent deployment environment (which is a valid thing to want!). You don't *need* an ultra-scalable system (with all of the associated complexity) for that, you just need a consistent environment, and there are many other options for that that aren't kubernetes-shaped, often with a significantly lower complexity cost.
commentary/addition on blogpost
@joepie91 @ryantownsend I don't think I went exploring that space before, could you please name some for my collection of tools?
commentary/addition on blogpost
@viq @ryantownsend Two options that immediately come to mind, with different philosophies behind them, would be NixOS and Dokku. Or, if you're willing to compromise a bit on environment consistency, the many deployment tools like Chef and Puppet (their main problem is that they can only *add* to the environment).
Dokku is essentially a single-machine self-hostable Heroku-like environment, that AIUI provides a consistent environment much like Heroku does. I'm personally not a fan of the Heroku model so I haven't used it, but from what I've heard from others, it works pretty much exactly as promised.
NixOS is a declarative Linux distribution that works kind of like a deployment system like Chef/Puppet, but implemented as a local package/system manager first and foremost, and with much stronger guarantees about environment consistency/determinism (also stronger than containers).
It can pretty trivially be extended to multiple systems with third-party tools if wanted, but it's system-local in its core; using it across systems basically just entails pre-building the other systems (which is just a build product, from its perspective) and pushing your build to them.
NixOS' documentation still needs a lot of work so its apparent complexity is higher than what's strictly needed to work with it, and Dokku is probably more immediately accessible, but I bring it up because it is a good example of what *can* be achieved without 'hyperscale' systems today.
commentary/addition on blogpost
@viq (Of course there are also many PaaS options, as @ryantownsend alluded to, but as those generally come with some form of vendor lock-in, I did not include them here)
commentary/addition on blogpost
@joepie91 @viq Heroku/Render don’t really have vendor lock-in as they use pretty standard open source tech like Postgres/Redis and their VMs just run whatever code you deploy, nothing fancy there either.
In a commercial environment, it doesn’t really make sense to run this yourself unless you have specialised infrastructure requirements – PaaS have entire teams dedicated to uptime/security etc, accessing that for as little as $50/mo is unachievable in-house.
commentary/addition on blogpost
@ryantownsend @viq Over the years, I've had a peek behind the curtains of a lot of infrastructure/service/security providers, in various capacities and to various extents, and I honestly can't say that I share that reasoning.
I'm certainly not saying that there are no competent people working at such companies, but there's often a very big cliff between "the security/reliability posture that is advertised or implied" and the "the posture that the provider actually has", frequently because of overworked infra/security teams or micromanagers meddling and not letting them do what's needed.
There are probably specific providers which have competent teams (though even that only gets you so far, at a certain scale it becomes unmanageable). But I think that outsourcing it to a competent-*appearing* provider and assuming that takes care of your security/reliability, is a very dangerous thing to be doing. You're mostly just paying for a security blanket at that point - which of course will be cheaper than the real thing.
Basically: if your company is dependent on IT infrastructure, you *must* have someone in your company who understands that infrastructure, its weaknesses, and who has both the ability and access to recover from its failures. Whether you outsource things to a third-party provider or not. And by that point, running a standard Linux server is not a very tall order either.
