In my previous post about software bloat I ended on a somewhat unsatisfying note.
Is there an actual solution?
I have no clue. Refuse to write and use shit software, I guess.
However, we might be seeing another, much brighter light at the end of this tunnel unfold before our very eyes: Twitter!
Of all places, I had never expected a code-bloat reduction revolution to come from silicon valley. Especially not a web company. All it took was a controversial acquisition from the Elon Musk of our time: Elon Musk.
If you haven’t been keeping up with these news, Elon is trying to do a few things:
The latter requires not having thousands of unnecessary engineers on payroll, like they do now, even after his initial downscaling. Doing that requires simplifying the codebase massively, because with a bloated codebase, you need the bloated workforce that created it. Untangling this mess is incredibly difficult, because nobody knows what’s going on in the codebase. Nobody knows what anything does other than their in their exact department. Twitter is a many-million-line behemoth and it doesn’t need to be.
Luckily, Elon seems to be conversing with actually skilled engineers about these problems, such as George Hotz and Jonathan Blow.
There is a natural crossing between the code reduction they’re doing, and the goal to bring back free speech. They seek to undo the huge mess of automated censorship and suppression models/algorithms that exist in Twitter’s codebase. For more about this, you can read Dave Rubin’s thread from when he had visited Elon and the Twitter gang. [Nitter link] It’s worth the read, because it is interesting for a whole bunch of reasons other than software bloat.
I asked Elon what I could share and he said, “anything that’s true.”
— @RubinReport, 6:51 PM · Jan 26, 2023
Basically, over the last couple of weeks they had gotten a vague sense that something was wrong. The numbers just didn’t seem quite right. That’s how complex this system is now, that things were good for a bit, and now they’re not, and nobody knows why. Dave Ruben came over to talk with them about this, because his account was affected by this downturn in numbers. Then, when digging around in the codebase after multiple weeks of day-and-night work by engineers at the top of their game, they discovered additional ways that the system was suppressing people on the platform. They didn’t know this was happening. They discovered it.
Accounts aren’t just hit with labels that are obvious to insiders. They now found more “secret” labels which are causing shadowbans. My account was hit with all three; “Recent abuse strike,” “Recent misinformation strike”, “Recent suspension strike.”
— @RubinReport, 7:03 PM · Jan 26, 2023
It’s unclear so far what these strikes actually do, but for sure they suppress views and recommendations, they are trying to figure out to what extent.
— @RubinReport, 7:06 PM · Jan 26, 2023
Do you have any idea how much of a problem it is when you dig around working ‘round the clock for multiple weeks to suddenly discover unwanted functionality in your codebase? It’s a billion line dumpster fire. Elon describes Twitter as “a fractal Rube Goldberg machine”. For every problem they solve, they discover more problems. And I’m sure he’s right. It’s what I’ve come to expect from big software companies at this point.
So, about that unsatisfying end note. This one will be more satisfying.
If Elon succeeds in his mission here, we will have the first big public example of how to do code cleanup at a massive scale. I really hope he does, or at least gets the total rewrite he’s talking about. And I really hope he survives the stress of seeing it through, so he can share what he learned along the way. Getting to mars, solving AI alignment and all that is nothing compared to trying to un-need 20 million lines of Scala.
Ok, marginally more satisfying at least.