I have been working on this blog post for over a year. At a talk at Accelarate 2019 yesterday, Accela CTO Renato Mascardo put into different words what I had been thinking about. He called it the ‘Tech Stack Explosion’. My original blog post title was ‘Rolling Up’, as I had wanted to write about how each new generation of software was ‘rolling up’ software from the previous generation, commodotizing each piece from the previous. And that’s still happening. But the rate at which is happening appears to be increasing.
When I first started programming in the 1980s, an application was largely a single stand-alone unit. It might have been a database, a word processor, or some specialized application of similar size and complexity. In the 1990s, applications became larger. They often contained other applications. A multimedia application might ship with an embedded database, for example. The next decade saw (and these are REALLY rough demarcations) integration with other applications in a different way. An application might interface with existing installed applications or might require the deployment of multiple components. Nowadays we have a mix of the two: many of us are writing microservices, lambdas and other small applications that both embed other sophisticated pieces of software or interface with them. I have built applications with fewer than 500 lines of code that embed full object-relational mapping libraries that talk to databases, other APIs and end users.
Mascardo spoke about how a solution he helped build required 20 microservices and had 50 external dependencies. Those external dependencies can provide a tremendous amount of value for customers. But they also introduce considerable risk. And your customers don’t care that those externals are not under your control. They are still part of your value proposition, and you will get blamed if they do not work.
So how do we handle these exploding tech stacks? First, we need bigger teams, and much more structure within those teams to deliver on those solutions. New roles are being created. The new risk that is introduced from so many moving parts has to be managed, and that takes a lot of overhead. We need to build more sophisticated interfaces, adapters and substitutions. QA becomes more difficult, and testing all combinations of our software has become even less feasible, so we as software developers need to have a better understanding of QA principles than we have in the past if we are going to build quality software. Welcome to 2019.
What will the next ten years look like? As always, it’s very difficult to say. But I can’t wait to find out. Maybe I’ll give it some thought and make another blog article… in a year or so.