The Rise of Platform Engineering
The Rise of Platform Engineering
Developer Efficiency - The Best Thing Since Sliced Bread
It all started with a Stack Overflow Survey in a galaxy far far away…
Star Wars aficionado here in case you didn’t know.
Anyway the chart above is what I’m referring to. This was the answer to a Stack Overflow question of what processes and tooling did organizations have in place. The results are astonishing to me. Anna Heim wrote about this in the Techcrunch post below and I wanted to expand upon it.
What’s shocking is how far we’ve come in terms of tooling, content, best practices shared, advisors and yet still only 37% (~1/3) of 35k developers have basic tooling like observability in place. Without the basics in place, imagine how hard it is for teams to move to shipping product more frequently, understanding when failure may occur / is occurring, or proactively testing how systems may work together before moving to production. These are things that we take for given in the tech only environment. However, this survey, gives us all a peek into the “real world”, where developers across other industries are still only just getting the basics in place.
This is what customer empathy for companies serving enterprises needs to look like. As participants in the tech industry, we need to understand the challenges that a customer may be going through. This is also a reason that founders who live the pain themselves (either directly by working at an enterprise or at a vendor who serves those enterprises) are the best builders of products in the enterprise software space. Until you’ve seen how an enterprise deals with managing legacy systems while trying to leverage modern technology, it’s very hard to build a product that solves that pain.
Take banks for instance. They leverage lots of modern technology and in many cases are the biggest customers (CapitalOne spends ~$49M per year on Snowflake). At the same time they still have COBOL developer roles open to manage their mainframes. So now imagine you’re a mainframe developer who is asked to help out on managing the cloud infrastructure on an application (perhaps because staff got cut so everyone needs to pitch in). That developer now needs to learn a whole new stack. Reducing that learning curve is incredibly important for products and teams to be successful.
This is one (purposefully embellished) example of where Platform Engineering teams come in. Their job is to unify the core developer tooling and developer experience that engineers get throughout the org. This allows developers to focus more on understanding the codebase and how systems interact rather than having to learn the AWS console and the GCP console provisioning and then read through documentation to understand how to use Github Actions after having used Jenkins. The amount of cycles spent learning, managing, governing and integrating the tooling is reduced by Platform Engineering teams. This may not be possible with all the legacy systems and processes in place at a large enterprise but as much as it can be, it’s helpful for efficiency. And as we can see from the survey, less than half of developers have basic tooling that we expect for them to be productive.
We still have a massive developer talent shortage gap with IDC predicting a shortfall of 4M developers by 2025 for different roles. This means that companies will have to do more with less talent. Customers and users are increasingly demanding fast, personalized, and powerful products that give them a consumer-like experience.
In order to fill that gap, enterprises need to make developers more efficient. Platform eng teams help unify the developer experience which encompasses things like tooling, documentation, processes, resource provisioning, and base level of security. This article by Daniel Bryant on how Infrastructure Meets DevEx with Platform Eng is a great read to accompany this shorter snack bite.
This is why even as we are in a recession (the depth of which is to be determined), Platform Engineering teams are still getting allocated more and more budget. By improving the developer experience, costs are reduced through efficiency, improving velocity, and needing less developers overall to do the same job.
A great example of this is CloudQuery which enables developers to query and manage their cloud infrastructure simply by using SQL. It can work across cloud systems, across policies, and across any database. It meets users where they are from the tech-forward companies using Snowflake with Okta, Slack, and Pagerduty set up to the orgs that are trying to get there but currently use Postgres and AWS only (which sounds pretty awesome actually). As long as there’s a developer who understands or can learn SQL, they can get a handle on their cloud and security infrastructure with just a few lines of code (and in many cases being able to simply use templates already provided by CloudQuery or the community).
Now imagine all the security teams at say a healthcare company having the ability to set policies like the standardized AWS foundational security policies and query their infrastructure across clinical, consumer, and internal apps to make sure that systems are functioning compliantly. Meeting users where they are and with easy to use tooling is what Platform Engineering teams are looking for.
We’ve been investing in this trend for a long time at boldstart but are seeing the direct accelerant of this trend on companies like Roadie, CloudQuery, Env0, Jeli, Spectro Cloud, Steadybit, AtomicJar, Jit, and countless others. Developer efficiency has long been something enterprises are willing to invest behind but for the most part was hidden. Now as Platform Engineering becomes a practice that more enterprises are adopting, we are seeing and believe we will continue to see acceleration in the focus on helping improve dev efficiency by nailing developer experience.
What I enjoyed this week:
Speaking of Platform Eng products. Boldstart company, Roadie helped Gergely with his piece he wrote on Backstage and why it’s such a fast growing OSS project given the trends I just discussed in the above post (highly recommend subbing to the Pragmatic Engineer which I think has >300k subscribers!).
Delighted to have been able to help with this article. Check it out if you want to learn how Backstage can improve discoverability for your engineers.What is Backstage and why are companies like Netflix, Grab, Wealthsimple, QuintoAndar, Wayfair and hundreds of other tech organizations adopting it? In this week's subscriber-only article, we went deep into Backstage, and alternatives to it: https://t.co/54dPV4Sbd3 https://t.co/jOvVsJlQyF
The Pragmatic Engineer @Pragmatic_EngThis whole thread by the CEO of Hashicorp is a must-read to understand the trends in cloud infrastructure teams, how these products get adopted into an org, and why Platform Eng budgets are increasing
Software Snack Bites the podcast is off to a hot start with close to 3k listens! Here’s some of the near term lineup in the future. Thanks to the guests and listeners who have made it a success so far!
This was so freaking cool. My friend, Bill Brewster got to interview Tom Gayner, CEO of Markel (a true smaller Berkshire Hathaway type model). It does not disappoint covering buy & sell decisions, opportunity cost of capital, how to manage investing float for the long term. In general, tons of great lessons from a brilliant investor and business operator.
Great deep dive into Cloud Security by the Francis and the Contrary Research team. It’s a huge space with a lot of moving pieces but this research memo does a great job distilling key insights.
Really fascinating. I think as more shops move to 'everything as code' and do more with less, we will see more efficiencies like the ones you mentioned being driven by a platform eng type of group. I'm curious though, what would the result be if you asked Platform Eng/DevX types how much of their effort/focus is in better security as well as I think there's a good overlap with everything else listed.
We can apply a lot of the insights here to the companies trying to disrupt CFO office workflows (Pigment, Runway Financial + many many others).
There are a lot of people/processes unique to each legacy company that, if moved to these new platforms, may be highly destabilizing. Excel is the COBOL of this space.
Crossing the chasm requires figuring out a solution to this.