Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Engineers at fast-paced startups often wear both Dev and DevOps hats. Early in my career, as a developer at a sentiment-analysis startup, I was tasked with writing scripts to spin up servers, configure them, install necessary software and deploy the code we wrote. My custom shell scripts, Python scripts and cron jobs did all the heavy lifting so that we could reliably deploy software and scale resources on AWS. Fast forward to today, and that problem of confidently building and securely deploying software has become an even more enormous challenge at scale.
The Cloud-Native Application Stack
A simple looking e-commerce app today can easily be a big set of coordinating microservices that are powered by containers, orchestrated using Kubernetes and hosted on multi-cloud/hybrid cloud. This combination enables faster delivery of code, cheaper hosting bills and more efficient use of server resources.
While the application components certainly became smaller over time, organizations today are tasked with managing tens of thousands of containers in a production environment. In a 2020 Cloud Native Computing Foundation (CNCF) survey, 92% of respondents said they use containers in production, a 300% increase from just 23% in the first survey from March 2016. Such coordination complexity at scale is being managed by software delivery pipelines. Just as Ford revolutionized the automobile industry with its assembly line, these integrated DevOps products are massively impacting verticals like finance and insurance.
DevOps Trends
Trend: Developers are now Decision Makers and Influencers
We are excited to see the increasing impact and influence of developers when it comes to the selection of tools. The top-down sales motion largely doesn’t work in the cloud-native world, and needs to be supplemented by bottom-up community adoption. The best developers pick peer-recommended tools to solve these tech challenges. Developers today leverage large communities, self-organize and pick the best open source tools to kickstart small projects that have potential to impact the growth curve of any organization. For vendors, developer evangelism becomes an important strategy for customer acquisition.
Trend: Integrated Software Delivery Pipelines
CIOs typically pick category-leading products for each category of the DevOps toolchain, like CI, CD, SAST and SCA. Historically speaking, integrating these tools to maintain an effective CI/CD pipeline has been painful and requires stitching together tools using custom scripts. We hear the frustration when we speak with maintainers of such scripts. To address this challenge, we think end-to-end integrated DevOps platforms will become the norm. Such platforms will offer a few best-in-class components with built-in automation, and will enable organizations to integrate other DevOps tools in a plug-and-play manner.
Trend: Shifting Left with Code
The shift left phenomena, which enables fixing of bugs/security issues early in the software development life cycle when it is cheapest to do so, is accelerating. By implementing a shared security responsibility model and enabling developers to take more ownership related to infrastructure, security and other critical production issues with code, organizations are shipping code faster and cheaper with less bugs.
Trend: Low-Code DevOps Automation
Many workflows are pretty standard in the DevOps world. Go look at an alert in production; prioritize it, look at the knowledge base for a fix, create incident, apply fix, resolve ticket and then repeat. This example is an oversimplification, but there are many repeat use cases that can be automatically managed. These workflows can and should be automated, which can make DevOps teams more effective. We think that customizable low-code automation interfaces will evolve to act as secret weapons for effective DevOps engineers.
Trend: Security at the Speed of Dev
Software release cycles a decade ago used to be four to six months long — if not longer — but today, the software delivery life cycle and release cadence is accelerating. The 2019 State of DevOps report suggests that the top performers deploy software 1,460 times per year, which is effectively four deployments per day. Security, compliance, vulnerability management, authentication and authorization all have to move at this speed, otherwise delivery of secure code is just not possible. We see the most innovative CISOs adopt a shared ownership model of security that enables application component owners to detect and fix vulnerabilities. In this shared model, a good balance between agility (so that development is not blocked) and security (so that risk can be managed) needs to be struck.
Are You Building the Next Big Thing?
Cloud infrastructure is an enormous opportunity that Gartner expects to grow to $412 billion by 2024 and many enterprises are just getting started. If you haven’t been paying attention, Satya Nadella now runs Microsoft and Andy Jassy will soon run Amazon. Both these leaders are pioneers in cloud, having led Azure at Microsoft and AWS at Amazon, respectively. This shows how much the largest companies, as measured by global market cap, believe in the opportunity related to the cloud. We couldn’t agree more.
