Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
The push toward digital transformation and cloud-native infrastructure is inevitable. This shift brings the need to manage operations with the same rigor and automation we apply to infrastructure or security. Many organizations have embraced the ideas of everything in a pipeline and all things as code. While platform engineering and other teams have created answers for building and delivering applications and the necessary frameworks in which to run them, the actual operations of service delivery are often disjointed and purely reactive.
Enter operations as code.
By leveraging tools such as Terraform with automation and CI/CD pipelines, site reliability engineering (SRE), DevOps and DevSecOps teams can standardize and automate operational tasks, ensuring consistency, efficiency and reliability.
Operations as code extends the principles of infrastructure as code (IaC) to operational procedures. It involves defining, managing and executing operational tasks — such as defining escalation policies, defining runbooks and executing playbooks — using code and automation tools. This approach ensures that operational practices are repeatable, version-controlled and can be executed with minimal human intervention.
Avoiding Bottlenecks
One of the greatest benefits of the operations as code approach is the ability to remove the dependency on centralized teams. As the need for speed across DevOps practices increases, these teams cannot be beholden to centralized ITSM or other teams for their needs to integrate new monitoring, enrich events or create new runbooks.
Similarly, centralized teams devoting expensive, specialized skills on monitoring integrations, event management, enrichment and automation that can be managed via operations as code makes little economic sense. These teams, especially in large organizations, are already stretched thin and the backlog of work grows daily. Leveraging Terraform or other mechanisms to achieve the same goals while delivering better outcomes makes more sense for all teams working together.
Leveraging Pipelines and Terraform for Operations
Terraform, traditionally used for IaC, has become the lingua franca of DevOps. By writing Terraform configurations, teams can automate the provisioning and management of not only infrastructure but also the operational workflows that ensure operational excellence. For instance, Terraform configurations can define tasks such as service definitions, configuring users, teams and roles, defining escalation policies and schedules, defining event correlation and orchestration and defining automation such as runbooks and automated diagnostics.
CI/CD pipelines play a crucial role in operations as code. By integrating operational tasks into CI/CD pipelines, you can ensure that changes are tested, reviewed and deployed in a controlled and automated manner.
Quality gates are traditionally used for code reviews, automated testing, security checks, etc. For operations as code, they can be leveraged for standardization by ensuring consistency of core functionality such as service standards, tiers of escalation policies, minimum requirements for runbooks, etc. Performing compliance checks to ensure operational changes comply with internal policies and external regulations. And eventually, they can be used for scoring applications for operational readiness.
Benefits of Operations as Code
Organizations that deploy operations as code will see several benefits, many with immediate return on investment (ROI).
Toil reduction is critical. Too much time is spent in ‘ClickOps’, and by shifting from manual configurations, more time is freed up to grow automation and free up resources for more valuable work. You can also reduce operational risk by ensuring the traceability of changes to configurations, version control and templates that reduce the risk of error. Similarly, you can operationalize governance and compliance by leveraging parsers, quality gates and approved templates, while leadership can define acceptable minimum standards and expected outcomes.
Operational excellence is improved by reducing the frequency, severity and duration of outages by ensuring repeatable outcomes and reduced errors. You can shift away from tribal knowledge by giving senior people a simplified, repeatable method to record their innate knowledge for reuse and creating a context for junior staff.
Developer experience is improved by reducing the ramp time of new team members, allowing them to focus more on high-value work and building capabilities, while putting in less time looking for ‘how to’ or escalating to experts. Most importantly, you can start a shift from run to build by reducing the amount of time in keeping the lights on and chasing break-fix work, while the senior staff can focus on reducing tech debt (or mining tech wealth if you are optimistic) to deliver great customer experiences.
Getting Started
Successfully rolling out operations as code involves several key steps:
- Firstly, you must define success. How are you going to measure the efficacy of your operations? Think beyond the mean time to repair (MTTR). What about the cost of keeping the lights on or reducing the time and cost of break-fix work? How can you better attack tech debt?
- Next, assess current operations and identify the initial areas that can benefit from automation and templates. What can you immediately de-risk or outcomes could you influence with standardizing operations?
- Then, ensure you have the appropriate tools that fit your environment and architectural goals and that your teams are trained in those tools and associated best practices. You will want to establish a center of excellence by building teams of enthusiasts and experts who can help with Q&A, become keepers of the templates and help build continuous automation and orchestration improvements.
- Lastly, focus on incremental implementation starting with simple but impactful areas, and then build using continuous improvement to regularly review and improve your processes based on feedback and metrics.
What’s Next?
Operations as code represents the next frontier in IT management, offering the promise of consistency, efficiency and reliability in operational tasks. By leveraging Terraform, CI/CD pipelines and robust tools, you can lead your teams in adopting this transformative approach. While challenges exist, they are surmountable with careful planning, execution and continuous improvement. Operations as code can be a cornerstone of operational excellence allowing your teams to help move from a world of toil and break-fix to building the capabilities that will help you win in the marketplace, better serve your teams and most importantly, your customers.
