These days DevOps is being widely adopted by many IT organizations. Such IT organizations become more agile and reliable, deploying code more frequently with fewer bugs and failures. DevOps helps to get better IT performance and build additional competitive advantages, and companies with high-performing IT organizations are twice as likely to exceed their profitability, market share and productivity goals.
The adoption of DevOps is being driven by factors such as agile development methodology, significantly increased rate of production releases, wide choice of public and private cloud platforms, and a wide range of tools for data center automation and configuration management.
Today there are many different automation tools on the market that are used by DevOps teams for provisioning, orchestration and configuration management (Puppet, Chef, CFengine, bcfg2, Vagrant, Fai, Kickstart, Preseed, Cobbler), for build and deployment automation (Jenkins, Maven, Ant, Cruisecontrol, Capistrano), for infrastructure monitoring (Zabbix, Nagios) and application performance measurement (NewRelic, AppDynamics, DripStat).
However IT organizations still struggle with assisting both developers and administrators that prioritize two completely different goals – speed of application development and feature completion versus production stability and uptime. Automation tools vendors promote their products to solve the challenge of DevOps teams, but still very often fail to face the complexity of workflows for operations engineers, or application releases and deployment speeds desired by developers.
Another issue is that IT organizations with application release and deployment issues usually use some kind of automation, but they want to have more flexibility to manage their automation without the need to invoke everything manually at the command-line. Ideally, this automation should be available for developers in their development environments to give developers more control of the environment and understanding of the infrastructure usage.
These issues have driven the evolution of DevOps tools. Today, cloud platform vendors consolidate and integrate automation and management tools directly into their cloud platforms. As a result, IT organizations can get better automation features in the same turnkey cloud platform. We can claim that PaaS solutions will become the next generation of DevOps tools. Cloud platforms have already removed the need to worry about many aspects of provisioning and configuration management of virtual machines/containers. These cloud platforms solve it in a very efficient way, because they have more control of resources and events, they manage the whole cluster of resources and take into account different metrics and status indicators at the same time. As a result, these platforms provide significantly better density of resource, better utilization, smart distribution of the load across the cluster, automation of new release deployments, automation of scalability and maintenance, out-of-the-box high availability and fault tolerance features.
In fact, many IT organizations use automation tools such as Puppet and Chef to solve the challenges of template management, updating middleware stacks, application configuration management and zero-downtime deployment of new releases. However, the latest cloud solutions include more “native cloud” features designed to solve the same tasks. There are many examples of this trend for cloud platforms to evolve in this new, DevOps, direction, providing native automation features in popular cloud platforms.
Let’s take one of the visionaries in the cloud computing industry – Amazon Web Services. One of their cloud services, CloudFormation (http://aws.amazon.com/cloudformation/) allows developers and administrators to create and manage resources, middleware stacks and application templates. It helps to manage configurations and updates of already provisioned resources and applications. Literally, it‘s designed to solve the same tasks as Puppet and Chef. You can find many threaded discussions related to the comparison of CloudFormation vs Puppet/Chef. OpsWorks (http://aws.amazon.com/opsworks/) is another service for application management that simplifies application deployment and lifecycle management. It helps to define application topology, specification of package installations and software and resource configurations. In addition, this service includes automation for scaling based on time or load and allows scripting of these tasks.
Another good example of DevOps evolution is the new cloud services released by Google Cloud Platform. Recently this public cloud platform released several different features that help to solve similar tasks to those mentioned above – Replica Pool (https://developers.google.com/compute/docs/replica-pool/), Startup Scripts https://developers.google.com/compute/docs/howtos/startupscript and Cloud Deployment Manager https://developers.google.com/deployment-manager/. These natively integrated cloud tools provide the ability to create and manage sets of virtual machines (VMs) from common templates, resize, update, or delete them, define different configuration attributes such as base image, attached disk details, attached networking objects and scripts to be executed after the boot, install new software and updates, define environment/OS variables persistently available to VMs whenever it starts. In addition it provides the ability to easily declare, deploy, and maintain complex applications using a simple template mechanism. It provisions, scales, and monitors VMs and applications. Moreover, it can perform regular health checking and can restart or re-build sets of VMs that fail to respond appropriately. Recently announced Cloud Debugger and Cloud Trace (http://googlecloudplatform.blogspot.com/2014/06/enabling-developers-to-tame-production-systems-in-the-cloud.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+ClPlBl+%28Cloud+Platform+Blog%29) allow developers simply setting a watchpoint on a line of code and to get a snapshot of all the local variables, parameters, instance variables and a full stack trace. It helps quickly identify and fix performance bottlenecks and enables engineers to compare performance from release to release with a detailed report. It works with zero setup uptime and no complex configuration, and is suitable for use in production. The tools described above cover a large breadth of DevOps tasks.
Docker (http://www.docker.com/) is another good example of a tightly integrated cloud automation mechanism and a further example of DevOps evolution. The promise and focus of this automation solution are to remove complexity of configuration and deployment into different environments. Many cloud vendors have announced integration with Docker. Many observers believe that Docker is still at an early stage and not yet production ready, but the product has generated lots of industry buzz and the technology seems to be significant.
In addition, there is something even bigger in the DevOps evolution – the open initiative for unification of cloud automation as Topology and Orchestration Specification for Cloud Applications (TOSCA) https://www.oasis-open.org/committees/tc_home.php?wg_abbrev=tosca. The goal of TOSCA is to enhance the portability of cloud applications and services, enabling the interoperable description of cloud services and relationships between parts of cloud services and operational behaviors. Examples are template configuration and provisioning, application deployment, software patches and updates, application scaling and maintenance, etc. In other words, it drives a unified standard for independent cloud platform vendors or cloud hosting service providers. As a result, it should simplify the work and tasks of DevOps teams by creating a vendor-neutral ecosystem enabling portable deployment to any compliant cloud platforms, smoother migration of existing applications to the cloud and the ability to build and run multi-cloud provider applications.
In conclusion, the future will bring more cloud native absorption of different automation features for DevOps needs. The evolution of cloud platforms in the DevOps direction is extremely beneficial for IT organizations because they do not need to manage and learn different tools and approaches for automation. This unneeded complexity will be replaced by seamless integration of different automation features inside the same cloud platform. It means in the near future very popular DevOps tools like Puppet and Chef will compete with built-in functionalities of cloud platforms.
However, during the evolution period, choosing the right cloud platform is critical. It should allow IT organizations to use multiple approaches – the old legacy automation methods and the new native features to give the maximum flexibility from built-in automation and an already collected knowledge base. Flexibility of choice is the key point in the DevOps evolution for IT organizations – the ability to use both worlds at the same time: the old automation tools and the next generation features of cloud platforms.
In addition, it makes sense to mention the importance of seamless integration between PaaS and IaaS for the evolution of DevOps. The public cloud platforms discussed above (AWS and Google Cloud) provide merged solutions today. The integration of both layers is very important for enterprises that heavily use private and hybrid clouds, as it replaces the unneeded maintenance complexity of two different solutions and removes extra overhead on integration of products from different vendors. In other words, merging PaaS and IaaS provides extreme flexibility of automation and ease of use in the same solution. Seamless cloud platforms provide the best combination of the automation features in the same turnkey cloud solution that help to drive innovation and maintain legacy apps.
The next generation of cloud platforms will establish a win-win relationship between the dev and ops departments, and when the outcome of dev and ops interactions is a win-win, IT organization performance wins as well.
About the Author
Ruslan Synyntsky, is founder and CTO of Jelastic. With over 15 years in the IT industry, Ruslan is an expert in large-scale distributed Java applications and enterprise platforms. Before founding Jelastic, he led engineering teams a and at the Space Agency, Ukrainet iQueLab, SolovatSoft, Datamesh.
