The question is “why is there no one-size-fits-all automation and orchestration solution”?
Automation, a component critical for continuous deployment particularly as it extends beyond app operations and into the broader environment to encompass network and security operations, too, continues to be a custom build kind of thing. There’s no single automation tool, framework or pattern that can be generally applied to a significant number of environments. Mi casa es no su casa, after all. That’s because environments don’t actually look like power point slides; your environment is not the simplistically diagrammed firewall –> load balancer –> web server typically found in decks extolling the virtue of one solution over another. They are far more complex, comprising not just multiple tiers of interdependent and interconnected apps and services, but multiple chains of infrastructure services spanning the gamut of security, availability, mobility, performance and access control. The specific flow through and between these services is unique. Indeed, the intimate relationship between network latency and app performance is well known, and thus the navigation through the critical yet complex site of services required to deliver an application could easily be considered a competitive advantage if it’s able to keep performance under the 5 second toleration limit on response time apparently now the standard for consumer satisfaction.
So it’s no surprise that automation and orchestration, responsible for automatically setting up the triggers and routes required for application traffic to correctly traverse the right set of services for optimal performance, would not be a simple, turn-key solution. One size does fit all. It must, like its business process focused counterpart – BPM – be customizable and easily changeable.
BPM? Are you seriously?
I’m dead seriously. BPM – business process management – has been around far longer than automation and orchestration of IT systems and its focus is, as its name implies, on process. Automation is included, but it is the orchestration of business processes that often change in the face of variable business conditions that makes it an invaluable tool for optimizing business and a source of competitive advantage. “The ability to change quickly is essential. Businesses change their key processes 4-7 times per year. The driver for change can be internal or external. New opportunities can arise. New partners or customers need you to support a different way of doing business. Federal or international regulations can require you to change your processes.” (Making the Case for BPM, Jim Rudden)
Consider for a moment this data from a 2012 Bearing Point survey on the goals and impact on culture:
Top BPM goals are increased efficiency (93%), increased transparency (90%) and the realization of standardization potentials (85%). A number of BPM goals have an impact on corporate culture, e.g. increased transparency (rank 2), increased customer orientation (rank 5), disclosure of weaknesses/ bottlenecks (rank 6) and the reduction of mistakes/ error rates (rank 9).
– Bearing Point Business Process Management Survey 2012
If many of these goals and impacts sound familiar to you, they should. Because what DevOps ultimately needs to focus on is exactly the same as the focus of BPM – process. In an application world, these processes require applications and services. IT must be able to support these process changes in the application and service infrastructure.
BPM (Business Process Management) has known this for years, and it is the orchestration of the process – the workflow – from task to task, from approver to doer that generates the efficiency gains achieved with that technology.
Indeed, adopters of DevOps who focus on the “big” picture would do well to consider the similarities to BPM which touts as benefits such things as:
- Process quality improvements
- Continuous process improvement
- Reduces costs
- Improves business agility
It would be difficult if presented with that list of benefits with no attribution to determine whether it was referring to DevOps or BPM. That’s because the operationalization of business through process orchestration is not all that different from the operationalization of IT through process orchestration.
And processes are not, nor should they be, turn key and one-size fits all. They should be tailored to address the needs of a specific operational or business goal, i.e. deploy an application or secure an order from a customer, based on the systems in use to achieve those goals. BPM shines where it is most able to address bottlenecks and obstacles caused by handoffs between systems and teams. DevOps will shine where it is most able to address bottlenecks and obstacles caused by handoffs between systems and teams. It isn’t about racing through a given task with automation. Speed isn’t efficiency and, if you do the math, a focus on automation without similar focus on process can lead to even greater inefficiency.
And that’s why there is no one size fits all automation and orchestration tooling; because the goal isn’t automating tasks, it’s orchestrating process. Coordinating workflows. Ferreting out inefficiencies.
Orchestration is the glue that binds automated tasks together. For example, recently a missed route update caused some unexpected down time during what was considered a routine* decommissioning of gear. A missed route update. In other words, someone forgot. That task might have been automated, it might have just needed to be kicked off. But it wasn’t.
That’s the kind of thing that would likely not happen if the process was governed by some form of orchestration; if there was a clear list of what needed to be updated or changed based on the removal of that particular component. If orchestration was involved to provide visibility and oversight of the operational processes needed to perform such a routine task. This does not mean the tasks should have been automated; in fact automation is not strictly required to achieve the visibility and oversight necessary because processes are about workflows and hand offs, not specific tasks completed during the steps that comprise a process.
Which leads us right back to Business Process Management. Take a deeper dive into the aforementioned Bearing Point survey and you’ll find more similarities with DevOps:
- Crucial elements for sustainable BPM are clearly defined roles (95%), top management support (92%) and a clear mandate (92%)
- Typical implementation barriers such as the lack of resources (53%), sponsorship (32%) and experience (25%) need to be identified early and addressed in a structured way
- BPM as a continuous (=sustainable) approach is already adopted by 59% of companies, coordinated and implemented either centrally or locally
- Involvement of stakeholders (90%), process oriented thinking (90%), organizational institutionalization (82%) and a well-communicated concept (75%) are elements of communication and change management which must accompany the establishment of sustainable BPM
That’s because both are ultimately about process, with automation being only one of the tools in the tool chest that support implementation of a continuous approach to enabling agility in the process layer. Indeed, about 38% of the companies in the survey use software to support their BPM initiatives while another 37% do not. A spreadsheet with the right Six Sigma calculations can be as powerful as any automation framework, toolset or API in identifying the dead space between tasks that introduce inefficiencies and delay application deployments.
BPM has, like DevOps, brought tangible, measurable results to the table. Bearing Point found that almost three quarters of the surveyed companies (72%) were able to achieve measurable benefits through BPM such as process lead time (72%), process cost (68%) and error rate improvements (67%).
So rather than focusing too much on automation, adopters and proponents of DevOps would do well to up-level the conversation and start looking at process orchestration and improvement as the goal. The successes BPM has had on the business side of the equation tell us that doing so will provide a better path to achieving success with DevOps in the operational side of the equation.
* I’d argue that in today’s environment of hyper-sensitivity to availability that nothing that touches a service or gear in the critical path of an application is “routine” but, hey, it is what it is.
