Using CI/CD can help digital service providers move as quickly as over-the-top content providers
Online video consumption has been increasing tremendously with rapid change in consumer expectations to have seamless viewing experience across various digital devices. To capture this growing demand, it is critical for digital service providers (DSPs) to deliver fast-track rollout of next-gen media services. Moreover, to counter the growing influence of over-the-top content providers (OTTs), DSPs must deliver media services that are high quality, right the first time and consistent, along with creating competitive advantage through speed to market. However, unlike OTT players, DSPs are challenged by extremely long development and rollout timelines for new services and offerings, primarily because of vendor-specific hardware and software applications that do not support rapid changes.
Embracing an effective continuous integration and deployment (CI/CD) setup can enable DSPs to achieve the same service innovations and delivery agility that OTT providers are offering. An important characteristic of an effective CI/CD setup is to create a fully automated deployment that is easy to operate and visualize. The setup also should have remote accessibility to ease parallel deployment at multiple geographic locations and a robust monitoring toolchain to monitor each step in the deployment process.
However, implementing a CI/CD setup with these characteristics for next-gen media landscape comes with its own challenges; it requires end-to-end CI/CD orchestration, which is operationally complex. This involves stitching together different types of video services such as live TV streaming, VoD, electronic program guide (EPG), 4K/HDR, user personalization and profiling. These services need to be rolled out in multi-geographic locations and multi-tenant environments, catering to multiple vendors. The service must offer a seamless viewing experience across all digital platforms including TV, mobile, tablet and desktop.
In setting up such complex processes, cross-functional and cross-geographic teams own different parts of the process, which can create gaps in design, infrastructure setup, configurations and eventually deployment. The CI/CD pipeline created can be costly and ineffective.
Setting Up Effective CI/CD
One-click automated delivery pipeline tool for predictable rollouts/rollbacks
Create a one-click automated delivery pipeline tool that can facilitate automated change requests (CRQ) to initiate CI/CD jobs. The tool triggers the entire deployment of all the app components in the respective tenants across multiple geographies.
Key considerations in setting up a one-click automated delivery pipeline:
- Track CRQ approvals over systems such as JIRA to trigger the automated delivery pipeline.
- Build pipeline as a code with intermediate CI/CD stages using open source tools such as Jenkins.
- Design a master job to accept component and configuration details. This performs multi-job configuration and triggers contextual deployment/rollback simultaneously across tenants.
- Interconnect different aspects of networks, infrastructure, components and applications to run the entire process smoothly.
Setting up an automated delivery pipeline provides greater accuracy and predictability by reducing the human error factor.
Automated snapshots, pre- and post-validation scripts to ensure consistency
Create a fully automated snapshot management process that includes automated snapshot creation, rollback and deletion across respective tenants. Automated snapshots before deploying the new version of application facilitates rollback to the previous consistent version with zero downtime, if any failure occurs.
Ensure automated multiple validation checks at every deployment stage to ensure consistency. These checks include validation of network connectivity, memory usage, CPU utilization, running applications, error and debug log files, license files, binary file size and configuration files. These validations can be done via tools such as groovy, ansible, Jenkins, python or shell scripts with proper libraries that match industry standards.
Auto-scaling tool to optimize resource usage
Ensure the application always has the right amount of capacity to handle traffic demands. This requires integrating an auto-scaling tool with predictive machine learning (ML) models and change request process to dynamically scale up/down the infrastructure.
Key considerations in setting up auto-scaling tool:
- Identify external and internal influencing factors to analyze system throughput.
- Leverage insights derived from a monitoring dashboard to manage changes in current payloads.
- Leverage triggers from a ML model to plan future expected system throughput.
- Create automated configurable scripts to support dynamic scaling based on the inputs received from change request and ML model.
Creating such tool enables better infrastructure availability with efficient cost management.
Tool-agnostic and on-demand environment creation framework built on open source
Leveraging open source tools provide flexibility and control over the entire exchange-to-exchange deployment process. Using customized framework with a layered approach makes the delivery pipeline tool-agnostic. It gives flexibility to easily replace/plug in any tools in the DevOps chain without much dependency.
This ensures increased reusability, as the same toolchain can be used for multiple environment setups. Further, a customized framework dynamically handles the resources and utilizes the infrastructure effectively.
Service assurance dashboard for stability and monitoring end-to-end processes
A service assurance dashboard can provide glimpse of end-to-end CI/CD performance in a single window. It provides real-time graphical insight into failures/blockages in the pipeline, highlighted in the context of the overall flow. The dashboard should dynamically fetch data from the deployment pipeline for respective components and monitor how the upgrade/rollouts are processing. It should provide insight into resource usage, which can trigger dynamic auto-scaling and give a visual comparison of previous firmware with the new one.
Implementing such a service assurance dashboard can provide greater visibility and help in easy diagnosis and error prevention.
Benefits
- Accelerated time to market with nearly 60 percent reduction in waiting time with parallel deployment across multiple tenants.
- Improved predictability with significantly reduced fallouts and downtime/failure rates.
- Optimized cost with approximately 50 percent lower IT spends in the deployment process and resource usage.
This article was co-authored by Sumit Thakur, manager, strategic insights at Prodapt; Vijay Ramachandran, senior project manager at Prodapt.
