There are competing products, both open source and licensed, in any DevOps area that you can think of. The Operations groups in companies tend to accumulate such applications over time as determined by ongoing projects. Team members’ familiarity with a certain application could be a factor in adding it to the tools stack.
Companies, therefore, end up with a mixed bag of freeware, open source or community versions of applications and software licensed from the get-go. In a startup environment where the Operations group might be on a shoestring budget, it might make sense to build the DevOps infrastructure on the cheap; using open source software (OSS) products would be one way to achieve that goal.
The Shoestring Budget
Below is a list of some of the applications and software for an effective DevOps environment. The list here includes only core applications that are needed to build the DevOps infrastructure. The basic communication and productivity tools that would be used by everyone in the company—and usually managed by IT—are not in the scope. Neither are the compilers and debugging and testing tools used by product development teams. However, in a small company setting, managing all kinds of tools might drop into the laps of the same people driving the DevOps initiatives.
The objective here is to make a list of software that could be used license-free. That means some of the obvious applications in certain areas might not make it to the list. That doesn’t mean that the open source alternative is an inferior choice; availability of reliable product support is a major factor in most of the software purchase decision-making processes, and licensed software scores over OSS on that front.
Git for Source Code Management (SCM)
A code repository might be the first DevOps infrastructure component that would be set up even before a product idea is formalized into a company. Also, both developers and DevOps engineers need it for collaboration, within their own teams and across various engineering teams. Typically in a startup, developers get started with the DevOps work such as build and deployment automation before a dedicated DevOps team would even be part of the engineering staff.
In this category, It is hard toimagine choosing anything other than Git. Even though GitHub is not free for maintaining private code repositories, Git server can be setup on-premise for free. AWS has CodeCommit, another SaaS version of Git, as one of the services, and there is no extra cost for using it if you are already on AWS.
Bugzilla for Bug Tracking
The popular Atlassian suite of products, JIRA in this category and Confluence for collaborative documentation, seems to be default choices in respective categories. But, both on-premise and SaaS offerings of JIRA and Confluence are licensed. Bugzilla is a powerful OSS alternative to JIRA that can support a large user base and it has been around for a while.
If you are open to customizing Bugzilla for specific requirements, it could work better than some of its licensed competitions. Though its primary use is bug tracking, Bugzilla is feature-rich enough to be used for managing all phases of software development and even as an IT ticketing system.
MediaWiki for Collaborative Documentation
Though there are multiple, open source wiki applications available, MediaWiki might be the best supported when it comes to the availability of third-party extensions. Out-of-the-box features of MediaWiki are limited, but it can be extended using the free extensions available. Such customizations are very straightforward.
Jenkins as Build & Continuous Integration Platform
Though there is an enterprise offering of Jenkins from CloudBees, its open source version combined with freely downloadable plugins would make up a powerful integration platform for build, release and deployment automation—and any other areas in between. Describing Jenkins any more here would be simply preaching to the choir.
Artifactory for Artifacts Management
This is one of the applications a lot of companies might not have in their DevOps stack in the initial stage. However, the need to store built artifacts and to cache third-party tools is real, and the packages and binaries would normally end up in S3 buckets or shared filer volumes. There is no reason for not using the OSS version of Artifactory right from the beginning, as it covers the storage requirements already mentioned and much more:
- Automated uploading of artifacts as part of the build process, typically done from Jenkins
- Caching third-party tools locally for builds and deployments
- Storing software artifacts for meeting build dependencies
- Distributing packages and third-party tools for deployments
Ansible for Configuration Management and Orchestration
There are two great products in this space that could be used for free: the OSS versions of Puppet and Ansible. If you are on AWS, Chef is also available, at no extra cost, in the form of OpsWorks. But I would recommend Ansible for those getting started for these reasons:
- It is an agentless tool and beginners would feel safer to roll out automation using it. A configuration management agent can play havoc with production systems if something gets misconfigured inadvertently.
- Using the playbooks, deployments can be orchestrated very well using Ansible. In many companies, Puppet or Chef primarily covers system management aspects and there would be additional tools for deployments. Ansible simplifies that scenario.
- Ansible’s support of Python turned out to be more useful, in my experience, than the support of Ruby in Puppet and Chef. The chances of a DevOps engineer knowing Python is much more than Ruby.
- Ansible’s handling of host inventory is advanced, especially the dynamic generation of inventory lists. The latter feature is compatible with the provisioning of cloud infrastructure where its elasticity demands the use of dynamic inventory lists in configuration management systems.
Nagios for Monitoring
A lot of us in the DevOps community despise Nagios for its blandness and lack of features out of the box. But if you are willing to customize it by developing application-specific plugins, it could be converted into a very effective monitoring platform that can be used to consolidate outputs from the rest of the monitoring and various admin applications. I recommend it for these reasons:
- Writing plugins for Nagios is very simple. Also, there are tons of free plugins available that would meet most of the system-level and third-party component level monitoring.
- Developing plugins to add application-level monitoring and to integrate with third-party admin applications is easy.
- When you want to move on to a next generation cloud-based monitoring application such as Datadog, existing Nagios plugins are normally migrated natively or as an integration.
In a sysadmin environment where coding skills would be in short supply, Nagios could be of limited use as a systems monitoring application. But DevOps teams should treat it as an easily extendable tool that can be used to implement specific and fine-grained monitoring goals of the company.
ELK for Log Management
Splunk was instrumental in changing the way we dealt with log files, and, now there are several good products including Loggly, Logentries and Sumo Logic in that space. Though some of these SaaS products have a free tier and that might be enough for a small operation, it is better to start with ELK (Elasticsearch, Logstash and Kibana stack), mainly because the size of the logs can grow very fast and so would the usage of the log management application in the company. ELK requires substantial DevOps resources to set it up and maintain, as there are three applications involved.
Closing Remarks
It is important to have these essential applications, or similar ones with comparable features, implemented or made available to empower the engineering staff. They facilitate the application development, lay the foundation for automation initiatives, and provide the toolkit to maintain production environments 24×7.
Though it is normal, the organic growth of the DevOps organization and infrastructure can bring in OSS products with overlapping features, mainly because they could be used license-free. Carefully selecting the tools would help avoid the otherwise-inevitable clutter and the confusion later.
One aspect not discussed is the total cost of use of a software. If SaaS offerings such as GitHub, Datadog and Loggly, and hosted versions of JIRA, Confluence and Artifactory are used, the cost would only be the subscription charges. OSS applications will need computing resources and some amount of customization and administrative overheads, which could add up to the overall cost.
If the objective is to minimize total cost, the upfront decision-making can become much more complicated than it looks. However, for a startup or a small company, getting started with the OSS stack for setting up the DevOps infrastructure can be advantageous if the applications are selected carefully.
