Automation is one of the key aspects of DevOps and agile development. In many ways, in fact, it is the key aspect, since the longer it takes for developers and operations techs to complete rote, manual tasks, the slower and less flexible the environment becomes.
But automation can be tricky to master, particularly in highly complex ecosystems. Aside from selecting the automation platform itself, there are myriad questions regarding what to automate, how, and to what extent should automation scale up and out across distributed infrastructure.
Deepak Giridharagopal, CTO of Puppet, argues that automation is no longer a luxury for the modern enterprise; it’s a necessity regardless of how far the DevOps transition has progressed.
“At a high level, there is no decrease in pressure in sight for IT departments,” he said. “It doesn’t matter what industry, as time goes by, the future will be more entities that need to be managed – physical, virtual, containerized, third party – and the surface area that they need to understand and manage will continue to expand.”
When it comes to DevOps, automation provides the means to develop and deploy faster while maintaining or even improving quality. It does this not by replacing the need for human labor but by amplifying the performance of individuals by taking over the many dull, repetitive tasks that occupy their time. It also helps by reducing the friction that arises when key, interrelated tasks are not orchestrated properly.
The problem arises, however, when determining the best way to integrate automation into a complex ecosystem. In any system, performance is limited by the weakest link in the chain, which in this case would be the step not automated in the DevOps workflow. But attempting to automate everything at once can be just as problematic, since it can lead to conflicts and confusion between newly automated tasks.
This is why Giridharagopal recommends completing a full audit of development and operations environments before deciding the initial candidates for automation. Only by knowing what you have and how it works can you then begin the process of improving it.
“Often, people think Step 1 is identifying the slowest thing and making it faster, but Step 0 is usually necessary because they don’t have a great understanding of what they have,” he said. “One person might know the physical layer, one the virtual systems, but no one has a consistent view. In order to do the best job, you have to start with a baseline understanding of the landscape and a base level of situational awareness.”
From there, you can look into the lifecycle of software delivery – everything from the initial commitment to the auto-build to testing, beta and release – and see what resources can be provisioned and deployed as code. After that, it becomes a relatively simple matter of overlaying time and cost on a digital assembly line.
It is also important to acknowledge the depth to which most IT environments are already automated. Transitioning these solutions for agile development is largely a matter of converting these functions into sharable, executable tasks that can be repeated at scale.
“A good, low-impact place to look at is what is already automated,” Giridharagopal said. “Chances are, you are already doing some of this but it is just ad hoc – it is not scalable. You are paying for it but not taking full advantage of it.”
In the end, it’s not the systems being automated for DevOps, but the tasks and processes. If there is any rule of thumb to follow, it is to implement end-to-end automation on one workflow at a time. This removes the weakest links in the process while still accommodating a gradual rollout in a non-disruptive manner.
Below are some of the leading automation solutions for DevOps environments:
Ansible
Ansible uses a visual dashboard and role-based access control to automate complex environments. The system provides real-time job status updates and multi-workflow orchestration to provide deep visibility into the state of DevOps pipelines. Recent additions include new clustering abilities for added scale and redundancy, plus integrated notifications for Slack, Hipchat and other tools.
Apache Ant
Apache Ant (Another Neat Tool) is a Java library and command-line tool that offers built-in tasks for compiling, assembling, testing and deploying both Java and non-Java applications. Ant does not require strict coding conventions or direct layouts and can be combined with the Apache Ivy project to integrate building and dependency management in the same workflow.
Capistrano
Capistrano is a remote server automation and deployment tool that extends the Rake DSL tool to run commands on servers. It can be used to deploy web applications across multiple machines, automate audits, perform arbitrary workflow scripting, and a host of other functions. It also provides advanced, customizable filtering capabilities for properties, hosts and roles for partial deploys and partial cluster maintenance.
CFEngine
CFEngine is a configuration and automation system that scales up to 50,000 nodes and can incorporate servers, cloud resources, embedded devices and other elements. It claims to provide full infrastructure reconfiguration in as little as five minutes, handling everything from configuration and process management to reporting, compliance and application deployment.
Chef
Chef automates key functions in the DevOps process, such as configuration management, sysadmin, networking and cloud management. Chef defines infrastructure as code and continuously evaluates server conditions against their desired state to automatically correct for configuration drift and ensure changes are applied universally.
Gradle
Gradle is a building tool that streamlines and automates the creation of mobile apps, microservices and other projects. It works with virtually any programming language and features a rich API and broad plug-in ecosystem for end-to-end automation of the delivery process.
Jenkins
Jenkins is an open source automation server that supports a wide array of plug-ins for third-party tools across the CI/CD pipeline. The system can function as a basic CI server or act as the central automation hub for continuous delivery for any project. It also provides out-of-the-box installation with set-up and configuration through an intuitive web interface.
Juju
Juju is an open source application modeling tool that automates software deployment and configuration on public and private clouds. It uses script “charms” that can be automatically linked and bundled to streamline infrastructure management, effectively encapsulating the enterprise’s operations knowledge to maintain consistent environments at scale. Juju is also available as a service offering.
MCollective
MCollective is a framework for building server orchestration tools and parallel job-execution systems. Its primary use case is to programmatically execute admin tasks on server clusters, which removes complex network functions and allows for filtered control of servers. It also features a pluggable core that allows key modules to be replaced by third-party solutions.
Puppet Enterprise
Puppet Enterprise allows organizations to enforce the desired state of any configuration and automatically remediate any unexpected changes. It offers real-time, interactive graphical reporting and intuitive visual workflows for application deployment and orchestration. It also supports task, code and node management to enable apps and infrastructure to be managed in unison.
SaltStack
SaltStack leverages the open source Salt automation system to provide intelligent, enterprise-grade IT orchestration. Using a customized GUI and proprietary API, SaltStack improves on Salt’s communications and data ingestion capabilities to boost scalability, security and event-driven management capabilities. The platform provides integrated control of any cloud or container, providing full-stack orchestration and continuous code integration and deployment.
Arthur Cole writes about infrastructure for IT Business Edge. Cole has been covering the high-tech media and computing industries for more than 20 years, having served as editor of TV Technology, Video Technology News, Internet News and Multimedia Weekly. His contributions have appeared in Communications Today and Enterprise Networking Planet and as web content for numerous high-tech clients like TwinStrata and Carpathia. Follow Art on Twitter @acole602.
