How Infrastructure as Code Automates IT Operations in the Application Delivery Lifecycle

As more technologies become software-defined, their adoption demands a significant shift in thinking about how a business organizes its value stream. This shift may be difficult, but it enables you to anticipate changes and position your business to react when software-defined technologies emerge.

Recently a new set of tools and practices has emerged to create and manage environments. Known as Infrastructure as Code (IaC), it enables infrastructure management through a software-defined layer.

Application Delivery: Getting It Right
All applications run inside what we call an "environment" - a stack of hardware and software components built to support the application. This stack includes: networking, storage, virtual machines, operating systems, databases, libraries, dependencies and the application itself. Building an environment requires many activities to bring up that stack, provisioning and configuring each component according to the requirements of the application.

All of this is done to serve the application, which demands that things be "just so." The processes used to get an environment ‘just right' (and keep it that way) have been the subject of much analysis and design over the years, becoming part of the body of work known as IT Service Management (ITSM).

Infrastructure as Code
IaC replaces many of the ITSM processes involved in the deployment and ongoing management of the complete hardware-software environment in which an application will run.

While IT professionals have always used some automation such as scripting to help deploy environments, IaC is a recent development characterized by use of the following:

1. Code - At the core of IaC is the code: definition files that declare the specification for each component of the environment and how it is configured. These files might be written in YAML or JSON, and will be checked into a version control system like Git.
2. Automation tooling - Specialized tools read the definition files and use them to construct the environment and configure components according to specification.
3. Application Programming Interfaces (APIs) - Automation tools perform the actions described in the definition files against APIs. Not only will the automation tools use APIs to provision and configure the components of the environment being managed, but the tool itself will be programmable through its own API.

The development of powerful automation tools, along with the widespread proliferation of APIs, has allowed IaC to emerge as a very effective means of managing IT operations processes.

Rather than working with GUIs, scripts and command-line interfaces to perform actions, we are able to work with documents (code) that exhaustively describe the environment. These are easily shared, reviewed and versioned.

With IaC, the actions at each step are executed, not performed, and are therefore much less prone to human error. The following steps demonstrate how IaC is actually used.

Putting it to work
While setting up an environment requires a number of different components and services, we can group these into three distinct steps:

1. Provisioning - The first step is to provision the foundational infrastructure systems - servers, networks, databases and storage. Provisioning tools perform this task and are usually supplied by the infrastructure vendor. For example, Amazon provides CloudFormation to create VPCs (networks) and spin up EC2 instances (Servers) and, likewise, Azure gives us Resource Manager to create Network Security Groups and bring up virtual machines. There are also some provisioning tools like Terraform that are vendor agnostic, making switching between infrastructure vendors easier.
2. Configuration - The second step is to configure the provisioned components; configuration management tools accomplish this task. This is a broader set of tools used to perform operations like transferring files, installing services, configuring settings and so on. There are many tools in this space, but the "Big Three" are Puppet, Chef and Ansible. Each has its own advantages and disadvantages; however they all accomplish the same goal - configure the components with the required dependencies and settings.
3. Deployment - The third step is to deploy the application. More and more this involves the use of container technologies like Docker. Container technologies are a recent advancement in IT that deserve their own article and explanation; suffice it to say that a container allows an application and its dependencies to be wrapped up into a package that is easy to deploy into its own isolated space on a machine. Containers provide an additional layer of abstraction from the provisioning and configuration.

The tooling landscape used to perform these steps is highly fragmented, and strategies often use an opinionated, best-of-breed approach with a different tool at each of the three steps. For example, a team might use CloudFormation to set up the virtual machines and connect them to the network, Chef to configure and secure the virtual machines, and Docker to load the application into an isolated container.

There is no "correct" way to set up your automation stack - this will depend on the limitations of the tools and the needs of the organization - but it should be understood that adopting this technology will invariably change the way the organization manages IT work.

Anticipating Change
IaC presents us with a large and increasingly complex software-defined layer that is used to perform infrastructure management functions. It is important to note, however, that what becomes software-defined here is not the infrastructure itself, although software-defined infrastructure is a prerequisite for IaC; rather they are the systems and processes that are used to manage the infrastructure, such as asset management, change management, configuration management and more. These functions can now be emulated in code.

This can create major challenges for traditional service management strategies - the skills, roles, responsibilities, methods and practices used to manage infrastructure change considerably. On the other hand, it creates great opportunities by providing a catalyst to launch DevOps initiatives and increase the scope of Agile practices across the value stream.

By understanding IaC as a software-defined technology, we can gain insight into the impact on the enterprise.

Redefining IT
As a software-defined technology, we should immediately expect that IaC will be highly impactful on the business and how it is organized. It is important to recognize IaC as a software-defined technology, because looking at it through this lens gives us an understanding of what to expect as we start to adopt this technology in the enterprise. Analysis using three key attributes of software definition will shed some light on the impacts:

1. Abstraction
IaC requires that all operations on infrastructure are declared in definition files and executed using an automation tool. This automation layer provides an abstraction from operations like deploying, configuring and managing components. This means that these operations shift left in the software supply chain - they are performed earlier and all together, rather than sequentially at the final stages of activity.

With this abstraction, the skill specialization for managing infrastructure shifts from traditional vendor and application specific sysadmin skills to the ability to write code and think through the abstraction. The roles and responsibilities for managing infrastructure can move to anyone proficient in writing code.

2. Control
Since infrastructure can be fully documented in code, we can "read" the environment - see everything that was deployed and how it was configured by simply reading the definition files.

The focus of service management and control systems therefore shifts to managing the automation tooling and definition files. For example, use of a version control system to manage the definition files brings about the idea of "versioned infrastructure," where the change record is reflected in the version history.

Similarly, change management can be accomplished through code reviews performed individually when the changes are checked in, rather than putting batched changes before a change review board.

3. Mutability
The environment is exhaustively described in definition files, which are not dependent on infrastructure attributes. Ideally, the infrastructure itself is "immutable" - no changes are made directly to the infrastructure once it is deployed. Infrastructure components are locked down and not directly accessible to humans - changes are deployed only with the automation tooling.

This has two impacts: first, it introduces commoditization to deployment process. The same definition file can be used to bring up one server or a hundred. Additional assets can be deployed as needed, just-in-time, and torn down when they are no longer required. Elastic assets means infrastructure is always ‘right-sized.'

Second, the underlying infrastructure becomes modular. We can use our definition file to bring up our components on-premise, or in the cloud - on AWS or on Azure. While there are some dependencies involved in the automation tooling, overall the infrastructure layer has few enough hardware or vendor dependencies that operators have more freedom to choose where to host their infrastructure.

Adoption Through Transformation
For a business that relies on a software supply chain to deliver value to customers, IaC represents a significant opportunity to increase efficiency, lower costs and reduce risk.

Further, IaC has emerged as a vital driver in transforming and modernizing the software supply chain. Digital-first businesses like Amazon, Netflix and Facebook live and breathe software-defined infrastructure. This is because they have been able to build their businesses, cultures and value streams in a greenfield without the encumbrance of legacy systems and practices.

As with all software-defined technologies, incumbent enterprises that have a well-established value stream will have difficulty with wide-scale adoption. Some of the challenges they face include:

• Culture: All software-defined technologies present the significant challenge of redefining roles, shifting responsibilities and altering work structures. Part of the transformation to adopt these technologies, therefore, involves a cultural change across the technical side of the organization. DevOps, as a culture, has emerged from this, and embraces these new roles and responsibilities. This needs to be nurtured and allowed to grow through a process of sharing and collaborating.
• Practices: IT professionals are typically accustomed to working within project-based work structures like PMP and Prince2. IaC, however, begs for the use of software development practices like Agile to manage work. Implementing a software-defined infrastructure will have the effect of proliferating management techniques like Scrum and Kanban. This is a great opportunity, but equally a challenge to get everyone onboard and trained on the new methodologies and the tools they use.
• Value: A fundamental characteristic of software-defined technologies is that they recast management strategies. With IaC, the IT supply chain is altered beyond recognition, requiring new thinking about service management strategies. The changes in where, when and how infrastructure will be managed and deployed mean organizations need to undergo a paradigm shift in thinking about how value delivery is organized.

In order to fully adopt software-defined infrastructure, incumbent enterprises will need to be ready to undergo a transformation. There needs to be a commitment and willingness to invest in new tools, new skills and new relationships.

Automation of any kind is a threat to the status quo, and change often brings on a crisis of identity for those who are content to stay the same. In this case, IT automation through IaC challenges the processes and systems used by ITSM. However, IT leaders will do well to be open-minded. Through a process of experimentation and discovery we can ease the adoption of this software-defined technology and use it as a catalyst to grow the value of IT across the business.