Ansible is a great software orchestration tool that I’ve enjoyed using for a while. A core concept in Ansible is the inventory: a description of all the hosts available for deployments. The inventory is not only a list of hosts, but provides ways to group hosts into logical named units. The names of the groups can describe what role the hosts play in your architecture, or their geographical location or something else. A simple inventory can look like this:

# file: prod 

[mysql-servers]
10.10.1.1 

[web-applications]
10.10.1.2 
10.10.1.3

This inventory has one host in the mysql-servers group, and two hosts in the web-applications group. This communicates that there is one host for running a MySQL server and two hosts for running a front-end web application.

Now we can write a simple playbook to configure the MySQL hosts:

---
# file: mysql-servers.yml
- hosts: mysql-servers
  roles:
    - mysql-server

Notice how we reference the logical inventory group mysql-servers in the playbook. We can then configure the MySQL hosts by running the Ansible playbook:

$ ansible-playbook -i prod mysql-servers.yml 

This is fairly straightforward and the naming makes everything easy to understand. Importantly, the playbook is generic and can be reused for a different deployment environment were we also need MySQL servers. This as long as the new inventory file contains a similar organization and grouping as the first one, using the same names.

Amazon EC2

Amazon Elastic Compute Cloud (EC2) is a popular platform for deploying applications and services. When managing a larger infrastructure, in particular a dynamic one, it can get difficult to manually manage the inventory file and keep track of all the different hosts. Ansible supports dynamic inventories that help with exactly this problem. In short, instead of referencing a simple text inventory file when running playbooks, you can give Ansible a script as the inventory. The task of this script is to figure out the current state of the infrastructure and return the inventory in a particular format that Ansible understands (a JSON object).

Ansible provides some example scripts for common platforms, including Amazon’s EC2. This script (ec2.py) returns all your EC2 hosts using different kinds of grouping concepts, including EC2 regions, instance types, and EC2 instance tags (key-value pairs). If I create an EC2 instance with the tags Name=mysql1 and Group=mysql-servers, the Ansible EC2 inventory script would return something like this:

$ ec2.py --list 
{
    ...
    "us-east-1": [
        "54.165.178.11"
    ], 
    "tag_Name_mysql1": [
        "54.165.178.11"
    ], 
    "type_m3_medium": [
        "54.165.178.11"
    ], 
    "ec2": [
        "54.165.178.11"
    ], 
    "tag_Group_mysql_servers": [
        "54.165.178.11"
    ]
}

Now I can reference all my instances in a particular region, or by instance type, or by my tags. But to configure my MySQL servers on EC2 using my very simple and supposedly reusable playbook from above, I need to change it to this:

---
# file: mysql-servers.yml
- hosts: tag_Group_mysql_servers
  roles:
    - role: mysql-server

However, I don’t want to have a separate set of playbooks for different platforms. There are several options when it comes to resolving this issue. One solution is to define a text-based inventory that does the mapping for us, something like this:

# file: prod-mapping 

[mysql-servers:children]
tag_Group_mysql_servers

Now you can reference mysql-servers in your playbooks and it will include all the hosts in the tag_Group_mysql_servers group. However, this has to be done for all groups, so you have to do some work in managing and maintaining the mapping.

The solution I have taken so far is to have a very simple script in front of the bundled ec2.py inventory script that allows us to augment the grouping of hosts. I’ve called this script ec2-augment.py and it can be configured to create additional host groups based on the instance tags. For example, for the group tag_Group_mysql_servers to add the same group of hosts under the name mysql-servers. So, if my script is used as the inventory script, it would return the following:

$ ec2-augment.py --list 
{
    ...
    "us-east-1": [
        "54.165.178.11"
    ], 
    "tag_Name_mysql1": [
        "54.165.178.11"
    ], 
    "type_m3_medium": [
        "54.165.178.11"
    ], 
    "ec2": [
        "54.165.178.11"
    ], 
    "mysql-servers": [
        "54.165.178.11"
    ], 
    "tag_Group_mysql_servers": [
        "54.165.178.11"
    ]
}

This will work as long as I tag all my EC2 instances with the key Group and the value of the group name I want. For example, Group=mysql-servers, or Group=web-applications. This is easy to do consistently if you create all EC2 instances using the Ansible ec2 module.

I can now run my existing playbooks against EC2:

$ ansible-playbook -i ec2-augment.py mysql-servers.yml 

I’ve published the current version of the script as a Gist.