ansible/cloud/amazon/GUIDELINES.md

Guidelines for AWS modules

Getting Started

Since Ansible 2.0, it is a requirement that all new AWS modules are written to use boto3.

Prior to 2.0, modules may of been written in boto or boto3. Modules written using boto can continue to be extended using boto.

Backward compatibility of older modules must be maintained.

Bug fixing

If you are writing a bugfix for a module that uses boto, you should continue to use boto to maintain backward compatibility.

If you are adding new functionality to an existing module that uses boto but the new functionality requires boto3, you must maintain backward compatibility of the module and ensure the module still works without boto3.

Naming your module

Base the name of the module on the part of AWS that you actually use. (A good rule of thumb is to take whatever module you use with boto as a starting point).

Don't further abbreviate names - if something is a well known abbreviation due to it being a major component of AWS, that's fine, but don't create new ones independently (e.g. VPC, ELB, etc. are fine)

Adding new features

Try and keep backward compatibility with relatively recent versions of boto. That means that if want to implement some functionality that uses a new feature of boto, it should only fail if that feature actually needs to be run, with a message saying which version of boto is needed.

Use feature testing (e.g. hasattr('boto.module', 'shiny_new_method')) to check whether boto supports a feature rather than version checking

e.g. from the ec2 module:

if boto_supports_profile_name_arg(ec2):
    params['instance_profile_name'] = instance_profile_name
else:
    if instance_profile_name is not None:
        module.fail_json(msg="instance_profile_name parameter requires boto version 2.5.0 or higher")

Using boto and boto3

Importing

Wrap import statements in a try block and fail the module later if the import fails

boto

try:
    import boto.ec2
    from boto.exception import BotoServerError
    HAS_BOTO = True
except ImportError:
    HAS_BOTO = False

def main():

    if not HAS_BOTO:
        module.fail_json(msg='boto required for this module')

boto3

try:
    import boto3
    HAS_BOTO3 = True
except ImportError:
    HAS_BOTO3 = False

def main():

    if not HAS_BOTO3:
        module.fail_json(msg='boto3 required for this module')

boto and boto3 combined

If you want to add boto3 functionality to a module written using boto, you must maintain backward compatibility. Ensure that you clearly document if a new parameter requires boto3. Import boto3 at the top of the module as normal and then use the HAS_BOTO3 bool when necessary, before the new feature.

try:
    import boto
    HAS_BOTO = True
except ImportError:
    HAS_BOTO = False

try:
    import boto3
    HAS_BOTO3 = True
except ImportError:
    HAS_BOTO3 = False

if my_new_feauture_Parameter_is_set:
    if HAS_BOTO3:
        # do feature
    else:
        module.fail_json(msg="boto3 is required for this feature")

Connecting to AWS

To connect to AWS, you should use get_aws_connection_info and then connect_to_aws.

The reason for using get_aws_connection_info and connect_to_aws rather than doing it yourself is that they handle some of the more esoteric connection options such as security tokens and boto profiles.

Some boto services require region to be specified. You should check for the region parameter if required.

boto

An example of connecting to ec2:

region, ec2_url, aws_connect_params = get_aws_connection_info(module)
if region:
    try:
        connection = connect_to_aws(boto.ec2, region, **aws_connect_params)
    except (boto.exception.NoAuthHandlerFound, AnsibleAWSError), e:
        module.fail_json(msg=str(e))
else:
    module.fail_json(msg="region must be specified")

boto3

An example of connecting to ec2 is shown below. Note that there is no 'NoAuthHandlerFound' exception handling like in boto. Instead, an AuthFailure exception will be thrown when you use 'connection'. See exception handling.

region, ec2_url, aws_connect_params = get_aws_connection_info(module, boto3=True)
if region:
    connection = boto3_conn(module, conn_type='client', resource='ec2', region=region, endpoint=ec2_url, **aws_connect_params)
else:
    module.fail_json(msg="region must be specified")

Exception Handling

You should wrap any boto call in a try block. If an exception is thrown, it is up to you decide how to handle it but usually calling fail_json with the error message will suffice.

boto

# Import BotoServerError
try:
    import boto.ec2
    from boto.exception import BotoServerError
    HAS_BOTO = True
except ImportError:
    HAS_BOTO = False

# Connect to AWS
...

# Make a call to AWS
try:
    result = connection.aws_call()
except BotoServerError, e:
    module.fail_json(msg=e.message)

boto3

For more information on botocore exception handling see [http://botocore.readthedocs.org/en/latest/client_upgrades.html#error-handling]

Boto3 provides lots of useful info when an exception is thrown so pass this to the user along with the message.

# Import ClientError from botocore
try:
    from botocore.exceptions import ClientError
    HAS_BOTO3 = True
except ImportError:
    HAS_BOTO3 = False

# Connect to AWS
...

# Make a call to AWS
try:
    result = connection.aws_call()
except ClientError, e:
    module.fail_json(msg=e.message, **camel_dict_to_snake_dict(e.response))

If you need to perform an action based on the error boto3 returned, use the error code.

# Make a call to AWS
try:
    result = connection.aws_call()
except ClientError, e:
    if e.response['Error']['Code'] == 'NoSuchEntity':
        return None
    else:
        module.fail_json(msg=e.message, **camel_dict_to_snake_dict(e.response))

Returning Values

When you make a call using boto3, you will probably get back some useful information that you should return in the module.

As well as information related to the call itself, you will also have some response metadata. It is OK to return this to the user as well as they may find it useful.

Boto3 returns all values CamelCased. Ansible follows Python standards for variable names and uses snake_case. There is a helper function in module_utils/ec2.py called camel_dict_to_snake_dict that allows you to easily convert the boto3 response to snake_case.

You should use this helper function and avoid changing the names of values returned by Boto3. E.g. if boto3 returns a value called 'SecretAccessKey' do not change it to 'AccessKey'.

# Make a call to AWS
result = connection.aws_call()

# Return the result to the user
module.exit_json(changed=True, **camel_dict_to_snake_dict(result))

Helper functions

Along with the connection functions in Ansible ec2.py module_utils, there are some other useful functions detailed below.

camel_dict_to_snake_dict

boto3 returns results in a dict. The keys of the dict are in CamelCase format. In keeping with Ansible format, this function will convert the keys to snake_case.

ansible_dict_to_boto3_filter_list

Converts a an Ansible list of filters to a boto3 friendly list of dicts. This is useful for any boto3 _facts modules.

boto3_tag_list_to_ansible_dict

Converts a boto3 tag list to an Ansible dict. Boto3 returns tags as a list of dicts containing keys called 'Key' and 'Value'. This function converts this list in to a single dict where the dict key is the tag key and the dict value is the tag value.

ansible_dict_to_boto3_tag_list

Opposite of above. Converts an Ansible dict to a boto3 tag list of dicts.

get_ec2_security_group_ids_from_names

Pass this function a list of security group names or combination of security group names and IDs and this function will return a list of IDs. You should also pass the VPC ID if known because security group names are not necessarily unique across VPCs.