cmueller/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Add support for EC2 dynamic data in ec2_facts (#21532)

Browse source 
* Add support for EC2 dynamic data in ec2_facts

- Flattens out JSON in the instance identity document and IAM info/credentials for easy access to facts
- This changes region fact from ‘ansible_ec2_placement_region’ to ’ansible_ec2_instance_identity_document_region’

* Maintain backwards compatibility by putting the region into the old key

* Improve JSON parsing logic and split security group IDs

* Add documentation, backwards compatibility, fix bug and formatting

- Update documentation for ec2_facts with return values
- Preserve JSON value from the metadata service for backwards compatibility
- Fix bug in fix_invalid_varnames
  - The keys in the dict were being modified in place; new dict now created to hold the sanitized keys
  - Consolidate two replace calls with a regex substitution
- Move imports for ec2_facts to the top

* Add support for parsing the IAM instance profile role
This commit is contained in:
Vinay Dandekar 2017-06-30 02:27:49 -04:00 committed by Will Thames
parent d1652aecf0
commit c884d4ab7f
1 changed files with 422 additions and 67 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

483
lib/ansible/modules/cloud/amazon/ec2_facts.py
View file

@ -24,40 +24,414 @@ ANSIBLE_METADATA = {'metadata_version': '1.0',
DOCUMENTATION = ''' DOCUMENTATION = '''
--- ---
module: ec2_facts module: ec2_facts
short_description: Gathers facts about remote hosts within ec2 (aws) short_description: Gathers facts (instance metadata) about remote hosts within ec2
version_added: "1.0" version_added: "1.0"
options: author:
validate_certs: - Silviu Dicu (@silviud)
description: - Vinay Dandekar (@roadmapper)
- If C(no), SSL certificates will not be validated. This should only be used
on personally controlled sites using self-signed certificates.
required: false
default: 'yes'
choices: ['yes', 'no']
version_added: '1.5.1'
description: description:
- This module fetches data from the metadata servers in ec2 (aws) as per - This module fetches data from the instance metadata endpoint in ec2 as per
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html.
The module must be called from within the EC2 instance itself. The module must be called from within the EC2 instance itself.
notes: notes:
- Parameters to filter on ec2_facts may be added later. - Parameters to filter on ec2_facts may be added later.
author: "Silviu Dicu (@silviud) <silviudicu@gmail.com>"
''' '''
EXAMPLES = ''' EXAMPLES = '''
# Conditional example # Gather EC2 facts
- name: Gather facts - ec2_facts:
ec2_facts:
- name: Conditional - debug:
debug:
msg: "This instance is a t1.micro" msg: "This instance is a t1.micro"
when: ansible_ec2_instance_type == "t1.micro" when: ansible_ec2_instance_type == "t1.micro"
''' '''
RETURN = '''
ansible_facts:
description: Dictionary of new facts representing discovered properties of the EC2 instance.
returned: changed
type: complex
contains:
ansible_ec2_ami_id:
description: The AMI ID used to launch the instance.
type: string
sample: "ami-XXXXXXXX"
ansible_ec2_ami_launch_index:
description:
- If you started more than one instance at the same time, this value indicates the order in which the instance was launched.
The value of the first instance launched is 0.
type: string
sample: "0"
ansible_ec2_ami_manifest_path:
description:
- The path to the AMI manifest file in Amazon S3.
If you used an Amazon EBS-backed AMI to launch the instance, the returned result is unknown.
type: string
sample: "(unknown)"
ansible_ec2_ancestor_ami_ids:
description:
- The AMI IDs of any instances that were rebundled to create this AMI.
This value will only exist if the AMI manifest file contained an ancestor-amis key.
type: string
sample: "(unknown)"
ansible_ec2_block_device_mapping_ami:
description: The virtual device that contains the root/boot file system.
type: string
sample: "/dev/sda1"
ansible_ec2_block_device_mapping_ebsN:
description:
- The virtual devices associated with Amazon EBS volumes, if any are present.
Amazon EBS volumes are only available in metadata if they were present at launch time or when the instance was last started.
The N indicates the index of the Amazon EBS volume (such as ebs1 or ebs2).
type: string
sample: "/dev/xvdb"
ansible_ec2_block_device_mapping_ephemeralN:
description: The virtual devices associated with ephemeral devices, if any are present. The N indicates the index of the ephemeral volume.
type: string
sample: "/dev/xvdc"
ansible_ec2_block_device_mapping_root:
description:
- The virtual devices or partitions associated with the root devices, or partitions on the virtual device,
where the root (/ or C) file system is associated with the given instance.
type: string
sample: "/dev/sda1"
ansible_ec2_block_device_mapping_swap:
description: The virtual devices associated with swap. Not always present.
type: string
sample: "/dev/sda2"
ansible_ec2_fws_instance_monitoring:
description: "Value showing whether the customer has enabled detailed one-minute monitoring in CloudWatch."
type: string
sample: "enabled"
ansible_ec2_hostname:
description:
- The private IPv4 DNS hostname of the instance.
In cases where multiple network interfaces are present, this refers to the eth0 device (the device for which the device number is 0).
type: string
sample: "ip-10-0-0-1.ec2.internal"
ansible_ec2_iam_info:
description:
- If there is an IAM role associated with the instance, contains information about the last time the instance profile was updated,
including the instance's LastUpdated date, InstanceProfileArn, and InstanceProfileId. Otherwise, not present.
type: complex
sample: ""
ansible_ec2_iam_info_instanceprofilearn:
description: The IAM instance profile ARN.
type: string
sample: "arn:aws:iam::<account id>:instance-profile/<role name>"
ansible_ec2_iam_info_instanceprofileid:
description: IAM instance profile ID.
type: string
sample: ""
ansible_ec2_iam_info_lastupdated:
description: IAM info last updated time.
type: string
sample: "2017-05-12T02:42:27Z"
ansible_ec2_iam_instance_profile_role:
description: IAM instance role.
type: string
sample: "role_name"
ansible_ec2_iam_security_credentials_<role name>:
description:
- If there is an IAM role associated with the instance, role-name is the name of the role,
and role-name contains the temporary security credentials associated with the role. Otherwise, not present.
type: string
sample: ""
ansible_ec2_iam_security_credentials_<role name>_accesskeyid:
description: IAM role access key ID.
type: string
sample: ""
ansible_ec2_iam_security_credentials_<role name>_code:
description: IAM code.
type: string
sample: "Success"
ansible_ec2_iam_security_credentials_<role name>_expiration:
description: IAM role credentials expiration time.
type: string
sample: "2017-05-12T09:11:41Z"
ansible_ec2_iam_security_credentials_<role name>_lastupdated:
description: IAM role last updated time.
type: string
sample: "2017-05-12T02:40:44Z"
ansible_ec2_iam_security_credentials_<role name>_secretaccesskey:
description: IAM role secret access key.
type: string
sample: ""
ansible_ec2_iam_security_credentials_<role name>_token:
description: IAM role token.
type: string
sample: ""
ansible_ec2_iam_security_credentials_<role name>_type:
description: IAM role type.
type: string
sample: "AWS-HMAC"
ansible_ec2_instance_action:
description: Notifies the instance that it should reboot in preparation for bundling.
type: string
sample: "none"
ansible_ec2_instance_id:
description: The ID of this instance.
type: string
sample: "i-XXXXXXXXXXXXXXXXX"
ansible_ec2_instance_identity_document:
description: JSON containing instance attributes, such as instance-id, private IP address, etc.
type: string
sample: ""
ansible_ec2_instance_identity_document_accountid:
description: ""
type: string
sample: "012345678901"
ansible_ec2_instance_identity_document_architecture:
description: Instance system architecture.
type: string
sample: "x86_64"
ansible_ec2_instance_identity_document_availabilityzone:
description: The Availability Zone in which the instance launched.
type: string
sample: "us-east-1a"
ansible_ec2_instance_identity_document_billingproducts:
description: Billing products for this instance.
type: string
sample: ""
ansible_ec2_instance_identity_document_devpayproductcodes:
description: Product codes for the launched AMI.
type: string
sample: ""
ansible_ec2_instance_identity_document_imageid:
description: The AMI ID used to launch the instance.
type: string
sample: "ami-01234567"
ansible_ec2_instance_identity_document_instanceid:
description: The ID of this instance.
type: string
sample: "i-0123456789abcdef0"
ansible_ec2_instance_identity_document_instancetype:
description: The type of instance.
type: string
sample: "m4.large"
ansible_ec2_instance_identity_document_kernelid:
description: The ID of the kernel launched with this instance, if applicable.
type: string
sample: ""
ansible_ec2_instance_identity_document_pendingtime:
description: The instance pending time.
type: string
sample: "2017-05-11T20:51:20Z"
ansible_ec2_instance_identity_document_privateip:
description:
- The private IPv4 address of the instance.
In cases where multiple network interfaces are present, this refers to the eth0 device (the device for which the device number is 0).
type: string
sample: "10.0.0.1"
ansible_ec2_instance_identity_document_ramdiskid:
description: The ID of the RAM disk specified at launch time, if applicable.
type: string
sample: ""
ansible_ec2_instance_identity_document_region:
description: The Region in which the instance launched.
type: string
sample: "us-east-1"
ansible_ec2_instance_identity_document_version:
description: Identity document version.
type: string
sample: "2010-08-31"
ansible_ec2_instance_identity_pkcs7:
description: Used to verify the document's authenticity and content against the signature.
type: string
sample: ""
ansible_ec2_instance_identity_rsa2048:
description: Used to verify the document's authenticity and content against the signature.
type: string
sample: ""
ansible_ec2_instance_identity_signature:
description: Data that can be used by other parties to verify its origin and authenticity.
type: string
sample: ""
ansible_ec2_instance_type:
description: The type of instance.
type: string
sample: "m4.large"
ansible_ec2_local_hostname:
description:
- The private IPv4 DNS hostname of the instance.
In cases where multiple network interfaces are present, this refers to the eth0 device (the device for which the device number is 0).
type: string
sample: "ip-10-0-0-1.ec2.internal"
ansible_ec2_local_ipv4:
description:
- The private IPv4 address of the instance.
In cases where multiple network interfaces are present, this refers to the eth0 device (the device for which the device number is 0).
type: string
sample: "10.0.0.1"
ansible_ec2_mac:
description:
- The instance's media access control (MAC) address.
In cases where multiple network interfaces are present, this refers to the eth0 device (the device for which the device number is 0).
type: string
sample: "00:11:22:33:44:55"
ansible_ec2_metrics_vhostmd:
description: Metrics.
type: string
sample: ""
ansible_ec2_network_interfaces_macs_<mac address>_device_number:
description:
- The unique device number associated with that interface. The device number corresponds to the device name;
for example, a device-number of 2 is for the eth2 device.
This category corresponds to the DeviceIndex and device-index fields that are used by the Amazon EC2 API and the EC2 commands for the AWS CLI.
type: string
sample: "0"
ansible_ec2_network_interfaces_macs_<mac address>_interface_id:
description: The elastic network interface ID.
type: string
sample: "eni-12345678"
ansible_ec2_network_interfaces_macs_<mac address>_ipv4_associations_<ip address>:
description: The private IPv4 addresses that are associated with each public-ip address and assigned to that interface.
type: string
sample: ""
ansible_ec2_network_interfaces_macs_<mac address>_ipv6s:
description: The IPv6 addresses associated with the interface. Returned only for instances launched into a VPC.
type: string
sample: ""
ansible_ec2_network_interfaces_macs_<mac address>_local_hostname:
description: The interface's local hostname.
type: string
sample: ""
ansible_ec2_network_interfaces_macs_<mac address>_local_ipv4s:
description: The private IPv4 addresses associated with the interface.
type: string
sample: ""
ansible_ec2_network_interfaces_macs_<mac address>_mac:
description: The instance's MAC address.
type: string
sample: "00:11:22:33:44:55"
ansible_ec2_network_interfaces_macs_<mac address>_owner_id:
description:
- The ID of the owner of the network interface.
In multiple-interface environments, an interface can be attached by a third party, such as Elastic Load Balancing.
Traffic on an interface is always billed to the interface owner.
type: string
sample: "01234567890"
ansible_ec2_network_interfaces_macs_<mac address>_public_hostname:
description:
- The interface's public DNS (IPv4). If the instance is in a VPC,
this category is only returned if the enableDnsHostnames attribute is set to true.
type: string
sample: "ec2-1-2-3-4.compute-1.amazonaws.com"
ansible_ec2_network_interfaces_macs_<mac address>_public_ipv4s:
description: The Elastic IP addresses associated with the interface. There may be multiple IPv4 addresses on an instance.
type: string
sample: "1.2.3.4"
ansible_ec2_network_interfaces_macs_<mac address>_security_group_ids:
description: The IDs of the security groups to which the network interface belongs. Returned only for instances launched into a VPC.
type: string
sample: "sg-01234567,sg-01234568"
ansible_ec2_network_interfaces_macs_<mac address>_security_groups:
description: Security groups to which the network interface belongs. Returned only for instances launched into a VPC.
type: string
sample: "secgroup1,secgroup2"
ansible_ec2_network_interfaces_macs_<mac address>_subnet_id:
description: The ID of the subnet in which the interface resides. Returned only for instances launched into a VPC.
type: string
sample: "subnet-01234567"
ansible_ec2_network_interfaces_macs_<mac address>_subnet_ipv4_cidr_block:
description: The IPv4 CIDR block of the subnet in which the interface resides. Returned only for instances launched into a VPC.
type: string
sample: "10.0.1.0/24"
ansible_ec2_network_interfaces_macs_<mac address>_subnet_ipv6_cidr_blocks:
description: The IPv6 CIDR block of the subnet in which the interface resides. Returned only for instances launched into a VPC.
type: string
sample: ""
ansible_ec2_network_interfaces_macs_<mac address>_vpc_id:
description: The ID of the VPC in which the interface resides. Returned only for instances launched into a VPC.
type: string
sample: "vpc-0123456"
ansible_ec2_network_interfaces_macs_<mac address>_vpc_ipv4_cidr_block:
description: The IPv4 CIDR block of the VPC in which the interface resides. Returned only for instances launched into a VPC.
type: string
sample: "10.0.0.0/16"
ansible_ec2_network_interfaces_macs_<mac address>_vpc_ipv4_cidr_blocks:
description: The IPv4 CIDR block of the VPC in which the interface resides. Returned only for instances launched into a VPC.
type: string
sample: "10.0.0.0/16"
ansible_ec2_network_interfaces_macs_<mac address>_vpc_ipv6_cidr_blocks:
description: The IPv6 CIDR block of the VPC in which the interface resides. Returned only for instances launched into a VPC.
type: string
sample: ""
ansible_ec2_placement_availability_zone:
description: The Availability Zone in which the instance launched.
type: string
sample: "us-east-1a"
ansible_ec2_placement_region:
description: The Region in which the instance launched.
type: string
sample: "us-east-1"
ansible_ec2_product_codes:
description: Product codes associated with the instance, if any.
type: string
sample: "aw0evgkw8e5c1q413zgy5pjce"
ansible_ec2_profile:
description: EC2 instance hardware profile.
type: string
sample: "default-hvm"
ansible_ec2_public_hostname:
description:
- The instance's public DNS. If the instance is in a VPC, this category is only returned if the enableDnsHostnames attribute is set to true.
type: string
sample: "ec2-1-2-3-4.compute-1.amazonaws.com"
ansible_ec2_public_ipv4:
description: The public IPv4 address. If an Elastic IP address is associated with the instance, the value returned is the Elastic IP address.
type: string
sample: "1.2.3.4"
ansible_ec2_public_key:
description: Public key. Only available if supplied at instance launch time.
type: string
sample: ""
ansible_ec2_ramdisk_id:
description: The ID of the RAM disk specified at launch time, if applicable.
type: string
sample: ""
ansible_ec2_reservation_id:
description: The ID of the reservation.
type: string
sample: "r-0123456789abcdef0"
ansible_ec2_security_groups:
description:
- The names of the security groups applied to the instance. After launch, you can only change the security groups of instances running in a VPC.
Such changes are reflected here and in network/interfaces/macs/mac/security-groups.
type: string
sample: "securitygroup1,securitygroup2"
ansible_ec2_services_domain:
description: The domain for AWS resources for the region; for example, amazonaws.com for us-east-1.
type: string
sample: "amazonaws.com"
ansible_ec2_services_partition:
description:
- The partition that the resource is in. For standard AWS regions, the partition is aws.
If you have resources in other partitions, the partition is aws-partitionname.
For example, the partition for resources in the China (Beijing) region is aws-cn.
type: string
sample: "aws"
ansible_ec2_spot_termination_time:
description:
- The approximate time, in UTC, that the operating system for your Spot instance will receive the shutdown signal.
This item is present and contains a time value only if the Spot instance has been marked for termination by Amazon EC2.
The termination-time item is not set to a time if you terminated the Spot instance yourself.
type: string
sample: "2015-01-05T18:02:00Z"
ansible_ec2_user_data:
description: The instance user data.
type: string
sample: "#!/bin/bash"
'''
import socket import socket
import re import re
import json
from ansible.module_utils._text import to_text from ansible.module_utils._text import to_text
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.urls import fetch_url, url_argument_spec
socket.setdefaulttimeout(5) socket.setdefaulttimeout(5)
@ -65,29 +439,14 @@ class Ec2Metadata(object):
ec2_metadata_uri = 'http://169.254.169.254/latest/meta-data/' ec2_metadata_uri = 'http://169.254.169.254/latest/meta-data/'
ec2_sshdata_uri = 'http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key' ec2_sshdata_uri = 'http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key'
ec2_userdata_uri = 'http://169.254.169.254/latest/user-data/' ec2_userdata_uri = 'http://169.254.169.254/latest/user-data/'
ec2_dynamicdata_uri = 'http://169.254.169.254/latest/dynamic/'
AWS_REGIONS = ('ap-northeast-1', def __init__(self, module, ec2_metadata_uri=None, ec2_sshdata_uri=None, ec2_userdata_uri=None, ec2_dynamicdata_uri=None):
'ap-northeast-2',
'ap-south-1',
'ap-southeast-1',
'ap-southeast-2',
'ca-central-1',
'eu-central-1',
'eu-west-1',
'eu-west-2',
'sa-east-1',
'us-east-1',
'us-east-2',
'us-west-1',
'us-west-2',
'us-gov-west-1',
)
def __init__(self, module, ec2_metadata_uri=None, ec2_sshdata_uri=None, ec2_userdata_uri=None):
self.module = module self.module = module
self.uri_meta = ec2_metadata_uri or self.ec2_metadata_uri self.uri_meta = ec2_metadata_uri or self.ec2_metadata_uri
self.uri_user = ec2_userdata_uri or self.ec2_userdata_uri self.uri_user = ec2_userdata_uri or self.ec2_userdata_uri
self.uri_ssh = ec2_sshdata_uri or self.ec2_sshdata_uri self.uri_ssh = ec2_sshdata_uri or self.ec2_sshdata_uri
self.uri_dynamic = ec2_dynamicdata_uri or self.ec2_dynamicdata_uri
self._data = {} self._data = {}
self._prefix = 'ansible_ec2_%s' self._prefix = 'ansible_ec2_%s'
@ -103,6 +462,8 @@ class Ec2Metadata(object):
new_fields = {} new_fields = {}
for key, value in fields.items(): for key, value in fields.items():
split_fields = key[len(uri):].split('/') split_fields = key[len(uri):].split('/')
if len(split_fields) == 3 and split_fields[0:2] == ['iam', 'security-credentials'] and '_' not in split_fields[2]:
new_fields[self._prefix % "iam-instance-profile-role"] = split_fields[2]
if len(split_fields) > 1 and split_fields[1]: if len(split_fields) > 1 and split_fields[1]:
new_key = "-".join(split_fields) new_key = "-".join(split_fields)
new_fields[self._prefix % new_key] = value new_fields[self._prefix % new_key] = value
@ -130,45 +491,43 @@ class Ec2Metadata(object):
new_uri = uri + '/' + field new_uri = uri + '/' + field
if new_uri not in self._data and not new_uri.endswith('/'): if new_uri not in self._data and not new_uri.endswith('/'):
content = self._fetch(new_uri) content = self._fetch(new_uri)
if field == 'security-groups': if field == 'security-groups' or field == 'security-group-ids':
sg_fields = ",".join(content.split('\n')) sg_fields = ",".join(content.split('\n'))
self._data['%s' % (new_uri)] = sg_fields self._data['%s' % (new_uri)] = sg_fields
else: else:
try:
dict = json.loads(content)
self._data['%s' % (new_uri)] = content self._data['%s' % (new_uri)] = content
for (key, value) in dict.items():
self._data['%s_%s' % (new_uri, key.lower())] = value
except:
self._data['%s' % (new_uri)] = content # not a stringifed JSON string
def fix_invalid_varnames(self, data): def fix_invalid_varnames(self, data):
"""Change ':'' and '-' to '_' to ensure valid template variable names""" """Change ':'' and '-' to '_' to ensure valid template variable names"""
for key in data: new_data = data.copy()
for key, value in data.items():
if ':' in key or '-' in key: if ':' in key or '-' in key:
newkey = key.replace(':', '_').replace('-', '_') newkey = re.sub(':|-', '_', key)
data[newkey] = data.pop(key) new_data[newkey] = value
del new_data[key]
def add_ec2_region(self, data): return new_data
"""Use the 'ansible_ec2_placement_availability_zone' key/value
pair to add 'ansible_ec2_placement_region' key/value pair with
the EC2 region name.
"""
# Only add a 'ansible_ec2_placement_region' key if the
# 'ansible_ec2_placement_availability_zone' exists.
zone = data.get('ansible_ec2_placement_availability_zone')
if zone is not None:
# Use the zone name as the region name unless the zone
# name starts with a known AWS region name.
region = zone
for r in self.AWS_REGIONS:
if zone.startswith(r):
region = r
break
data['ansible_ec2_placement_region'] = region
def run(self): def run(self):
self.fetch(self.uri_meta) # populate _data self.fetch(self.uri_meta) # populate _data with metadata
data = self._mangle_fields(self._data, self.uri_meta) data = self._mangle_fields(self._data, self.uri_meta)
data[self._prefix % 'user-data'] = self._fetch(self.uri_user) data[self._prefix % 'user-data'] = self._fetch(self.uri_user)
data[self._prefix % 'public-key'] = self._fetch(self.uri_ssh) data[self._prefix % 'public-key'] = self._fetch(self.uri_ssh)
self.fix_invalid_varnames(data)
self.add_ec2_region(data) self._data = {} # clear out metadata in _data
self.fetch(self.uri_dynamic) # populate _data with dynamic data
dyndata = self._mangle_fields(self._data, self.uri_dynamic)
data.update(dyndata)
data = self.fix_invalid_varnames(data)
# Maintain old key for backwards compatibility
data['ansible_ec2_placement_region'] = data['ansible_ec2_instance_identity_document_region']
return data return data
@ -186,9 +545,5 @@ def main():
module.exit_json(**ec2_facts_result) module.exit_json(**ec2_facts_result)
# import module snippets
from ansible.module_utils.basic import *
from ansible.module_utils.urls import *
if __name__ == '__main__': if __name__ == '__main__':
main() main()