cmueller/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Merge pull request #6171 from joshuaconner/update_to_docker-py_0.3.0

Browse source 
docker: update to docker-py 0.3.0, plus full port binding support etc. from #5639
This commit is contained in:
jctanner 2014-02-28 08:31:56 -05:00
commit 42a2586dfa
1 changed files with 257 additions and 71 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

328
library/cloud/docker
View file

@ -1,6 +1,7 @@
#!/usr/bin/python #!/usr/bin/python
# (c) 2013, Cove Schneider # (c) 2013, Cove Schneider
# (c) 2014, Joshua Conner <joshua.conner@gmail.com>
# (c) 2014, Pavel Antonov <antonov@adwz.ru> # (c) 2014, Pavel Antonov <antonov@adwz.ru>
# #
# This file is part of Ansible, # This file is part of Ansible,
@ -52,18 +53,28 @@ options:
required: false required: false
default: null default: null
aliases: [] aliases: []
version_added: "1.5"
ports: ports:
description: description:
- Set private to public port mapping specification (e.g. ports=22,80 or ports=:8080 maps 8080 directly to host) - Set private to public port mapping specification using docker CLI-style syntax [([<host_interface>:[host_port]])|(<host_port>):]<container_port>[/udp]
required: false required: false
default: null default: null
aliases: [] aliases: []
links: version_added: "1.5"
expose:
description: description:
- Add links to another container (e.g. links=redis,postgresql:db) - Set container ports to expose for port mappings or links. (If the port is already exposed using EXPOSE in a Dockerfile, you don't need to expose it again.)
required: false required: false
default: null default: null
aliases: [] aliases: []
version_added: "1.5"
publish_all_ports:
description:
- Publish all exposed ports to the host interfaces
required: false
default: false
aliases: []
version_added: "1.5"
volumes: volumes:
description: description:
- Set volume(s) to mount on the container - Set volume(s) to mount on the container
@ -76,6 +87,13 @@ options:
required: false required: false
default: null default: null
aliases: [] aliases: []
links:
description:
- Link container(s) to other container(s) (e.g. links=redis,postgresql:db)
required: false
default: null
aliases: []
version_added: "1.5"
memory_limit: memory_limit:
description: description:
- Set RAM allocated to container - Set RAM allocated to container
@ -144,8 +162,15 @@ options:
required: false required: false
default: default:
aliases: [] aliases: []
author: Cove Schneider, Pavel Antonov name:
requirements: [ "docker-py" ] description:
- Set the name of the container (cannot use with count)
required: false
default: null
aliases: []
version_added: "1.5"
author: Cove Schneider, Joshua Conner, Pavel Antonov
requirements: [ "docker-py >= 0.3.0" ]
''' '''
EXAMPLES = ''' EXAMPLES = '''
@ -156,7 +181,7 @@ on the host:
sudo: yes sudo: yes
tasks: tasks:
- name: run tomcat servers - name: run tomcat servers
docker: image=centos command="service tomcat6 start" ports=:8080 docker: image=centos command="service tomcat6 start" ports=8080
The tomcat server's port is NAT'ed to a dynamic port on the host, but you can determine which port the server was The tomcat server's port is NAT'ed to a dynamic port on the host, but you can determine which port the server was
mapped to using docker_containers: mapped to using docker_containers:
@ -167,7 +192,7 @@ mapped to using docker_containers:
- name: run tomcat servers - name: run tomcat servers
docker: image=centos command="service tomcat6 start" ports=8080 count=5 docker: image=centos command="service tomcat6 start" ports=8080 count=5
- name: Display IP address and port mappings for containers - name: Display IP address and port mappings for containers
debug: msg={{inventory_hostname}}:{{item.NetworkSettings.Ports['8080/tcp'][0].HostPort}} debug: msg={{inventory_hostname}}:{{item['HostConfig']['PortBindings']['8080/tcp'][0]['HostPort']}}
with_items: docker_containers with_items: docker_containers
Just as in the previous example, but iterates over the list of docker containers with a sequence: Just as in the previous example, but iterates over the list of docker containers with a sequence:
@ -180,7 +205,7 @@ Just as in the previous example, but iterates over the list of docker containers
- name: run tomcat servers - name: run tomcat servers
docker: image=centos command="service tomcat6 start" ports=8080 count={{start_containers_count}} docker: image=centos command="service tomcat6 start" ports=8080 count={{start_containers_count}}
- name: Display IP address and port mappings for containers - name: Display IP address and port mappings for containers
debug: msg={{inventory_hostname}}:{{docker_containers[{{item}}].NetworkSettings.Ports['8080/tcp'][0].HostPort}}" debug: msg="{{inventory_hostname}}:{{docker_containers[{{item}}]['HostConfig']['PortBindings']['8080/tcp'][0]['HostPort']}}"
with_sequence: start=0 end={{start_containers_count - 1}} with_sequence: start=0 end={{start_containers_count - 1}}
Stop, remove all of the running tomcat containers and list the exit code from the stopped containers: Stop, remove all of the running tomcat containers and list the exit code from the stopped containers:
@ -193,16 +218,79 @@ Stop, remove all of the running tomcat containers and list the exit code from th
- name: Display return codes from stopped containers - name: Display return codes from stopped containers
debug: msg="Returned {{inventory_hostname}}:{{item}}" debug: msg="Returned {{inventory_hostname}}:{{item}}"
with_items: docker_containers with_items: docker_containers
Create a named container:
- hosts: web
sudo: yes
tasks:
- name: run tomcat server
docker: image=centos name=tomcat command="service tomcat6 start" ports=8080
Create multiple named containers:
- hosts: web
sudo: yes
tasks:
- name: run tomcat servers
docker: image=centos name={{item}} command="service tomcat6 start" ports=8080
with_items:
- crookshank
- snowbell
- heathcliff
- felix
- sylvester
Create containers named in a sequence:
- hosts: web
sudo: yes
tasks:
- name: run tomcat servers
docker: image=centos name={{item}} command="service tomcat6 start" ports=8080
with_sequence: start=1 end=5 format=tomcat_%d.example.com
Create two linked containers:
- hosts: web
sudo: yes
tasks:
- name: ensure redis container is running
docker: image=crosbymichael/redis name=redis
- name: ensure redis_ambassador container is running
docker: image=svendowideit/ambassador ports=6379:6379 links=redis:redis name=redis_ambassador_ansible
Create containers with options specified as key-value pairs and lists:
- hosts: web
sudo: yes
tasks:
- docker:
image: namespace/image_name
links:
- postgresql:db
- redis:redis
Create containers with options specified as strings and lists as comma-separated strings:
- hosts: web
sudo: yes
tasks:
docker: image=namespace/image_name links=postgresql:db,redis:redis
''' '''
HAS_DOCKER_PY = True
import sys
from urlparse import urlparse
try: try:
import sys
import docker.client import docker.client
from requests.exceptions import * from requests.exceptions import *
from urlparse import urlparse
except ImportError, e: except ImportError, e:
print "failed=True msg='failed to import python module: %s'" % e HAS_DOCKER_PY = False
sys.exit(1)
def _human_to_bytes(number): def _human_to_bytes(number):
suffixes = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'] suffixes = ['B', 'KB', 'MB', 'GB', 'TB', 'PB']
@ -232,20 +320,18 @@ def _docker_id_quirk(inspect):
return inspect return inspect
class DockerManager: class DockerManager:
counters = {'created':0, 'started':0, 'stopped':0, 'killed':0, 'removed':0, 'restarted':0, 'pull':0} counters = {'created':0, 'started':0, 'stopped':0, 'killed':0, 'removed':0, 'restarted':0, 'pull':0}
def __init__(self, module): def __init__(self, module):
self.module = module self.module = module
self.binds = None self.binds = None
self.volumes = None self.volumes = None
if self.module.params.get('volumes'): if self.module.params.get('volumes'):
self.binds = {} self.binds = {}
self.volumes = {} self.volumes = {}
vols = self.module.params.get('volumes') vols = self.parse_list_from_param('volumes')
if not isinstance(vols, list):
vols = vols.split(" ")
for vol in vols: for vol in vols:
parts = vol.split(":") parts = vol.split(":")
# host mount (e.g. /mnt:/tmp, bind mounts host's /tmp to /mnt in the container) # host mount (e.g. /mnt:/tmp, bind mounts host's /tmp to /mnt in the container)
@ -259,50 +345,106 @@ class DockerManager:
self.lxc_conf = None self.lxc_conf = None
if self.module.params.get('lxc_conf'): if self.module.params.get('lxc_conf'):
self.lxc_conf = [] self.lxc_conf = []
options = self.module.params.get('lxc_conf').split(" ") options = self.parse_list_from_param('lxc_conf')
for option in options: for option in options:
parts = option.split(':') parts = option.split(':')
self.lxc_conf.append({"Key": parts[0], "Value": parts[1]}) self.lxc_conf.append({"Key": parts[0], "Value": parts[1]})
self.ports = None self.exposed_ports = None
if self.module.params.get('ports'): if self.module.params.get('expose'):
ports = self.module.params.get('ports') expose = self.parse_list_from_param('expose')
self.ports = ports if isinstance(ports, list) else ports.split(",") self.exposed_ports = self.get_exposed_ports(expose)
self.port_bindings = None self.port_bindings = None
if self.module.params.get('port_bindings'): if self.module.params.get('ports'):
self.port_bindings = {} ports = self.parse_list_from_param('ports')
bindings = self.module.params.get('port_bindings') self.port_bindings = self.get_port_bindings(ports)
bindings = bindings if isinstance(bindings, list) else bindings.split(",")
for binding in bindings:
parts = binding.split(":")
if len(parts) == 3:
self.port_bindings[parts[0]] = (parts[1], parts[2])
elif len(parts) == 2:
self.port_bindings[parts[0]] = parts[1]
else:
self.port_bindings[parts[0]] = None
self.links = None self.links = None
if self.module.params.get('links'): if self.module.params.get('links'):
self.links = {} links = self.parse_list_from_param('links')
links = self.module.params.get('links') self.links = dict(map(lambda x: x.split(':'), links))
if not isinstance(links, list):
links = links.split(",")
for link in links:
parts = link.split(":")
self.links[parts[0]] = parts[1] if len(parts) == 2 else parts[0]
self.env = None self.env = None
if self.module.params.get('env'): if self.module.params.get('env'):
env = self.module.params.get('env') env = self.parse_list_from_param('env')
self.env = dict(map(lambda x: x.split("="), env if isinstance(env, list) else env.split(","))) self.env = dict(map(lambda x: x.split("="), env))
# connect to docker server # connect to docker server
docker_url = urlparse(module.params.get('docker_url')) docker_url = urlparse(module.params.get('docker_url'))
self.client = docker.Client(base_url=docker_url.geturl()) self.client = docker.Client(base_url=docker_url.geturl())
def parse_list_from_param(self, param_name, delimiter=','):
"""
Get a list from a module parameter, whether it's specified as a delimiter-separated string or is already in list form.
"""
param_list = self.module.params.get(param_name)
if not isinstance(param_list, list):
param_list = param_list.split(delimiter)
return param_list
def get_exposed_ports(self, expose_list):
"""
Parse the ports and protocols (TCP/UDP) to expose in the docker-py `create_container` call from the docker CLI-style syntax.
"""
if expose_list:
exposed = []
for port in expose_list:
if port.endswith('/tcp') or port.endswith('/udp'):
port_with_proto = tuple(port.split('/'))
else:
# assume tcp protocol if not specified
port_with_proto = (port, 'tcp')
exposed.append(port_with_proto)
return exposed
else:
return None
def get_port_bindings(self, ports):
"""
Parse the `ports` string into a port bindings dict for the `start_container` call.
"""
binds = {}
for port in ports:
parts = port.split(':')
container_port = parts[-1]
if '/' not in container_port:
container_port = int(parts[-1])
p_len = len(parts)
if p_len == 1:
# Bind `container_port` of the container to a dynamically
# allocated TCP port on all available interfaces of the host
# machine.
bind = ('0.0.0.0',)
elif p_len == 2:
# Bind `container_port` of the container to port `parts[0]` on
# all available interfaces of the host machine.
bind = ('0.0.0.0', int(parts[0]))
elif p_len == 3:
# Bind `container_port` of the container to port `parts[1]` on
# IP `parts[0]` of the host machine. If `parts[1]` empty bind
# to a dynamically allocacted port of IP `parts[0]`.
bind = (parts[0], int(parts[1])) if parts[1] else (parts[0],)
if container_port in binds:
old_bind = binds[container_port]
if isinstance(old_bind, list):
# append to list if it already exists
old_bind.append(bind)
else:
# otherwise create list that contains the old and new binds
binds[container_port] = [binds[container_port], bind]
else:
binds[container_port] = bind
return binds
def get_split_image_tag(self, image): def get_split_image_tag(self, image):
if '/' in image: if '/' in image:
image = image.split('/')[1] image = image.split('/')[1]
@ -310,15 +452,15 @@ class DockerManager:
if image.find(':') > 0: if image.find(':') > 0:
return image.split(':') return image.split(':')
else: else:
return image, tag return image, tag
def get_summary_counters_msg(self): def get_summary_counters_msg(self):
msg = "" msg = ""
for k, v in self.counters.iteritems(): for k, v in self.counters.iteritems():
msg = msg + "%s %d " % (k, v) msg = msg + "%s %d " % (k, v)
return msg return msg
def increment_counter(self, name): def increment_counter(self, name):
self.counters[name] = self.counters[name] + 1 self.counters[name] = self.counters[name] + 1
@ -328,7 +470,7 @@ class DockerManager:
return True return True
return False return False
def get_inspect_containers(self, containers): def get_inspect_containers(self, containers):
inspect = [] inspect = []
for i in containers: for i in containers:
@ -378,8 +520,7 @@ class DockerManager:
def create_containers(self, count=1): def create_containers(self, count=1):
params = {'image': self.module.params.get('image'), params = {'image': self.module.params.get('image'),
'command': self.module.params.get('command'), 'command': self.module.params.get('command'),
'name': self.module.params.get('name'), 'ports': self.exposed_ports,
'ports': self.ports,
'volumes': self.volumes, 'volumes': self.volumes,
'volumes_from': self.module.params.get('volumes_from'), 'volumes_from': self.module.params.get('volumes_from'),
'mem_limit': _human_to_bytes(self.module.params.get('memory_limit')), 'mem_limit': _human_to_bytes(self.module.params.get('memory_limit')),
@ -387,7 +528,7 @@ class DockerManager:
'dns': self.module.params.get('dns'), 'dns': self.module.params.get('dns'),
'hostname': self.module.params.get('hostname'), 'hostname': self.module.params.get('hostname'),
'detach': self.module.params.get('detach'), 'detach': self.module.params.get('detach'),
'privileged': self.module.params.get('privileged'), 'name': self.module.params.get('name'),
} }
def do_create(count, params): def do_create(count, params):
@ -409,10 +550,17 @@ class DockerManager:
return containers return containers
def start_containers(self, containers): def start_containers(self, containers):
params = {
'lxc_conf': self.lxc_conf,
'binds': self.binds,
'port_bindings': self.port_bindings,
'publish_all_ports': self.module.params.get('publish_all_ports'),
'privileged': self.module.params.get('privileged'),
'links': self.links,
}
for i in containers: for i in containers:
self.client.start(i['Id'], lxc_conf=self.lxc_conf, binds=self.binds, self.client.start(i['Id'], **params)
port_bindings = self.port_bindings, links=self.links) self.increment_counter('started')
self.increment_counter('started')
def stop_containers(self, containers): def stop_containers(self, containers):
for i in containers: for i in containers:
@ -425,7 +573,7 @@ class DockerManager:
for i in containers: for i in containers:
self.client.remove_container(i['Id']) self.client.remove_container(i['Id'])
self.increment_counter('removed') self.increment_counter('removed')
def kill_containers(self, containers): def kill_containers(self, containers):
for i in containers: for i in containers:
self.client.kill(i['Id']) self.client.kill(i['Id'])
@ -436,18 +584,44 @@ class DockerManager:
self.client.restart(i['Id']) self.client.restart(i['Id'])
self.increment_counter('restarted') self.increment_counter('restarted')
def check_dependencies(module):
"""
Ensure `docker-py` >= 0.3.0 is installed, and call module.fail_json with a
helpful error message if it isn't.
"""
if not HAS_DOCKER_PY:
module.fail_json(msg="`docker-py` doesn't seem to be installed, but is required for the Ansible Docker module.")
else:
HAS_NEW_ENOUGH_DOCKER_PY = False
if hasattr(docker, '__version__'):
# a '__version__' attribute was added to the module but not until
# after 0.3.0 was added pushed to pip. If it's there, use it.
if docker.__version__ >= '0.3.0':
HAS_NEW_ENOUGH_DOCKER_PY = True
else:
# HACK: if '__version__' isn't there, we check for the existence of
# `_get_raw_response_socket` in the docker.Client class, which was
# added in 0.3.0
if hasattr(docker.Client, '_get_raw_response_socket'):
HAS_NEW_ENOUGH_DOCKER_PY = True
if not HAS_NEW_ENOUGH_DOCKER_PY:
module.fail_json(msg="The Ansible Docker module requires `docker-py` >= 0.3.0.")
def main(): def main():
module = AnsibleModule( module = AnsibleModule(
argument_spec = dict( argument_spec = dict(
count = dict(default=1), count = dict(default=1),
image = dict(required=True), image = dict(required=True),
command = dict(required=False, default=None), command = dict(required=False, default=None),
name = dict(required=False, default=None), expose = dict(required=False, default=None),
ports = dict(required=False, default=None), ports = dict(required=False, default=None),
port_bindings = dict(required=False, default=None), publish_all_ports = dict(default=False, type='bool'),
links = dict(required=False, default=None),
volumes = dict(default=None), volumes = dict(default=None),
volumes_from = dict(default=None), volumes_from = dict(default=None),
links = dict(default=None),
memory_limit = dict(default=0), memory_limit = dict(default=0),
memory_swap = dict(default=0), memory_swap = dict(default=0),
docker_url = dict(default='unix://var/run/docker.sock'), docker_url = dict(default='unix://var/run/docker.sock'),
@ -461,18 +635,24 @@ def main():
state = dict(default='present', choices=['absent', 'present', 'stopped', 'killed', 'restarted']), state = dict(default='present', choices=['absent', 'present', 'stopped', 'killed', 'restarted']),
debug = dict(default=False, type='bool'), debug = dict(default=False, type='bool'),
privileged = dict(default=False, type='bool'), privileged = dict(default=False, type='bool'),
lxc_conf = dict(default=None) lxc_conf = dict(default=None),
name = dict(default=None)
) )
) )
check_dependencies(module)
try: try:
manager = DockerManager(module) manager = DockerManager(module)
state = module.params.get('state') state = module.params.get('state')
count = int(module.params.get('count')) count = int(module.params.get('count'))
name = module.params.get('name')
if count < 0:
module.fail_json(msg="Count must be greater than zero")
if count > 1 and name:
module.fail_json(msg="Count and name must not be used together")
if count < 1:
module.fail_json(msg="Count must be positive number")
running_containers = manager.get_running_containers() running_containers = manager.get_running_containers()
running_count = len(running_containers) running_count = len(running_containers)
delta = count - running_count delta = count - running_count
@ -483,11 +663,17 @@ def main():
# start/stop containers # start/stop containers
if state == "present": if state == "present":
# make sure a container with `name` is running
if name and "/" + name not in map(lambda x: x.get('Name'), running_containers):
containers = manager.create_containers(1)
manager.start_containers(containers)
# start more containers if we don't have enough # start more containers if we don't have enough
if delta > 0: elif delta > 0:
containers = manager.create_containers(delta) containers = manager.create_containers(delta)
manager.start_containers(containers) manager.start_containers(containers)
# stop containers if we have too many # stop containers if we have too many
elif delta < 0: elif delta < 0:
containers_to_stop = running_containers[0:abs(delta)] containers_to_stop = running_containers[0:abs(delta)]
@ -495,20 +681,20 @@ def main():
manager.remove_containers(containers_to_stop) manager.remove_containers(containers_to_stop)
facts = manager.get_running_containers() facts = manager.get_running_containers()
# stop and remove containers # stop and remove containers
elif state == "absent": elif state == "absent":
facts = manager.stop_containers(deployed_containers) facts = manager.stop_containers(deployed_containers)
manager.remove_containers(deployed_containers) manager.remove_containers(deployed_containers)
# stop containers # stop containers
elif state == "stopped": elif state == "stopped":
facts = manager.stop_containers(running_containers) facts = manager.stop_containers(running_containers)
# kill containers # kill containers
elif state == "killed": elif state == "killed":
manager.kill_containers(running_containers) manager.kill_containers(running_containers)
# restart containers # restart containers
elif state == "restarted": elif state == "restarted":
manager.restart_containers(running_containers) manager.restart_containers(running_containers)
@ -517,7 +703,7 @@ def main():
msg = "%s container(s) running image %s with command %s" % \ msg = "%s container(s) running image %s with command %s" % \
(manager.get_summary_counters_msg(), module.params.get('image'), module.params.get('command')) (manager.get_summary_counters_msg(), module.params.get('image'), module.params.get('command'))
changed = manager.has_changed() changed = manager.has_changed()
module.exit_json(failed=failed, changed=changed, msg=msg, ansible_facts=_ansible_facts(facts)) module.exit_json(failed=failed, changed=changed, msg=msg, ansible_facts=_ansible_facts(facts))
except docker.client.APIError, e: except docker.client.APIError, e:
@ -527,7 +713,7 @@ def main():
except RequestException, e: except RequestException, e:
changed = manager.has_changed() changed = manager.has_changed()
module.exit_json(failed=True, changed=changed, msg=repr(e)) module.exit_json(failed=True, changed=changed, msg=repr(e))
# import module snippets # import module snippets
from ansible.module_utils.basic import * from ansible.module_utils.basic import *