cmueller/ansible
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Remove old boto documentation and promote AnsibleAWSModule (#36763)

Browse source 
There should no longer be a need for boto documentation - it
just adds significant extra clutter to the guidelines.

AnsibleAWSModule greatly improves the codebase to such an extent
that we should mandate it for new code unless there is a
documented good reason.
This commit is contained in:
Will Thames 2018-03-03 05:03:12 +10:00 committed by ansibot
parent 3a161fa157
commit 836d7a42a4
1 changed files with 61 additions and 103 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

164
lib/ansible/modules/cloud/amazon/GUIDELINES.md
View file

@ -14,9 +14,35 @@ Please do not add new dependencies on the old boto library.
Prior to 2.0, modules may have been written in boto or boto3. The effort to port all modules to
boto3 has begun. From Ansible 2.4 it is permissible for modules which previously required boto to
start to require boto3 in order to deliver the functionality previously supported with boto and the
start to migrate to boto3 in order to deliver the functionality previously supported with boto and the
boto dependency can be deleted.
From 2.6, all new modules should use AnsibleAWSModule as a base, or have a documented reason not
to. Using AnsibleAWSModule greatly simplifies exception handling and library management, reducing
the amount of boilerplate code.
## Porting code to AnsibleAWSModule
Change
```
from ansible.module_utils.basic import AnsibleModule
...
module = AnsibleModule(...)
```
to
```
from ansible.module_utils.aws.core import AnsibleAWSModule
...
module = AnsibleAWSModule(...)
```
Few other changes are required. One possible issue that you might encounter is that AnsibleAWSModule
does not inherit methods from AnsibleModule by default, but most useful methods
are included. If you do find an issue, please raise a bug report.
## Bug fixing
Bug fixes to code that relies on boto will still be accepted. When possible, the code should be
@ -57,35 +83,17 @@ else:
The `ansible.module_utils.ec2` module and `ansible.module_utils.core.aws` modules will both
automatically import boto3 and botocore. If boto3 is missing from the system then the variable
`HAS_BOTO3` will be set to false. Normally, this means that modules don't need to import either
botocore or boto3 directly. There is no need to check `HAS_BOTO3` when using AnsibleAWSModule
`HAS_BOTO3` will be set to false. Normally, this means that modules don't need to import
boto3 directly. There is no need to check `HAS_BOTO3` when using AnsibleAWSModule
as the module does that check.
If you want to import the modules anyway (for example `from botocore.exception import
ClientError`) Wrap import statements in a try block and fail the module later using `HAS_BOTO3` if
the import fails. See below as well (`Exception Handling for boto3 and botocore`) for more detail
on how to safely import botocore exceptions.
#### boto
```python
try:
import boto.ec2
from boto.exception import BotoServerError
HAS_BOTO = True
except ImportError:
HAS_BOTO = False
def main():
if not HAS_BOTO:
module.fail_json(msg='boto required for this module')
```
#### boto3
```python
from ansible.module_utils.aws.core import AnsibleAWSModule
try:
import botocore
except ImportError:
pass # handled by AnsibleAWSModule
```
or
@ -94,6 +102,10 @@ or
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.ec2 import HAS_BOTO3
try:
import botocore
except ImportError:
pass # handled by imported HAS_BOTO3
def main():
@ -103,25 +115,8 @@ def main():
#### boto and boto3 combined
Ensure that you clearly document if a new parameter requires requires a specific version. Import
boto3 at the top of the module as normal and then use the `HAS_BOTO3` bool when necessary, before the
new feature.
Modules should be ported to use boto3 rather than use both boto and boto3.
```python
from ansible.module_utils.ec2 import HAS_BOTO3
try:
import boto
HAS_BOTO = True
except ImportError:
HAS_BOTO = False
if my_new_feature_Parameter_is_set:
if HAS_BOTO3:
# do feature
else:
module.fail_json(msg="boto3 is required for this feature")
```
### Connecting to AWS
@ -133,8 +128,6 @@ to connect to AWS as these handle the same range of connection options.
These helpers also for missing profiles or a region not set when it needs to be, so you don't have to.
#### boto3
An example of connecting to ec2 is shown below. Note that unlike boto there is no `NoAuthHandlerFound`
exception handling like in boto. Instead, an `AuthFailure` exception will be thrown when you use the
connection. To ensure that authorization, parameter validation and permissions errors are all caught,
@ -158,22 +151,9 @@ region, ec2_url, aws_connect_params = get_aws_connection_info(module, boto3=True
connection = boto3_conn(module, conn_type='client', resource='ec2', region=region, endpoint=ec2_url, **aws_connect_params)
```
#### boto
An example of connecting to ec2:
Some boto services require that the region is specified. You should check for the region parameter
if required.
```python
region, ec2_url, aws_connect_params = get_aws_connection_info(module)
if region:
try:
connection = connect_to_aws(boto.ec2, region, **aws_connect_params)
except (boto.exception.NoAuthHandlerFound, AnsibleAWSError) as e:
module.fail_json(msg=str(e))
else:
module.fail_json(msg="region must be specified")
region, ec2_url, aws_connect_params = get_aws_connection_info(module, boto3=True)
connection = boto3_conn(module, conn_type='client', resource='ec2', region=region, endpoint=ec2_url, **aws_connect_params)
```
### Common Documentation Fragments for Connection Parameters
@ -198,35 +178,7 @@ extends_documentation_fragment:
'''
```
### Exception Handling for boto
You should wrap any boto call in a try block. If an exception is thrown, it is up to you decide how
to handle it but usually calling fail_json with the error or helpful message and traceback will
suffice.
#### boto
```python
# Import BotoServerError
try:
import boto.ec2
from boto.exception import BotoServerError
HAS_BOTO = True
except ImportError:
HAS_BOTO = False
# Connect to AWS
...
# Make a call to AWS
try:
result = connection.aws_call()
except BotoServerError as e:
module.fail_json(msg="helpful message here", exception=traceback.format_exc(),
**camel_dict_to_snake_dict(e.message))
```
### Exception Handling for boto3 and botocore
### Exception Handling
You should wrap any boto3 or botocore call in a try block. If an exception is thrown, then there
are a number of possibilities for handling it.
@ -240,14 +192,14 @@ For more information on botocore exception handling see [the botocore error docu
#### using fail_json_aws()
_fail_json_aws() is a new method and may be subject to change. You can use it in modules which are
being contributed back to Ansible, however if you are publishing your module separately please
don't use it before the start of 2018 / Ansible 2.4_
In the AnsibleAWSModule there is a special method, `module.fail_json_aws()` for nice reporting of
exceptions. Call this on your exception and it will report the error together with a traceback for
use in Ansible verbose mode.
You should use the AnsibleAWSModule for all new modules, unless not possible. If adding significant
amounts of exception handling to existing modules, we recommend migrating the module to use AnsibleAWSModule
(there are very few changes required to do this)
```python
from ansible.module_utils.aws.core import AnsibleAWSModule
@ -473,6 +425,20 @@ functions detailed below.
boto3 returns results in a dict. The keys of the dict are in CamelCase format. In keeping with
Ansible format, this function will convert the keys to snake_case.
`camel_dict_to_snake_dict` takes an optional parameter called `ignore_list` which is a list of
keys not to convert (this is usually useful for the `tags` dict, whose child keys should remain with
case preserved)
Another optional parameter is `reversible`. By default, `HTTPEndpoint` is converted to `http_endpoint`,
which would then be converted by `snake_dict_to_camel_dict` to `HttpEndpoint`.
Passing `reversible=True` converts HTTPEndpoint to `h_t_t_p_endpoint` which converts back to `HTTPEndpoint`.
#### snake_dict_to_camel_dict
`snake_dict_to_camel_dict` converts snake cased keys to camel case. By default, because it was
first introduced for ECS purposes, this converts to dromedaryCase. An optional
parameter called `capitalize_first`, which defaults to `False`, can be used to convert to CamelCase.
#### ansible_dict_to_boto3_filter_list
Converts a an Ansible list of filters to a boto3 friendly list of dicts. This is useful for any
@ -482,15 +448,7 @@ boto3 `_facts` modules.
Pass an exception returned from boto or boto3, and this function will consistently get the message from the exception.
```
import traceback
from ansible.module_utils.ec2 import boto_exception
try:
...
except boto.exception.BotoServerError as err:
error_msg = boto_exception(err)
module.fail_json(msg=error_msg, exception=traceback.format_exc())
```
Deprecated: use `AnsibleAWSModule`'s `fail_json_aws` instead.
#### boto3_tag_list_to_ansible_dict