add new module with integration tests to manage gitlab project variables (#56574)

* add new module with integration tests to manage gitlab project variables

* fix invalid yaml in DOCUMENTATION variable and don't import * from ansible module

* remove extends_documentation_fragment and put imports after DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA

* fix author in documentation and remove import from display

* add alias file for integration test

* split long lines and try to fix the author key

remove tailing whitespace

* replace email address with github username

* adding the at style to username

* add metaclass and future import

* add state variable to be able to delete selected variables

* add test with state = absent

* update documentation. scheme is necessary

* use singular in exmaple section

* use key purge instead of purge_vars

use purge instead of purge_vars also in the integration test

* create gitlab object in the ansible main function

* remove usedless .format

* follow best practice fail message

* add return documentation, return information about which variables were added, updated or removed and catch gitlab api auth error

* use module_utils.api with api_url and api_token

* use dict instead of list for vars

* use project name instead of name as playbook key

* add ansible checkmode_support, reduce variables in gitlab_project_variables class, remove wrong/duplicated HAS_GITLAB_PACKAGE check

* use extends_documentation_fragment and don't pop elements from basic_auth_argument_spec

* use just project_variable as output variable

* update mutually_exclusive as suggested

* re-add api_token documentation, because it is not included in api basic auth

* remove useless statement

remove unnecessary if

* add one test with a changing value

* put type at first position

* keep item to reduce api calls, build array and keep indexes by replacing with None instead of poping

* more asserts

* Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

* remove unused return key from documentation

msg is only returned when failed

* Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

* remove error key, because it is not returned

* change also documentation from purged_vars to purge

* Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update test/integration/targets/gitlab_project_variable/tasks/main.yml

Co-Authored-By: Felix Fontein <felix@fontein.de>

Update test/integration/targets/gitlab_project_variable/tasks/main.yml

Co-Authored-By: Felix Fontein <felix@fontein.de>

* remove extra spaces

fix wrong spelling

* expand return value documentation with examples

* add check_mode test

reorder tests. first the check_mode test, later all other tests

* Update lib/ansible/modules/source_control/gitlab_project_variable.py

Co-Authored-By: Felix Fontein <felix@fontein.de>

* fix existing keys in 'present' array

rework key handling (reduce code)

fix integration tests

use untouched instead of present to identify unchanged variable keys

fix wrong replacement

minor fixes on request

set aliases to unsupported, because the test succeed

remove posix group1 because it conflicts with unsupported

remove useless item from aliases

* rework gitlab connection
This commit is contained in:
Markus Bergholz 2019-08-02 09:08:37 +02:00 committed by Felix Fontein
parent 6d074d8a94
commit 18aae0a02b
3 changed files with 505 additions and 0 deletions

View file

@ -0,0 +1,254 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-
# Copyright: (c) 2019, Markus Bergholz (markuman@gmail.com)
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
ANSIBLE_METADATA = {'metadata_version': '1.1',
'status': ['preview'],
'supported_by': 'community'}
DOCUMENTATION = '''
module: gitlab_project_variable
short_description: Creates/updates/deletes GitLab Projects Variables
description:
- When a project variable does not exist, it will be created.
- When a project variable does exist, its value will be updated when the values are different.
- Variables which are untouched in the playbook, but are not untouched in the GitLab project,
they stay untouched (I(purge) is C(false)) or will be deleted (I(purge) is C(true)).
version_added: "2.9"
author:
- "Markus Bergholz (@markuman)"
requirements:
- python >= 2.7
- python-gitlab python module
extends_documentation_fragment:
- auth_basic
options:
state:
description:
- Create or delete project variable.
- Possible values are present and absent.
default: present
type: str
choices: ["present", "absent"]
api_token:
description:
- GitLab access token with API permissions.
required: true
type: str
project:
description:
- The path and name of the project.
required: true
type: str
purge:
description:
- When set to true, all variables which are not untoucheded in the task will be deleted.
default: false
type: bool
vars:
description:
- A list of key value pairs.
default: {}
type: dict
'''
EXAMPLES = '''
- name: Set or update some CI/CD variables
gitlab_project_variable:
api_url: https://gitlab.com
api_token: secret_access_token
project: markuman/dotfiles
purge: false
vars:
ACCESS_KEY_ID: abc123
SECRET_ACCESS_KEY: 321cba
- name: Delete one variable
gitlab_project_variable:
api_url: https://gitlab.com
api_token: secret_access_token
project: markuman/dotfiles
state: absent
vars:
ACCESS_KEY_ID: abc123
'''
RETURN = '''
project_variable:
description: Four lists of the variablenames which were added, updated, removed or exist.
returned: always
type: dict
contains:
added:
description: A list of variables which were created.
returned: always
type: list
sample: "['ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']"
untouched:
description: A list of variables which exist.
returned: always
type: list
sample: "['ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']"
removed:
description: A list of variables which were deleted.
returned: always
type: list
sample: "['ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']"
updated:
description: A list of variables whose values were changed.
returned: always
type: list
sample: "['ACCESS_KEY_ID', 'SECRET_ACCESS_KEY']"
'''
import traceback
from ansible.module_utils.basic import AnsibleModule, missing_required_lib
from ansible.module_utils._text import to_native
from ansible.module_utils.api import basic_auth_argument_spec
GITLAB_IMP_ERR = None
try:
import gitlab
HAS_GITLAB_PACKAGE = True
except Exception:
GITLAB_IMP_ERR = traceback.format_exc()
HAS_GITLAB_PACKAGE = False
class GitlabProjectVariables(object):
def __init__(self, module, gitlab_instance):
self.repo = gitlab_instance
self.project = self.get_project(module.params['project'])
self._module = module
def get_project(self, project_name):
return self.repo.projects.get(project_name)
def list_all_project_variables(self):
return self.project.variables.list()
def create_variable(self, key, value):
if self._module.check_mode:
return
return self.project.variables.create({"key": key, "value": value})
def update_variable(self, var, value):
if var.value == value:
return False
if self._module.check_mode:
return True
var.value = value
var.save()
return True
def delete_variable(self, key):
if self._module.check_mode:
return
return self.project.variables.delete(key)
def native_python_main(this_gitlab, purge, var_list, state):
change = False
return_value = dict(added=list(), updated=list(), removed=list(), untouched=list())
gitlab_keys = this_gitlab.list_all_project_variables()
existing_variables = [x.get_id() for x in gitlab_keys]
for key in var_list:
if key in existing_variables:
index = existing_variables.index(key)
existing_variables[index] = None
if state == 'present':
single_change = this_gitlab.update_variable(
gitlab_keys[index], var_list[key])
change = single_change or change
if single_change:
return_value['updated'].append(key)
else:
return_value['untouched'].append(key)
elif state == 'absent':
this_gitlab.delete_variable(key)
change = True
return_value['removed'].append(key)
elif key not in existing_variables and state == 'present':
this_gitlab.create_variable(key, var_list[key])
change = True
return_value['added'].append(key)
existing_variables = list(filter(None, existing_variables))
if purge:
for item in existing_variables:
this_gitlab.delete_variable(item)
change = True
return_value['removed'].append(item)
else:
return_value['untouched'].extend(existing_variables)
return change, return_value
def main():
argument_spec = basic_auth_argument_spec()
argument_spec.update(
api_token=dict(type='str', required=True, no_log=True),
project=dict(type='str', required=True),
purge=dict(type='bool', required=False, default=False),
vars=dict(type='dict', required=False, default=dict()),
state=dict(type='str', default="present", choices=["absent", "present"])
)
module = AnsibleModule(
argument_spec=argument_spec,
mutually_exclusive=[
['api_username', 'api_token'],
['api_password', 'api_token'],
],
required_together=[
['api_username', 'api_password'],
],
required_one_of=[
['api_username', 'api_token']
],
supports_check_mode=True
)
api_url = module.params['api_url']
gitlab_token = module.params['api_token']
purge = module.params['purge']
var_list = module.params['vars']
state = module.params['state']
if not HAS_GITLAB_PACKAGE:
module.fail_json(msg=missing_required_lib("python-gitlab"), exception=GITLAB_IMP_ERR)
try:
gitlab_instance = gitlab.Gitlab(url=api_url, private_token=gitlab_token)
gitlab_instance.auth()
except (gitlab.exceptions.GitlabAuthenticationError, gitlab.exceptions.GitlabGetError) as e:
module.fail_json(msg="Failed to connect to GitLab server: %s" % to_native(e))
except (gitlab.exceptions.GitlabHttpError) as e:
module.fail_json(msg="Failed to connect to Gitlab server: %s. \
Gitlab remove Session API now that private tokens are removed from user API endpoints since version 10.2" % to_native(e))
this_gitlab = GitlabProjectVariables(module=module, gitlab_instance=gitlab_instance)
change, return_value = native_python_main(this_gitlab, purge, var_list, state)
module.exit_json(changed=change, project_variable=return_value)
if __name__ == '__main__':
main()

View file

@ -0,0 +1 @@
unsupported

View file

@ -0,0 +1,250 @@
- name: Install required libs
pip:
name: python-gitlab
state: present
- name: purge all variables for check_mode test
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
purge: True
- name: add a variable value in check_mode
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: checkmode
check_mode: yes
register: gitlab_project_variable_state
- name: check_mode state must be changed
assert:
that:
- gitlab_project_variable_state is changed
- name: apply add value from check_mode test
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: checkmode
register: gitlab_project_variable_state
- name: state must be changed
assert:
that:
- gitlab_project_variable_state is changed
- name: change a variable value in check_mode again
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: checkmode
check_mode: yes
register: gitlab_project_variable_state
- name: check_mode state must not be changed
assert:
that:
- gitlab_project_variable_state is not changed
- name: apply again the value change from check_mode test
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: checkmode
register: gitlab_project_variable_state
- name: state must not be changed
assert:
that:
- gitlab_project_variable_state is not changed
- name: purge all variables at the beginning
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
purge: True
- name: set two test variables
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: abc123
SECRET_ACCESS_KEY: 321cba
register: gitlab_project_variable_state
- name: set two test variables state must be changed
assert:
that:
- gitlab_project_variable_state is changed
- gitlab_project_variable_state.project_variable.added|length == 2
- gitlab_project_variable_state.project_variable.untouched|length == 0
- gitlab_project_variable_state.project_variable.removed|length == 0
- gitlab_project_variable_state.project_variable.updated|length == 0
- name: re-set two test variables
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: abc123
SECRET_ACCESS_KEY: 321cba
register: gitlab_project_variable_state
- name: re-set two test variables state must not be changed
assert:
that:
- gitlab_project_variable_state is not changed
- gitlab_project_variable_state.project_variable.added|length == 0
- gitlab_project_variable_state.project_variable.untouched|length == 2
- gitlab_project_variable_state.project_variable.removed|length == 0
- gitlab_project_variable_state.project_variable.updated|length == 0
- name: edit one variable
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: changed
purge: False
register: gitlab_project_variable_state
- name: edit one variable state must be changed
assert:
that:
- gitlab_project_variable_state.changed
- gitlab_project_variable_state.project_variable.added|length == 0
- gitlab_project_variable_state.project_variable.untouched|length == 1
- gitlab_project_variable_state.project_variable.removed|length == 0
- gitlab_project_variable_state.project_variable.updated|length == 1
- gitlab_project_variable_state.project_variable.updated[0] == "ACCESS_KEY_ID"
- name: append one variable
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
some: value
purge: False
register: gitlab_project_variable_state
- name: append one variable state must be changed
assert:
that:
- gitlab_project_variable_state.changed
- gitlab_project_variable_state.project_variable.added|length == 1
- gitlab_project_variable_state.project_variable.untouched|length == 2
- gitlab_project_variable_state.project_variable.removed|length == 0
- gitlab_project_variable_state.project_variable.updated|length == 0
- gitlab_project_variable_state.project_variable.added[0] == "some"
- name: re-set all variables
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
ACCESS_KEY_ID: changed
SECRET_ACCESS_KEY: 321cba
some: value
register: gitlab_project_variable_state
- name: re-set all variables state must not be changed
assert:
that:
- not gitlab_project_variable_state.changed
- gitlab_project_variable_state.project_variable.added|length == 0
- gitlab_project_variable_state.project_variable.untouched|length == 3
- gitlab_project_variable_state.project_variable.removed|length == 0
- gitlab_project_variable_state.project_variable.updated|length == 0
- name: set one variables and purge all others
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
some: value
purge: True
register: gitlab_project_variable_state
- name: set one variables and purge all others state must be changed
assert:
that:
- gitlab_project_variable_state.changed
- gitlab_project_variable_state.project_variable.added|length == 0
- gitlab_project_variable_state.project_variable.untouched|length == 1
- gitlab_project_variable_state.project_variable.removed|length == 2
- gitlab_project_variable_state.project_variable.updated|length == 0
- name: only one variable is left
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
vars:
some: value
purge: False
register: gitlab_project_variable_state
- name: only one variable is left state must not be changed
assert:
that:
- not gitlab_project_variable_state.changed
- gitlab_project_variable_state.project_variable.added|length == 0
- gitlab_project_variable_state.project_variable.untouched|length == 1
- gitlab_project_variable_state.project_variable.removed|length == 0
- gitlab_project_variable_state.project_variable.updated|length == 0
- gitlab_project_variable_state.project_variable.untouched[0] == "some"
- name: delete the last left variable
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
state: absent
vars:
some: value
register: gitlab_project_variable_state
- name: no variable is left state must be changed
assert:
that:
- gitlab_project_variable_state.changed
- gitlab_project_variable_state.project_variable.added|length == 0
- gitlab_project_variable_state.project_variable.untouched|length == 0
- gitlab_project_variable_state.project_variable.removed|length == 1
- gitlab_project_variable_state.project_variable.updated|length == 0
- gitlab_project_variable_state.project_variable.removed[0] == "some"
- name: check that no variables are left
gitlab_project_variable:
api_url: "{{ gitlab_host }}"
api_token: "{{ gitlab_login_token }}"
project: "{{ gitlab_project_name }}"
purge: True
register: gitlab_project_variable_state
- name: check that no variables are untoucheded state must be changed
assert:
that:
- not gitlab_project_variable_state.changed
- gitlab_project_variable_state.project_variable.added|length == 0
- gitlab_project_variable_state.project_variable.untouched|length == 0
- gitlab_project_variable_state.project_variable.removed|length == 0
- gitlab_project_variable_state.project_variable.updated|length == 0