Commit graph

118 commits

Author SHA1 Message Date
Sviatoslav Sydorenko
ee290bf5b1 Declare support for Python 3.8 @ Trove classifiers 2019-10-28 10:06:03 -07:00
Toshio Kuratomi
5f227fe260 Install ansible-test (#60718)
* Install ansible-test

Modify the install script to install ansible-test and its supporting
code.  Alternative to #60701 that doesn't change package_dir ansible for
fear that it might regress https://github.com/ansible/ansible/issues/10437

Also:
* No longer use package_data.  Everything in the package dirs is going
  to be installed.  Anything that shouldn't be installed needs to be
  moved elsewhere.
* modify the algorithm to store symlinks which are in the same tree
  instead of same directory

* Add ansible_test files to package-data sanity test

* MANIFEST.in cleanups

* Add lib/ansible/config/*.yml
* Make most things in code directories (lib/ansible and test/lib/ansible_test/)
  use explicit file extensions instead of wildcards for maintainability
* Exclude common file extensions that we don't want included in the code
  directories

* Change package-data test to be more complete

* Now compares the repository, sdist, and install
* Compares both that everything in the sdist is in the repo and
  everything in the install is in the sdist in addition to comparing that
  everything in the repo that we want is in the install

* Leave out test artifacts

Only include the directory structure for test/results and test/cache not
any files that may have been generated by test runs

Remove test/utils files from the sdist as these are only needed for our CI

cleanup of docs in MANIFEST.in; getting rid of build files.

* Add the ability to output sdist and snapshot to specific directory

* Add a warning about modifying the heuristic to setup.py

* Address generated files

* Use make snapshot instead of sdist to generate changelog and man pages
  and make sure they're included
* Ignore both the test/utils and generated test files (results, cache)
* Deal with Python3 __pycache__ byte code caches
* Don't check documentation, that isn't built for the sdist
* Restructure for clarity

* Add cli web docs to make clean

This was causing problems when attempting to test that the sdist didn't
have extra files

* Fix bug constructing python names from __pycache__ names

* Create a clean repo to work from

* Exclude test/legacy and be more explicit on extensions

* Exclude the legacy directory from sdist
2019-08-20 23:53:35 -07:00
Matt Martz
95f4282c42
Verify package data in setup.py installs all files (#59537)
* Add sanity test to ensure all non-py files are installed

* Fix mode and regex

* Fix role skel inventory package_data

* Add docs

* Update package_data for inventory files

* Address pylint concerns

* Another tweak to package_data

* Address review feedback

* Change index to 1

* add to ansible-only.txt
2019-07-24 13:58:13 -05:00
Matt Martz
65bedaf074
Ensure package_data matches collections_galaxy_meta.yml. Fixes #59451 (#59452) 2019-07-23 15:52:38 -05:00
John R. Dennis
0a5a74f192 Install hidden galaxy data files (e.g. .travis.yml) (#56005)
The package_data globs in setup.py were missing the necessary glob
to install:

galaxy/data/default/.travis.yml

A Python glob pattern will not by default match a hidden file
(i.e. a file basename beginning with a dot). If you want a glob
to pick up a hidden file in a directory you must explicitly specify
a glob pattern with "/.*".

Closes: #1777
Signed-off-by: John Dennis <jdennis@redhat.com>
2019-06-25 11:28:51 -07:00
Toshio Kuratomi
5ba7063f4f
Build manpages as part of sdist (#53728)
* Build fixups

This is in pursuit of making snapshots easier.

* Allow overriding the python command used for generating man pages
* Build the changelog prior to creating the sdist
* Add the uninstalled ansible library to PYTHONPATH for changelog
  generation
* Warn that python setup.py sdist may be incomplete; use make sdist
  or make snapshot instead.
* Implement a snapshot make command

* Fix environ variable test to use a string

Co-Authored-By: abadger <a.badger@gmail.com>
2019-03-18 12:43:38 -07:00
Jordan Borean
e972287c35 win_exec: refactor PS exec runner (#45334)
* win_exec: refactor PS exec runner

* more changes for PSCore compatibility

* made some changes based on the recent review

* split up module exec scripts for smaller payload

* removed C# module support to focus on just error msg improvement

* cleaned up c# test classifier code
2018-10-02 15:55:53 -07:00
Toshio Kuratomi
ccf41bb2fe
Update the minimum python versions to install (#46230)
* Update the minimum python versions to install

This will prevent modern pip from installing ansible-2.7.x in
python-2.6.  Users will get the Ansible-2.6.x maintenance releases
instead.
2018-09-27 13:49:06 -07:00
mrmagooey
991f61c138 Improve symlink install error message in setup.py
PR #45132

Co-authored-by: Sviatoslav Sydorenko <wk@sydorenko.org.ua>
2018-09-21 23:40:30 +03:00
Alan Rominger
34b36d5178 add missing entry to package_data (#44585) 2018-08-23 14:00:22 -05:00
Sviatoslav Sydorenko
fc25d280be
Add CoC and mailing lists links to PYPI
PR #42101
2018-06-30 22:55:50 +03:00
Sviatoslav Sydorenko
14cdf3b03c Suppress a UserWarning about unknown dist option 2018-04-12 23:53:52 +02:00
Sviatoslav Sydorenko
8912b43051 Refactor setup.py helper code 2018-04-12 23:53:52 +02:00
Sviatoslav Sydorenko
922e7039dc Shield setup module from accidental setup exec
This might happen when third-parties like analizers import this
module for testing purposes.
2018-04-12 23:53:52 +02:00
Sviatoslav Sydorenko
75092c9f29 Update python requirements in distribution meta
This has been discussed during #38413 review, but got missed.
2018-04-12 23:53:52 +02:00
Sviatoslav Sydorenko
a173cf51e8 Add a list or extra URLs to distribution metadata
This should enable Warehouse PYPI implementation pypa/warehouse#3604
to pull data from GitHub and show it on the project page.
2018-04-12 23:53:52 +02:00
Toshio Kuratomi
988808c066 README.rst should always exist so no need to set a default value 2018-04-09 11:59:55 -07:00
Stephen Weber
7afa34ae8a Convert README from Markdown to ReStructured Text and use as longdesc (#22330)
* Convert README from Markdown to ReStructured Text and use as longdesc

Discussion in #13758 led to deciding to switch README to rst and having
setup.py consume it as the long_description.

* Fix long string in setup.py for pep8 compliance

* Open README.rst as read-only

* Update usages of root README.md to README.rst

Unsure about the file ./packaging/debian/docs as it (only) contains the
text README.md. I believe it's referencing ./packaging/debian/README.md
but maybe someone who knows debian packaging could review it?

* Pick up fixes that had been merged into README.md after the initial conversion to rst
2018-04-06 12:38:40 -07:00
Matt Martz
bd43776acf
Add ansible-config and ansible-inventory to setup.py scripts (#37151) 2018-03-07 16:51:22 -06:00
Alan Rominger
ef42eaeaa1 set the zip_safe flag to False (#32194)
The references to __file__ within code base make the zip_safe
method of installing not possible.
2017-10-27 12:47:45 -04:00
Toshio Kuratomi
e132918b92 On Py3, nonexistent files raise OSError (#28899) 2017-08-31 17:47:26 -04:00
Brian Coca
f921369445 Ansible Config part2 (#27448)
* Ansible Config part2

- made dump_me nicer, added note this is not prod
- moved internal key removal function to vars
- carry tracebacks in errors we can now show tracebacks for plugins on vvv
- show inventory plugin tracebacks on vvv
- minor fixes to cg groups plugin
- draft config from plugin docs
- made search path warning 'saner' (top level dirs only)
- correctly display config entries and others
- removed unneeded code
- commented out some conn plugin specific from base.yml
- also deprecated sudo/su
- updated ssh conn docs
- shared get option method for connection plugins
- note about needing eval for defaults
- tailored yaml ext
- updated strategy entry
- for connection pliugins, options load on plugin load
- allow for long types in definitions
- better display in ansible-doc
- cleaned up/updated source docs and base.yml
- added many descriptions
- deprecated include toggles as include is
- draft backwards compat get_config
- fixes to ansible-config, added --only-changed
- some code reoorg
- small license headers
- show default in doc type
- pushed module utils details to 5vs
- work w/o config file
- PEPE ATE!
- moved loader to it's own file
- fixed rhn_register test
- fixed boto requirement in make tests
- I ate Pepe
- fixed dynamic eval of defaults
- better doc code

skip ipaddr filter tests when missing netaddr
removed devnull string from config
better becoem resolution

* killed extra space with extreeme prejudice

cause its an affront against all that is holy that 2 spaces touch each other!

shippable timing out on some images, but merging as it passes most
2017-08-15 16:38:59 -04:00
Thomas Stringer
e4cd899363 Add support for Azure 2.0.0 (#27920)
* Adapt azure_rm_resource_group to azure 2.0.0 + azure Cli support

* Fix exceptions in Azure ARM plugins

* update azure_rm_networkinterface documention to reflect required params

* change state param to not required for docs in azure_rm_subnet

* fix import to reflect azure==2.0.0 changes

* add aliases and fix docs for azure_rm_storageblob

* add resource_group_name alias to azure_rm_storageaccount_facts

* fix import bug due to change in azure==2.0.0

* fix args bug and enum modules issue

* update docs to reflect azure==2.0.0

* pin management clients to a specific api_version

* update docs to reflect the new azure-ansible-base python package

* add fallback for older api resource group listing

* rework azure dependencies installation

* refactor path joining to a cross-plat solution
2017-08-11 08:22:02 -07:00
Matt Davis
dae8857d3d powershell setup fixes (#27516)
* fixes #27374
* recursively include top 2 levels of .psm1's under module_utils/powershell
* recursively include top 2 levels of .ps1's under modules/windows (for future restructuring)
2017-07-31 12:16:26 -07:00
Toshio Kuratomi
eff757eeb8 Maintain symlinks in setup.py sdist, build, and install commands (#27149)
* Maintain symlinks in setup.py sdist, build, and install commands

Symlinks are meaningful for ansible modules.  They differentiate between
aliases and deprecated modules.  They're also useful for saving space
and where downstream patches should be applied to the bin scripts.

Fixes #27105

* Add a fallback for install and build to try to cache symlinks on their own

Needed when someone tries to invoke setup.py build or setup.py install
directly from the source checkout without an intermediate sdist.
2017-07-24 09:25:12 -07:00
Toshio Kuratomi
e238ae999b Cyptography pr 20566 rebase (#25560)
Make pyca/cryptography the preferred backend for cryptographic needs (mainly vault) falling back to pycrypto

pyca/cryptography is already implicitly a dependency in many cases
through paramiko (2.0+) as well as the new openssl_publickey module,
which requires pyOpenSSL 16.0+. Additionally, pyca/cryptography is
an optional dep for better performance with vault already.

This commit leverages cryptography's padding, constant time comparisons,
and CBC/CTR modes to reduce the amount of code ansible needs to
maintain.

* Handle wrong password given for VaultAES format

* Do not display deprecation warning for cryptography on python-2.6

* Namespace all of the pycrypto imports and always import them

  Makes unittests better and the code less likely to get stupid mistakes
  (like using HMAC from cryptogrpahy when the one from pycrypto is needed)

* Add back in atfork since we need pycrypto to reinitialize its RNG just in case we're being used with old paramiko

* contrib/inventory/gce: Remove spurious require on pycrypto

(cherry picked from commit 9e16b9db275263b3ea8d1b124966fdebfc9ab271)

* Add cryptography to ec2_win_password module requirements
  * Fix python3 bug which would pass text strings to a function which
    requires byte strings.

* Attempt to add pycrypto version to setup deps

* Change hacking README for dual pycrypto/cryptography

* update dependencies for various CI scripts

* additional CI dockerfile/script updates

* add paramiko to the windows and sanity requirement set

  This is needed because ansible lists it as a requirement. Previously
  the missing dep wasn't enforced, but cryptography imports pkg_resources
  so you can't ignore a requirement any more

* Add integration test cases for old vault and for wrong passwords

* helper script for manual testing of pycrypto/cryptography

* Skip the pycrypto tests so that users without it installed can still run the unittests

* Run unittests for vault with both cryptography and pycrypto backend
2017-06-27 06:00:15 -07:00
nemobis
5d28d76277 Set license GPLv3+ ("or any later version") in setup.py
Most file headers in the project contain the standard
> either version 3 of the License, or (at your option) any later version
https://github.com/ansible/ansible/search?utf8=%E2%9C%93&q=%22any+later+version%22

So this project appears to be GPLv3+, as stated in the classifiers function.
2017-06-26 09:57:47 -07:00
Brian Coca
74842adc07 1st part of ansible config, adds ansible-config to view/manage configs (#12797)
* Start of ansible config project

moved configuration definitions to external yaml file vs hardcoded
 * updated constants to be a data strcutures that are looped over and also return origin of setting
changed to manager/data scheme for base classes
new cli ansible-config to view/manage ansible configuration settings
 * prints green for default/unchanged and yellow for those that have been overriden
 * added list action to show all configurable settings and their associated ini and env var names
 * allows specifying config file to see what result would look like
 * TBD update, edit and view options

removed test for functions that have been removed

env_Vars are now list of dicts
allows for version_added and deprecation in future
added a couple of descriptions for future doc autogeneration
ensure test does not fail if delete_me exists
normalized 'path expansion'
added yaml config to setup packaging
removed unused imports
better encoding handling

updated as per feedback

* pep8
2017-06-14 11:08:34 -04:00
Toshio Kuratomi
9bbd48ad5f Make setting of metadata for crypto backend settable at buildtime
Working on several separate ideas to make ansible work with both
cryptography or pycrypto.  This particular change will let a user
installing ansible change which crypto backend gets encoded in the
metadata (and hence, which backend is required by pkg_resource).

This works with both setup.py and pip::

  ANSIBLE_CRYPTO_BACKEND=cryptography pip install ansible
  ANSIBLE_CRYPTO_BACKEND=cryptography python2 ./setup.py

Note that the code to use cryptography is not yet merged so using
ANSIBLE_CRYPTO_BACKEND is not yet a good idea (as it can lead to not
installing pycrypto and hence the current vault code will fail).

Related to: #20566
2017-06-02 12:05:45 -07:00
Dag Wieers
4efec414e7 test/: PEP8 compliancy (#24803)
* test/: PEP8 compliancy

- Make PEP8 compliant

* Python3 chokes on casting int to bytes (#24952)

But if we tell the formatter that the var is a number, it works
2017-05-30 18:05:19 +01:00
Adrian Likins
6f2cd64cff Include .git_keep files in role template skeleton (#24381)
setup.py will not install package_data this is just an empty
directory, even if the globs in package_data match it.

So the role skeleton that was being installed to galaxy/data/*/
was excluding the files/ and templates/ directories since they were
empty.

Since the skeleton dir doesnt include those dirs
'ansible-galaxy init' would not set them up.

So this adds a .* glob to those directories so the .git_keep
will be included as well, so that setup.py will create the otherwise
empty directoty. ansible-galaxy init already knows to ignore those
files, so no other changes are needed.

(Including the .git_keep files is a little odd, but the alternative
would be creating our own placeholder files. Since ansible-galaxy
already understands .git_keep files we just reuse that)

Fixes #23597
2017-05-09 09:57:30 -04:00
Toshio Kuratomi
57a7d7704f Make scripts into symlinks
pip can't handle zip files with symlinks.  Attempt to workaround that by
transforming the bin scripts into symlinks after the zip file has been
unarchived into the build tree.
2017-04-04 11:19:27 -07:00
Robin Schneider
3700bcb6dd Use HTTPS instead of legacy HTTP for ansible.com (#16870)
Mechanical edit done by this "one-liner":

```Shell
git ls-files -z "$(git rev-parse --show-toplevel)" | xargs --null -I '{}' find '{}' -type f -print0 | xargs --null sed --in-place --regexp-extended 's#http://(www\.|galaxy\.|)ansible\.com#https://\1ansible.com#g;'
```

Related to: https://github.com/ansible/ansible/issues/16869
2017-02-15 16:09:33 -08:00
Toshio Kuratomi
d7b7cbac1a Move to using a requirements.txt to install the python packages. (#21430)
Move to using a requirements.txt to install the python packages.

This makes it easy to keep the documentation and actual package
dependencies in sync.

Fixes #18453
2017-02-15 06:58:09 -08:00
Matt Martz
87aa59af79 Legacy pep8 updates for setup.py and tests 2017-02-11 16:04:52 -08:00
Toshio Kuratomi
d1a6b07fe1 Move ssh and local connection plugins from using raw select to selectors
At the moment, this change will use EPoll on Linux, KQueue on *BSDs,
etc, so it should alleviate problems with too many open file
descriptors.

* Bundle a copy of selectors2 so that we have the selectors API everywhere.
* Add licensing information to selectors2 file so it's clear what the
  licensing terms and conditions are.
* Exclude the bundled copy of selectors2 from our boilerplate code-smell test
* Rewrite ssh_run tests to attempt to work around problem with mocking
  select on shippable

Fixes #14143
2017-02-03 08:23:50 -08:00
Matt Clay
10d9318de7 PEP 8 indent cleanup. (#20800)
* PEP 8 E121 cleanup.

* PEP 8 E126 cleanup.

* PEP 8 E122 cleanup.
2017-01-29 07:28:53 +00:00
James Cammarata
4f55f032ea Updating setup.py to remove extras specific paths 2016-12-08 11:35:19 -05:00
Nathaniel Case
4680fafd9b Add ansible-connection to scripts in setup.py 2016-12-05 11:23:04 -05:00
David Côté-Tremblay
221cafe769 Removing the shebang and the execution permission so the command be prefixed with python2 or python3 as needed 2016-11-07 08:00:57 -08:00
Toshio Kuratomi
a3ba96c512 Fix setup.py install.
In setuptools prior to 21.2.1, package_data can't directly reference
a directory.  Modify the entry for galaxy data so that it includes the
files but not the directories

Fixes #18231
2016-11-03 17:46:11 -07:00
Bill Nottingham
77e7ae6838 Change <support@ansible.com> - it's being retired. 2016-10-15 16:42:13 -07:00
Toshio Kuratomi
487e6562ca Fix ziploader for the cornercase of ansible invoking ansible.
* Make ziploader's ansible and ansible.module_utils libraries into
  namespace packages.
* Move __version__ and __author__ from ansible/__init__ to
  ansible/release.py.  This is because namespace packages only load one
  __init__.py.  If that is not the __init__.py with the author and
  version info then those won't be available.
* In ziplaoder, move the version ito ANSIBLE_CONSTANTS.
* Change PluginLoader to properly construct the path to the plugins even
  when namespace packages are present.
2016-04-29 08:47:49 -07:00
Brian Coca
369b3b317e renamed shell to console in last spot 2016-03-09 13:57:45 -05:00
Brian Coca
e74ab3ecdd draft 1st release of ansible-console
porting @dominis 's ansible-shell tool from 1.9 and integrating it into ansible
added verbosity control
made more resilitent to several errors
added highlight color, to configurable colors
more resilient on exception and interruptions
prompt coloring, goes red and changes to # when using become = true and root
become setting is now explicit and not a toggle
2016-03-07 20:25:21 -05:00
Toshio Kuratomi
baa309309d Bundle a new version of python-six for compatibility along with some code to make it easy for distributions to override the bunndled copy if they have a new enough version. 2015-10-16 08:21:28 -07:00
Brian Coca
66c3461f3c now include galaxy/data/ stuff for use with ansible-galaxy 2015-09-01 10:58:58 -04:00
billwanjohi
b2739cec6d add distutils package classifiers
I was particularly interested in the programming language ones,
but the others might be useful to others browsing PyPI.

Now with GPLv3+, and Utilities topic.
2015-07-27 19:33:54 +00:00
James Cammarata
8509f22839 Merge pull request #11731 from danasmera/devel
Delete unused import glob
2015-07-25 02:47:58 -04:00
danasmera
966193e233 Delete unused import glob 2015-07-24 23:57:38 -04:00