maxmustermann/pulumi
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
pulumi/sdk/python/Makefile

49 lines
1.5 KiB
Makefile
Raw Normal View History

Get the empty Python program working This change gets enough of the Python SDK up and running that the empty Python program will work. Mostly just scaffolding, but the basic structure is now in place. The primary remaining work is to wire up resource creation to the gRPC interfaces. In summary: * The basic structure is as follows: - Everything goes into sdk/python/. - sdk/python/cmd/pulumi-langhost-python is a Go language host that simply knows how to spawn Python processes to run out entrypoint in response to requests by the engine. - sdk/python/cmd/pulumi-langhost-python-exec is a little Python shim that is invoked by the language host to run Python programs, and is responsible for setting up the minimal goo before we can do so (RPC connections and the like). - sdk/python/lib/ contains a Python Pip package suitable for PyPi. - In there, we have two packages: the root pulumi package that contains all of the basic Pulumi programming model abstractions, and pulumi.runtime, which contains the implementation of resource registration, RPC interfacing with the engine, and so on. * Add logic in our test framework to conditionalize on the language type and react accordingly. This will allow us to skip Yarn for Python projects and eventually run Pip if there's a requirements.txt. * Created the basic project structure, including all of the usual Make targets for installing into the proper places. * Building also runs Pylint and we are clean. There are a few other minor things in here: * Add an "empty" test for both Node.js and Python. These pass. * Fix an existing bug in plugin shutdown logic. At some point, we started waiting for stderr/stdout to flush before shutting down the plugin; but if certain failures happen "early" during the plugin launch process, these channels will never get initialized and so waiting for them deadlocks. * Recently we seem to have added logic to delete test temp directories if a failure happened during initialization of said temp directories. This is unfortunate, because you often need to look at the temp directory to see what failed. We already clean them up elsewhere after the full test completes successfully, so I don't think we need to be doing this, and I've removed it. Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
PROJECT_NAME := Pulumi Python SDK
move sdk/python/cmd/pulumi-language-python -> pkg/cmd/pulumi-language-python
2020-03-18 18:19:22 +01:00
LANGHOST_PKG := github.com/pulumi/pulumi/pkg/cmd/pulumi-language-python
Fix Python package versions. (#3505) Earlier changes to the get-version script were not adopted by the Python SDK Makefile. This caused package publishing to skip uploads due to missing versions.
2019-11-13 20:34:59 +01:00
VERSION := $(shell ../../scripts/get-py-version HEAD)
Implement the Python langhost RPC server This change adds a basic Python langhost RPC server. It's fairly barebones and merely acts as a jumping off point for the Pulumi engine to spawn a Python program. The host is written in Go, in contrast to implementing the host in Python, and more closely resembles how I expect the Node.js language host to work once pulumi/pulumi#331 is done.
2017-12-28 03:11:38 +01:00
Use Pipenv to manage Python environments (#1553) * Use Pipenv to manage Python environments * Rename PIPENV_ARGS to PIPENV_PYTHON_VERSION to avoid confusion. Also remove two unused variables
2018-06-22 03:01:23 +02:00
PYENV := ./env
Reenable Pylint (#1024) This change uses virtualenv to insulate us from platform differences in our building of the Python SDK, and to create an isolated Python 2 environment. This includes meaning we don't need to worry about the specific location and behavior of Pylint. I *think* this will work no matter whether it's Mac, Ubuntu, ArchLinux, Windows, and so on. We do install to the --user directory in the install target using `pip install -e`, however, which enables the machine-wide symlinking that we need to support various workflows. This fixes pulumi/pulumi#1007.
2018-03-10 00:11:37 +01:00
PYENVSRC := $(PYENV)/src
Get the empty Python program working This change gets enough of the Python SDK up and running that the empty Python program will work. Mostly just scaffolding, but the basic structure is now in place. The primary remaining work is to wire up resource creation to the gRPC interfaces. In summary: * The basic structure is as follows: - Everything goes into sdk/python/. - sdk/python/cmd/pulumi-langhost-python is a Go language host that simply knows how to spawn Python processes to run out entrypoint in response to requests by the engine. - sdk/python/cmd/pulumi-langhost-python-exec is a little Python shim that is invoked by the language host to run Python programs, and is responsible for setting up the minimal goo before we can do so (RPC connections and the like). - sdk/python/lib/ contains a Python Pip package suitable for PyPi. - In there, we have two packages: the root pulumi package that contains all of the basic Pulumi programming model abstractions, and pulumi.runtime, which contains the implementation of resource registration, RPC interfacing with the engine, and so on. * Add logic in our test framework to conditionalize on the language type and react accordingly. This will allow us to skip Yarn for Python projects and eventually run Pip if there's a requirements.txt. * Created the basic project structure, including all of the usual Make targets for installing into the proper places. * Building also runs Pylint and we are clean. There are a few other minor things in here: * Add an "empty" test for both Node.js and Python. These pass. * Fix an existing bug in plugin shutdown logic. At some point, we started waiting for stderr/stdout to flush before shutting down the plugin; but if certain failures happen "early" during the plugin launch process, these channels will never get initialized and so waiting for them deadlocks. * Recently we seem to have added logic to delete test temp directories if a failure happened during initialization of said temp directories. This is unfortunate, because you often need to look at the temp directory to see what failed. We already clean them up elsewhere after the full test completes successfully, so I don't think we need to be doing this, and I've removed it. Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
include ../../build/common.mk
ensure::
Use pipenv from the toolset - Remove the forked copy of the toolset - Stop installing `pipenv` in sdk/python/Makefile After this, we'll require that you already have `pipenv` present before building.
2018-11-06 21:25:58 +01:00
pipenv --python 3 install --dev
Reenable Pylint (#1024) This change uses virtualenv to insulate us from platform differences in our building of the Python SDK, and to create an isolated Python 2 environment. This includes meaning we don't need to worry about the specific location and behavior of Pylint. I *think* this will work no matter whether it's Mac, Ubuntu, ArchLinux, Windows, and so on. We do install to the --user directory in the install target using `pip install -e`, however, which enables the machine-wide symlinking that we need to support various workflows. This fixes pulumi/pulumi#1007.
2018-03-10 00:11:37 +01:00
mkdir -p $(PYENVSRC)
Include README in pypi package Use `pandoc` to convert our README.md to a README.rst and then include that as the long_description in our pypi package.
2018-07-02 20:43:35 +02:00
Add a Dockerfile for the Pulumi CLI This introduces a Dockerfile for the Pulumi CLI. This makes it easier to develop and test the engine in a self-contained environment, in addition to being suitable for running the actual CLI itself. For instance, $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up will run the Pulumi program mounted under the /app volume. This will be used in some upcoming CI/CD scenarios. This uses multi-stage builds, and Debian Stretch as the base, for relatively fast and lean build times and resulting images. We are intentional about restoring dep packages independent of the actual source code so that we don't end up needlessly re-depping, which can consume quite a bit of time. After fixing https://github.com/pulumi/pulumi/issues/1986, we should explore an Alpine base image option. I made the decision to keep this image scoped to just the Go builds. Therefore, none of the actual SDK packages themselves are built, just the engine, CLI, and language plugins for Node.js, Python, and Go. It's possible to create a mega-container that has all of these full environments so that we can rebuild them too, but for now I figured it was better to rely on package management for them. Another alternative would have been to install released binaries, rather than building them. To keep the useful flow for development, however, I decided to go the build route for now. If we build at the same hashes, the resulting binaries "should" be ~identical anyhow. I've created a pulumi/pulumi Docker Hub repo that we can publish this into. For now, there is no CI publishing of the image. This fixes pulumi/pulumi#1991.
2018-09-29 20:43:35 +02:00
build_package::
Reenable Pylint (#1024) This change uses virtualenv to insulate us from platform differences in our building of the Python SDK, and to create an isolated Python 2 environment. This includes meaning we don't need to worry about the specific location and behavior of Pylint. I *think* this will work no matter whether it's Mac, Ubuntu, ArchLinux, Windows, and so on. We do install to the --user directory in the install target using `pip install -e`, however, which enables the machine-wide symlinking that we need to support various workflows. This fixes pulumi/pulumi#1007.
2018-03-10 00:11:37 +01:00
rm -rf $(PYENVSRC) && cp -R ./lib/. $(PYENVSRC)/
sed -i.bak "s/\$${VERSION}/$(VERSION)/g" $(PYENVSRC)/setup.py && rm $(PYENVSRC)/setup.py.bak
Remove the need for `pandoc` during the build We had been using `pandoc` to convert our README.md into a README.rst for use with `setup.py` and the python package ecosystem. It turns out that we can use markdown if we set a content type. So let's do that and make things a little simpler. While I was in the area, I made the encoding of UTF-8 explicit when opening README.md.
2019-09-07 00:06:40 +02:00
cp ../../README.md $(PYENVSRC)
Use Pipenv to manage Python environments (#1553) * Use Pipenv to manage Python environments * Rename PIPENV_ARGS to PIPENV_PYTHON_VERSION to avoid confusion. Also remove two unused variables
2018-06-22 03:01:23 +02:00
cd $(PYENVSRC) && pipenv run python setup.py build bdist_wheel --universal
Add a Dockerfile for the Pulumi CLI This introduces a Dockerfile for the Pulumi CLI. This makes it easier to develop and test the engine in a self-contained environment, in addition to being suitable for running the actual CLI itself. For instance, $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up will run the Pulumi program mounted under the /app volume. This will be used in some upcoming CI/CD scenarios. This uses multi-stage builds, and Debian Stretch as the base, for relatively fast and lean build times and resulting images. We are intentional about restoring dep packages independent of the actual source code so that we don't end up needlessly re-depping, which can consume quite a bit of time. After fixing https://github.com/pulumi/pulumi/issues/1986, we should explore an Alpine base image option. I made the decision to keep this image scoped to just the Go builds. Therefore, none of the actual SDK packages themselves are built, just the engine, CLI, and language plugins for Node.js, Python, and Go. It's possible to create a mega-container that has all of these full environments so that we can rebuild them too, but for now I figured it was better to rely on package management for them. Another alternative would have been to install released binaries, rather than building them. To keep the useful flow for development, however, I decided to go the build route for now. If we build at the same hashes, the resulting binaries "should" be ~identical anyhow. I've created a pulumi/pulumi Docker Hub repo that we can publish this into. For now, there is no CI publishing of the image. This fixes pulumi/pulumi#1991.
2018-09-29 20:43:35 +02:00
build_plugin::
move pkg/version -> sdk/go/common/version
2020-03-18 23:25:25 +01:00
go install -ldflags "-X github.com/pulumi/pulumi/sdk/go/common/version.Version=${VERSION}" ${LANGHOST_PKG}
Prefer Python2 tools; fail-fast when wrong This change includes a few things: 1) Prefer python2 and pip2 when on the PATH, over the undecorated names python and pip. This is the standard convention for package managers like Pip, etc., to support Python2 and Python3 side-by-side. 2) Fail-fast if neither can be found on the PATH. 3) Check the reported version number for python, pip, and pylint, and fail-fast if it doesn't report back 2.7, just to safeguard against undecorated binaries with unsupported versions. Also, we had not listed wheel as a dependency in the requirements.txt file. This needs to be there to support building bdist_wheels. Fixed.
2018-03-07 00:29:52 +01:00
Add a Dockerfile for the Pulumi CLI This introduces a Dockerfile for the Pulumi CLI. This makes it easier to develop and test the engine in a self-contained environment, in addition to being suitable for running the actual CLI itself. For instance, $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up will run the Pulumi program mounted under the /app volume. This will be used in some upcoming CI/CD scenarios. This uses multi-stage builds, and Debian Stretch as the base, for relatively fast and lean build times and resulting images. We are intentional about restoring dep packages independent of the actual source code so that we don't end up needlessly re-depping, which can consume quite a bit of time. After fixing https://github.com/pulumi/pulumi/issues/1986, we should explore an Alpine base image option. I made the decision to keep this image scoped to just the Go builds. Therefore, none of the actual SDK packages themselves are built, just the engine, CLI, and language plugins for Node.js, Python, and Go. It's possible to create a mega-container that has all of these full environments so that we can rebuild them too, but for now I figured it was better to rely on package management for them. Another alternative would have been to install released binaries, rather than building them. To keep the useful flow for development, however, I decided to go the build route for now. If we build at the same hashes, the resulting binaries "should" be ~identical anyhow. I've created a pulumi/pulumi Docker Hub repo that we can publish this into. For now, there is no CI publishing of the image. This fixes pulumi/pulumi#1991.
2018-09-29 20:43:35 +02:00
build:: build_package build_plugin
Get the empty Python program working This change gets enough of the Python SDK up and running that the empty Python program will work. Mostly just scaffolding, but the basic structure is now in place. The primary remaining work is to wire up resource creation to the gRPC interfaces. In summary: * The basic structure is as follows: - Everything goes into sdk/python/. - sdk/python/cmd/pulumi-langhost-python is a Go language host that simply knows how to spawn Python processes to run out entrypoint in response to requests by the engine. - sdk/python/cmd/pulumi-langhost-python-exec is a little Python shim that is invoked by the language host to run Python programs, and is responsible for setting up the minimal goo before we can do so (RPC connections and the like). - sdk/python/lib/ contains a Python Pip package suitable for PyPi. - In there, we have two packages: the root pulumi package that contains all of the basic Pulumi programming model abstractions, and pulumi.runtime, which contains the implementation of resource registration, RPC interfacing with the engine, and so on. * Add logic in our test framework to conditionalize on the language type and react accordingly. This will allow us to skip Yarn for Python projects and eventually run Pip if there's a requirements.txt. * Created the basic project structure, including all of the usual Make targets for installing into the proper places. * Building also runs Pylint and we are clean. There are a few other minor things in here: * Add an "empty" test for both Node.js and Python. These pass. * Fix an existing bug in plugin shutdown logic. At some point, we started waiting for stderr/stdout to flush before shutting down the plugin; but if certain failures happen "early" during the plugin launch process, these channels will never get initialized and so waiting for them deadlocks. * Recently we seem to have added logic to delete test temp directories if a failure happened during initialization of said temp directories. This is unfortunate, because you often need to look at the temp directory to see what failed. We already clean them up elsewhere after the full test completes successfully, so I don't think we need to be doing this, and I've removed it. Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
lint::
Enable mypy (#3758)
2020-01-17 23:45:08 +01:00
pipenv run mypy ./lib/pulumi --config-file=mypy.ini
Enable and fix Pylint warnings (#2146) * Enable pylint warnings * Fix or ignore pylint warnings
2018-11-01 22:29:06 +01:00
pipenv run pylint ./lib/pulumi --rcfile=.pylintrc
Implement the Python langhost RPC server This change adds a basic Python langhost RPC server. It's fairly barebones and merely acts as a jumping off point for the Pulumi engine to spawn a Python program. The host is written in Go, in contrast to implementing the host in Python, and more closely resembles how I expect the Node.js language host to work once pulumi/pulumi#331 is done.
2017-12-28 03:11:38 +01:00
Add a Dockerfile for the Pulumi CLI This introduces a Dockerfile for the Pulumi CLI. This makes it easier to develop and test the engine in a self-contained environment, in addition to being suitable for running the actual CLI itself. For instance, $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up will run the Pulumi program mounted under the /app volume. This will be used in some upcoming CI/CD scenarios. This uses multi-stage builds, and Debian Stretch as the base, for relatively fast and lean build times and resulting images. We are intentional about restoring dep packages independent of the actual source code so that we don't end up needlessly re-depping, which can consume quite a bit of time. After fixing https://github.com/pulumi/pulumi/issues/1986, we should explore an Alpine base image option. I made the decision to keep this image scoped to just the Go builds. Therefore, none of the actual SDK packages themselves are built, just the engine, CLI, and language plugins for Node.js, Python, and Go. It's possible to create a mega-container that has all of these full environments so that we can rebuild them too, but for now I figured it was better to rely on package management for them. Another alternative would have been to install released binaries, rather than building them. To keep the useful flow for development, however, I decided to go the build route for now. If we build at the same hashes, the resulting binaries "should" be ~identical anyhow. I've created a pulumi/pulumi Docker Hub repo that we can publish this into. For now, there is no CI publishing of the image. This fixes pulumi/pulumi#1991.
2018-09-29 20:43:35 +02:00
install_package::
Get the empty Python program working This change gets enough of the Python SDK up and running that the empty Python program will work. Mostly just scaffolding, but the basic structure is now in place. The primary remaining work is to wire up resource creation to the gRPC interfaces. In summary: * The basic structure is as follows: - Everything goes into sdk/python/. - sdk/python/cmd/pulumi-langhost-python is a Go language host that simply knows how to spawn Python processes to run out entrypoint in response to requests by the engine. - sdk/python/cmd/pulumi-langhost-python-exec is a little Python shim that is invoked by the language host to run Python programs, and is responsible for setting up the minimal goo before we can do so (RPC connections and the like). - sdk/python/lib/ contains a Python Pip package suitable for PyPi. - In there, we have two packages: the root pulumi package that contains all of the basic Pulumi programming model abstractions, and pulumi.runtime, which contains the implementation of resource registration, RPC interfacing with the engine, and so on. * Add logic in our test framework to conditionalize on the language type and react accordingly. This will allow us to skip Yarn for Python projects and eventually run Pip if there's a requirements.txt. * Created the basic project structure, including all of the usual Make targets for installing into the proper places. * Building also runs Pylint and we are clean. There are a few other minor things in here: * Add an "empty" test for both Node.js and Python. These pass. * Fix an existing bug in plugin shutdown logic. At some point, we started waiting for stderr/stdout to flush before shutting down the plugin; but if certain failures happen "early" during the plugin launch process, these channels will never get initialized and so waiting for them deadlocks. * Recently we seem to have added logic to delete test temp directories if a failure happened during initialization of said temp directories. This is unfortunate, because you often need to look at the temp directory to see what failed. We already clean them up elsewhere after the full test completes successfully, so I don't think we need to be doing this, and I've removed it. Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
cp ./cmd/pulumi-language-python-exec "$(PULUMI_BIN)"
Python Dynamic Providers (#2900) Dynamic providers in Python. This PR uses [dill](https://pypi.org/project/dill/) for code serialization, along with a customization to help ensure deterministic serialization results. One notable limitation - which I believe is a general requirement of Python - is that any serialization of Python functions must serialize byte code, and byte code is not safely versioned across Python versions. So any resource created with Python `3.x.y` can only be updated by exactly the same version of Python. This is very constraining, but it's not clear there is any other option within the realm of what "dynamic providers" are as a feature. It is plausible that we could ensure that updates which only update the serialized provider can avoid calling the dynamic provider operations, so that version updates could still be accomplished. We can explore this separately. ```py from pulumi import ComponentResource, export, Input, Output from pulumi.dynamic import Resource, ResourceProvider, CreateResult, UpdateResult from typing import Optional from github import Github, GithubObject auth = "<auth token>" g = Github(auth) class GithubLabelArgs(object): owner: Input[str] repo: Input[str] name: Input[str] color: Input[str] description: Optional[Input[str]] def __init__(self, owner, repo, name, color, description=None): self.owner = owner self.repo = repo self.name = name self.color = color self.description = description class GithubLabelProvider(ResourceProvider): def create(self, props): l = g.get_user(props["owner"]).get_repo(props["repo"]).create_label( name=props["name"], color=props["color"], description=props.get("description", GithubObject.NotSet)) return CreateResult(l.name, {**props, **l.raw_data}) def update(self, id, _olds, props): l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id) l.edit(name=props["name"], color=props["color"], description=props.get("description", GithubObject.NotSet)) return UpdateResult({**props, **l.raw_data}) def delete(self, id, props): l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id) l.delete() class GithubLabel(Resource): name: Output[str] color: Output[str] url: Output[str] description: Output[str] def __init__(self, name, args: GithubLabelArgs, opts = None): full_args = {'url':None, 'description':None, 'name':None, 'color':None, **vars(args)} super().__init__(GithubLabelProvider(), name, full_args, opts) label = GithubLabel("foo", GithubLabelArgs("lukehoban", "todo", "mylabel", "d94f0b")) export("label_color", label.color) export("label_url", label.url) ``` Fixes https://github.com/pulumi/pulumi/issues/2902.
2019-07-19 19:18:25 +02:00
cp ./dist/pulumi-resource-pulumi-python "$(PULUMI_BIN)"
Add a Dockerfile for the Pulumi CLI This introduces a Dockerfile for the Pulumi CLI. This makes it easier to develop and test the engine in a self-contained environment, in addition to being suitable for running the actual CLI itself. For instance, $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up will run the Pulumi program mounted under the /app volume. This will be used in some upcoming CI/CD scenarios. This uses multi-stage builds, and Debian Stretch as the base, for relatively fast and lean build times and resulting images. We are intentional about restoring dep packages independent of the actual source code so that we don't end up needlessly re-depping, which can consume quite a bit of time. After fixing https://github.com/pulumi/pulumi/issues/1986, we should explore an Alpine base image option. I made the decision to keep this image scoped to just the Go builds. Therefore, none of the actual SDK packages themselves are built, just the engine, CLI, and language plugins for Node.js, Python, and Go. It's possible to create a mega-container that has all of these full environments so that we can rebuild them too, but for now I figured it was better to rely on package management for them. Another alternative would have been to install released binaries, rather than building them. To keep the useful flow for development, however, I decided to go the build route for now. If we build at the same hashes, the resulting binaries "should" be ~identical anyhow. I've created a pulumi/pulumi Docker Hub repo that we can publish this into. For now, there is no CI publishing of the image. This fixes pulumi/pulumi#1991.
2018-09-29 20:43:35 +02:00
install_plugin::
Get the empty Python program working This change gets enough of the Python SDK up and running that the empty Python program will work. Mostly just scaffolding, but the basic structure is now in place. The primary remaining work is to wire up resource creation to the gRPC interfaces. In summary: * The basic structure is as follows: - Everything goes into sdk/python/. - sdk/python/cmd/pulumi-langhost-python is a Go language host that simply knows how to spawn Python processes to run out entrypoint in response to requests by the engine. - sdk/python/cmd/pulumi-langhost-python-exec is a little Python shim that is invoked by the language host to run Python programs, and is responsible for setting up the minimal goo before we can do so (RPC connections and the like). - sdk/python/lib/ contains a Python Pip package suitable for PyPi. - In there, we have two packages: the root pulumi package that contains all of the basic Pulumi programming model abstractions, and pulumi.runtime, which contains the implementation of resource registration, RPC interfacing with the engine, and so on. * Add logic in our test framework to conditionalize on the language type and react accordingly. This will allow us to skip Yarn for Python projects and eventually run Pip if there's a requirements.txt. * Created the basic project structure, including all of the usual Make targets for installing into the proper places. * Building also runs Pylint and we are clean. There are a few other minor things in here: * Add an "empty" test for both Node.js and Python. These pass. * Fix an existing bug in plugin shutdown logic. At some point, we started waiting for stderr/stdout to flush before shutting down the plugin; but if certain failures happen "early" during the plugin launch process, these channels will never get initialized and so waiting for them deadlocks. * Recently we seem to have added logic to delete test temp directories if a failure happened during initialization of said temp directories. This is unfortunate, because you often need to look at the temp directory to see what failed. We already clean them up elsewhere after the full test completes successfully, so I don't think we need to be doing this, and I've removed it. Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
GOBIN=$(PULUMI_BIN) go install \
move pkg/version -> sdk/go/common/version
2020-03-18 23:25:25 +01:00
-ldflags "-X github.com/pulumi/pulumi/sdk/go/common/version.Version=${VERSION}" ${LANGHOST_PKG}
Implement the Python langhost RPC server This change adds a basic Python langhost RPC server. It's fairly barebones and merely acts as a jumping off point for the Pulumi engine to spawn a Python program. The host is written in Go, in contrast to implementing the host in Python, and more closely resembles how I expect the Node.js language host to work once pulumi/pulumi#331 is done.
2017-12-28 03:11:38 +01:00
Add a Dockerfile for the Pulumi CLI This introduces a Dockerfile for the Pulumi CLI. This makes it easier to develop and test the engine in a self-contained environment, in addition to being suitable for running the actual CLI itself. For instance, $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up will run the Pulumi program mounted under the /app volume. This will be used in some upcoming CI/CD scenarios. This uses multi-stage builds, and Debian Stretch as the base, for relatively fast and lean build times and resulting images. We are intentional about restoring dep packages independent of the actual source code so that we don't end up needlessly re-depping, which can consume quite a bit of time. After fixing https://github.com/pulumi/pulumi/issues/1986, we should explore an Alpine base image option. I made the decision to keep this image scoped to just the Go builds. Therefore, none of the actual SDK packages themselves are built, just the engine, CLI, and language plugins for Node.js, Python, and Go. It's possible to create a mega-container that has all of these full environments so that we can rebuild them too, but for now I figured it was better to rely on package management for them. Another alternative would have been to install released binaries, rather than building them. To keep the useful flow for development, however, I decided to go the build route for now. If we build at the same hashes, the resulting binaries "should" be ~identical anyhow. I've created a pulumi/pulumi Docker Hub repo that we can publish this into. For now, there is no CI publishing of the image. This fixes pulumi/pulumi#1991.
2018-09-29 20:43:35 +02:00
install:: install_package install_plugin
Get the empty Python program working This change gets enough of the Python SDK up and running that the empty Python program will work. Mostly just scaffolding, but the basic structure is now in place. The primary remaining work is to wire up resource creation to the gRPC interfaces. In summary: * The basic structure is as follows: - Everything goes into sdk/python/. - sdk/python/cmd/pulumi-langhost-python is a Go language host that simply knows how to spawn Python processes to run out entrypoint in response to requests by the engine. - sdk/python/cmd/pulumi-langhost-python-exec is a little Python shim that is invoked by the language host to run Python programs, and is responsible for setting up the minimal goo before we can do so (RPC connections and the like). - sdk/python/lib/ contains a Python Pip package suitable for PyPi. - In there, we have two packages: the root pulumi package that contains all of the basic Pulumi programming model abstractions, and pulumi.runtime, which contains the implementation of resource registration, RPC interfacing with the engine, and so on. * Add logic in our test framework to conditionalize on the language type and react accordingly. This will allow us to skip Yarn for Python projects and eventually run Pip if there's a requirements.txt. * Created the basic project structure, including all of the usual Make targets for installing into the proper places. * Building also runs Pylint and we are clean. There are a few other minor things in here: * Add an "empty" test for both Node.js and Python. These pass. * Fix an existing bug in plugin shutdown logic. At some point, we started waiting for stderr/stdout to flush before shutting down the plugin; but if certain failures happen "early" during the plugin launch process, these channels will never get initialized and so waiting for them deadlocks. * Recently we seem to have added logic to delete test temp directories if a failure happened during initialization of said temp directories. This is unfortunate, because you often need to look at the temp directory to see what failed. We already clean them up elsewhere after the full test completes successfully, so I don't think we need to be doing this, and I've removed it. Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
test_fast::
Use Pipenv to manage Python environments (#1553) * Use Pipenv to manage Python environments * Rename PIPENV_ARGS to PIPENV_PYTHON_VERSION to avoid confusion. Also remove two unused variables
2018-06-22 03:01:23 +02:00
pipenv run pip install ./env/src
Implement RPC for Python 3 (#2111) * Implement RPC for Python 3 * Try not setting PYTHONPATH * Remove PYTHONPATH line * Implement Invoke for Python 3 * Implement register resource * progress * Rewrite the whole thing * Fix a few bugs * All tests pass * Fix an abnormal shutdown bug * CR feedback * Provide a hook for resources to rename properties As dictionaries and other classes come from the engine, the translate_property hook can be used to intercept them and rename properties if desired. * Fix variable names and comments * Disable Python integration tests for now
2018-10-31 21:35:31 +01:00
pipenv run python -m unittest discover -s lib/test -v
Added dist target for make, will help with Homebrew (#1731) * Added dist target for make, will help with Homebrew * Try to install go dependencies before building * Make sure dep ensure is called before trying to build SDKs * Removed dep ensure from dist initial step
2018-08-08 22:00:42 +02:00
Simplify API for passing providers to a ComponentResource. (#2609)
2019-06-07 01:20:12 +02:00
test_all:: test_fast
Added dist target for make, will help with Homebrew (#1731) * Added dist target for make, will help with Homebrew * Try to install go dependencies before building * Make sure dep ensure is called before trying to build SDKs * Removed dep ensure from dist initial step
2018-08-08 22:00:42 +02:00
dist::
move pkg/version -> sdk/go/common/version
2020-03-18 23:25:25 +01:00
go install -ldflags "-X github.com/pulumi/pulumi/sdk/go/common/version.Version=${VERSION}" ${LANGHOST_PKG}
Use go env to resolve GOPATH but escape it properly
2018-08-10 19:56:29 +02:00
cp ./cmd/pulumi-language-python-exec "$$(go env GOPATH)"/bin/
Python Dynamic Providers (#2900) Dynamic providers in Python. This PR uses [dill](https://pypi.org/project/dill/) for code serialization, along with a customization to help ensure deterministic serialization results. One notable limitation - which I believe is a general requirement of Python - is that any serialization of Python functions must serialize byte code, and byte code is not safely versioned across Python versions. So any resource created with Python `3.x.y` can only be updated by exactly the same version of Python. This is very constraining, but it's not clear there is any other option within the realm of what "dynamic providers" are as a feature. It is plausible that we could ensure that updates which only update the serialized provider can avoid calling the dynamic provider operations, so that version updates could still be accomplished. We can explore this separately. ```py from pulumi import ComponentResource, export, Input, Output from pulumi.dynamic import Resource, ResourceProvider, CreateResult, UpdateResult from typing import Optional from github import Github, GithubObject auth = "<auth token>" g = Github(auth) class GithubLabelArgs(object): owner: Input[str] repo: Input[str] name: Input[str] color: Input[str] description: Optional[Input[str]] def __init__(self, owner, repo, name, color, description=None): self.owner = owner self.repo = repo self.name = name self.color = color self.description = description class GithubLabelProvider(ResourceProvider): def create(self, props): l = g.get_user(props["owner"]).get_repo(props["repo"]).create_label( name=props["name"], color=props["color"], description=props.get("description", GithubObject.NotSet)) return CreateResult(l.name, {**props, **l.raw_data}) def update(self, id, _olds, props): l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id) l.edit(name=props["name"], color=props["color"], description=props.get("description", GithubObject.NotSet)) return UpdateResult({**props, **l.raw_data}) def delete(self, id, props): l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id) l.delete() class GithubLabel(Resource): name: Output[str] color: Output[str] url: Output[str] description: Output[str] def __init__(self, name, args: GithubLabelArgs, opts = None): full_args = {'url':None, 'description':None, 'name':None, 'color':None, **vars(args)} super().__init__(GithubLabelProvider(), name, full_args, opts) label = GithubLabel("foo", GithubLabelArgs("lukehoban", "todo", "mylabel", "d94f0b")) export("label_color", label.color) export("label_url", label.url) ``` Fixes https://github.com/pulumi/pulumi/issues/2902.
2019-07-19 19:18:25 +02:00
cp ./dist/pulumi-resource-pulumi-python "$$(go env GOPATH)"/bin/
Reference in a new issue Copy permalink