This change does two things:
- It ensures that as part of publishing the SDK, we also publish an
updated pulumi/pulumi docker image (tagged with both `latest` and
`vX.Y.Z`
- Makes this image published by this repository less perscriptive in a
workflow. The instead of having wrapper scripts that try to invoke
Pulumi based on conventions. It is now just a base image that has
the pulumi CLI installed, as well as the SDKs for the major cloud
providers. We'll use this base layer in our github actions image,
which will layer on a github actions centric workflow
Fixes#1991
We publish some artifacts from this repository, which are specific to
an OS. In the past, we only had a single Linux leg and so we did not
need any additional conditional logic.
However, we now have multiple Linux legs (across different node
versions) running in parallel, so if we try to publish the same
artifacts at the same time, we can run into issues because two `aws s3
cp` jobs try to write to the same file at the same time.
Follow similar logic to our skips we do around publishing NPM and PyPI
packages here, as well.
The logic to append the `+dirty` tag to a generated version on Windows
was incorrect in the case where we were producing a version for a
tagged build. In that case we would append `+dirty` when the work tree
was clean and not append it when it was dirty.
With this change, you can now use the `dev` tag of `@pulumi/pulumi` to
get the latest version of the SDK and install it from get.pulumi.com
This is helpful for some of our internal testing.
Under our old versioning system, when we started a new point release,
we'd tag the HEAD commit of master with a tag like `v0.16.6-dev` and
our scripts would use this to generate a new version number. This
required a great deal of gymnastics when producing a release and
caused us to litter these -dev tags everywhere.
To improve this, we change version number generation to the following
strategy:
1. If the commit we are building has a tag applied to it, use that tag
as the version (appending the dirty bit metadata to the version, if
needed).
2. If the commit we are bulding does not have a tag applied to it,
take the version from the next reachable tag, increment the patch
version and then append the `-dev` pre-release tag. As part of this,
we also make a slight tweek to our semver generation such that instead
of `-dev<TIMESTAMP>` we use `-dev.<TIMESTAMP>` which is more in line
with what semver recommends.
* Remove TODO for issue since fixed in PPCs.
* Update issue reference to source
* Update comment wording
* Remove --ppc arg of stack init
* Remove PPC references in int. testing fx
* Remove vestigial PPC API types
For CI situations, we'll support a simple stacks map file, e.g.
{
"refs/heads/master": "production",
"refs/heads/testing": "test'
}
and, when PULUMI_CI is set, we'll use it to select the stack.
This is purely for experimental purposes; we're not sure this is
exactly what we want right now, but it's better than the manual
munging we've been doing with various bash scripts, etc. right now.
This adds a package installation step just before calling the
`pulumi` CLI, making it easier to just volume mount a project
without needing to have `npm install`ed its contents yet.
Pulumi used to have a much more complicated install process, whereas
today, this is no longer the case. You simply unpack pulumi to a
folder of your choice, add it to the `$PATH` and then go.
The `install.sh` was writtten back when Pulumi had to be installed
into its own directory. It assumed that it "owned" this directory and
when the script hit an error would clean up its half processed state
before trying to exit. While this was fine out of the box (since we
default to installing to `/usr/local/pulumi`) if you overrode the
install location to just say `/usr/local` *and* we hit an error in the
script, the script would try to remove everything from `/usr/local` as
part of cleaning itself up.
Since we no longer need any of this extra install logic, we'll just
remove `install.sh` completely. The SDK tarball will now contain a
single top level directory (named `pulumi`) with all of our binaries
under it. Manual installs will now just mean unpacking the tarball
somewhere and putting that `pulumi` folder on your path, or as a
simplification, copying all the binaries from the `pulumi` folder into
an existing folder that is already on your path.
This also removes the need to ever ask the user to `sudo` during an
install. Users now have complete control over where they put our
binaries, which is exactly what you want from a manual install
process.
Stop cloning pulumi/home. This doesn't work in Travis because public
repositories can not have private SSH keys, which we'd need to clone
this repository. All the scripts we consume from there are now in
pulumi/scripts and so we'll just consume them from there.
This change includes the Python and Golang language hosts in the Windows
SDK. As part of this change, I had to adjust how we launched the second
stage of the language host, since we can't depend on the shebang, so now
we invoke `python` passing the executor and then the arguments.
Fixes#1509
Previously, we published builds to rel.pulumi.com and only put actual
released builds (eg. rc's and final builds) on get.pulumi.com. We
should just publish all of the SDK builds to get.pulumi.com.
This also makes a slight tweak to the filename of the package we
upload (we took the switch over to get.pulumi.com to make this change
and now that we are uploading automatically, we need to encode this
change instead of doing it by hand).
All scripts that are generally useful across all builds have been moved
into `pulumi/scripts`. These changes clone that repository and retarget
the various scripts to their new location.
We retained these modules to support using v0.11.X and earlier
versions of @pulumi/pulumi, which required a native module to do
closure serialization. 0.12.X does not need this, so lets stop
including it.
Our logic for converting npm style versions to PEP-440 style versions
was not correct in some cases. This change fixes this.
As part of this change we no longer produce a NPM version that would
be just X.Y.Z-dev, instead for development versions we always include
both the timestamp of the commit and the commit hash.
Instead of trying to use a bunch of sed logic to do our conversions,
we now have a small go program that uses a newly added library in
pkg/util. A side effect of this is that we can more easily write tests
to ensure the conversion works as expected.
Fixes#1243
Instead of using a shell script to jump from the language host into
node, just invoke node directly. This makes our start-up path a little
simpler to understand and indirectly fixespulumi/home#156, where we
would fail on Windows if the `-exec` script was in a folder that had
spaces in it (due to a subtle interaction between how go launches cmd
files and how cmd.exe parses arguments).
This change eliminates our dependencies on the SDK repo. Now that
SDKs are comprised solely of pulumi/pulumi artifacts, a separate repo
isn't required. This allows us to simplify some of the distribution.
The install.sh script is modified slightly, to permit overriding the
default install location using $PULUMI_INSTALL_PATH.
While we no longer use the native runtime module, older versions of
@pulumi/pulumi still require it. Let's continue to have the launcher
put the native module location on the `$PATH`. And we'll include them
in the SDK for a while longer.
Fixes#1177
After the move to stop including packages in the SDK, we no longer
published an SDK per build. This corrects this. Since the only things
in the SDK today are the language plugins and the CLI itself, we can
publish an SDK per build from pulumi/pulumi.
This change re-uses the existing infrastructure we have in
pulumi/sdk.
Fixes#1076
* Lift snapshot management out of the engine
This PR is a prerequisite for parallelism by addressing a major problem
that the engine has to deal with when performing parallel resource
construction: parallel mutation of the global snapshot. This PR adds
a `SnapshotManager` type that is responsible for maintaining and
persisting the current resource snapshot. It serializes all reads and
writes to the global snapshot and persists the snapshot to persistent
storage upon every write.
As a side-effect of this, the core engine no longer needs to know about
snapshot management at all; all snapshot operations can be handled as
callbacks on deployment events. This will greatly simplify the
parallelization of the core engine.
Worth noting is that the core engine will still need to be able to read
the current snapshot, since it is interested in the dependency graphs
contained within. The full implications of that are out of scope of this
PR.
Remove dead code, Steps no longer need a reference to the plan iterator that created them
Fixing various issues that arise when bringing up pulumi-aws
Line length broke the build
Code review: remove dead field, fix yaml name error
Rebase against master, provide implementation of StackPersister for cloud backend
Code review feedback: comments on MutationStatus, style in snapshot.go
Code review feedback: move SnapshotManager to pkg/backend, change engine to use an interface SnapshotManager
Code review feedback: use a channel for synchronization
Add a comment and a new test
* Maintain two checkpoints, an immutable base and a mutable delta, and
periodically merge the two to produce snapshots
* Add a lot of tests - covers all of the non-error paths of BeginMutation and End
* Fix a test resource provider
* Add a few tests, fix a few issues
* Rebase against master, fixed merge
* Implement closure scope chain analysis in pure TypeScript
This change makes use of four V8 intrinsics to avoid having to use a
native module to inspect the scope chains of live Function objects. This
unfortunately leads to the limitation of not allowing captures of 'this'
in arrow functions, but that is something we are willing to live with
for now.
* Remove native module build and restore from the Makefile
* CR feedback: Be a little more efficient when scanning the scope chain
* Nuke everything related to custom Node versions and the native Node module
* CR feedback: rename native.ts -> v8.ts, document some interfaces in v8.ts
`npm publish`'s default was to tag the package we published with
`latest` tag. The NPM ecosystem has expected semantics around this
tag (it uses it by default if you don't pass a version).
From the NPM Docs:
> Typically, projects only use the latest tag for stable release
> versions, and use other tags for unstable versions such as
> prereleases.
We were not doing this, but now we will. We'll have a new tag `dev`
which is the latest build out of CI, and we'll tag builds without a
pre-release tag with "latest".
This change actually makes our Python version numbers conformant
to PEP440. Previously we were including the Git commit hash in the
alpha "version number" part, which is incorrect. This simply led to
warnings upon publication and installation, but that warning very
clearly states that support for invalid versions will stop at some
point. This change puts any "informative" parts, like the Git hash,
inside of a local version tag, where such things are permitted.
Also move away from the inline sed silliness so that we can more
easily share this logic across all of our repos.