Previously, the checkpoint manifest contained the full path to a plugin
binary, in places of its friendly name. Now that we must move to a model
where we install plugins in the PPC based on the manifest contents, we
actually need to store the name, in addition to the version (which is
already there). We still also capture the path for debugging purposes.
I was reminded of this yesterday with unprintable characters as I
debugged some things on Windows. Inspired by Yarn, this change adds
a new flag --emoji (-e for short) that can be used to control whether
we show ASCII-only characters or not in the console. On Mac, it
defaults to true, and on Windows and Linux, it defaults to false.
This also brings back the retro ASCII-friendly progress spinner
when --emoji is disabled.
Backup copies of local stack checkpoints are now saved to the
user's home directory (`~/.pulumi/backups`) by default.
This enables users to recover after accidentally deleting their
local `.pulumi` directory (e.g. via `git clean`).
The behavior can be disabled by setting the
PULUMI_DISABLE_CHECKPOINT_BACKUPS environment variable, which
we use to disable backups when running all tests other than the
test for this functionality.
We have had a long-standing bug in here where we waiting on a
stdout channel that never got populated, when the language plugin
fails to load entirely. This would lead to hung processes. The
fix is simple: only wait for stdout/stderr channels to drain that
have actually been wired up to enjoy the requisite signaling.
This adds support for two things:
* Installing all plugins that a project requires with a single command:
$ pulumi plugin install
* Listing the plugins that this project requires:
$ pulumi plugin ls --project
$ pulumi plugin ls -p
Prior to this change, we had a flat list of files in the
~/.pulumi/plugins directory. This was simple but unfortunately
too naive, since we in fact have multi-file plugins already.
Dumping them in the same directory increases the risk of a
collision. Instead, let's put them in their own directories.
This means, for example, you'll see things like
~/.pulumi/plugins/
resource-aws-v0.11.0-dev-8-g57a0d62/
README.txt
pulumi-resource-aws
Notice that the binary name stays the same -- e.g., in this
case pulumi-resource-aws -- and does not include the version.
This makes it simple to add it to your $PATH in the usual ways
and have it loaded as a preferred location.
The API/REST logic auto-prepended "/api", which we don't want
for the release downloads. This change just alters callsites
to specify the full path (which I prefer being explicit anyway).
This brings back the Node.js language plugin's GetRequiredPlugins
function, reimplemented in Go now that the language host has been
rewritten from JavaScript. Fairly rote translation, along with
some random fixes required to get tests passing again.
This change implements the Node.js language host's GetRequiredPlugins
function. This merely scans all node_modules/*/package.json files in
the program directory, looking for those that have associated plugins.
It returns a list of any found along with their version numbers.
This change adds a GetRequiredPlugins RPC method to the language
host, enabling us to query it for its list of plugin requirements.
This is language-specific because it requires looking at the set
of dependencies (e.g., package.json files).
It also adds a call up front during any update/preview operation
to compute the set of plugins and require that they are present.
These plugins are populated in the cache and will be used for all
subsequent plugin-related operations during the engine's activity.
We now cache the language plugins, so that we may load them
eagerly too, which we never did previously due to the fact that
we needed to pass the monitor address at load time. This was a
bit bizarre anyhow, since it's really the Run RPC function that
needs this information. So, to enable caching and eager loading
-- which we need in order to invoke GetRequiredPlugins -- the
"phone home" monitor RPC address is passed at Run time.
In a subsequent change, we will switch to faulting in the plugins
that are missing -- rather than erroring -- in addition to
supporting the `pulumi plugin install` CLI command.
This change introduces a workspace.GetPluginPath function that probes
the central workspace cache of plugins for a matching plugin binary that
matches the desired kind, name, and, optionally, version. It also permits
overriding this with $PATH for developer scenarios.
The analyzer, language, and resource plugin logic now uses this function
for deciding which binary path to load at runtime.
This change implements basic plugin management, but we do not yet
actually use the plugins for anything (that comes next).
Plugins are stored in `~/.pulumi/plugins`, and are expected to be
in the format `pulumi-<KIND>-<NAME>-v<VERSION>[.exe]`. The KIND is
one of `analyzer`, `language`, or `resource`, the NAME is a hyphen-
delimited name (e.g., `aws` or `foo-bar`), and VERSION is the
plugin's semantic version (e.g., `0.9.11`, `1.3.7-beta.a736cf`, etc).
This commit includes four new CLI commands:
* `pulumi plugin` is the top-level plugin command. It does nothing
but show the help text for associated child commands.
* `pulumi plugin install` can be used to install plugins manually.
If run with no additional arguments, it will compute the set of
plugins used by the current project, and download them all. It
may be run to explicitly download a single plugin, however, by
invoking it as `pulumi plugin install KIND NAME VERSION`. For
example, `pulumi plugin install resource aws v0.9.11`. By default,
this command uses the cloud backend in the usual way to perform the
download, although a separate URL may be given with --cloud-url,
just like all other commands that interact with our backend service.
* `pulumi plugin ls` lists all plugins currently installed in the
plugin cache. It displays some useful statistics, like the size
of the plugin, when it was installed, when it was last used, and
so on. It sorts the display alphabetically by plugin name, and
for plugins with multiple versions, it shows the newest at the top.
The command also summarizes how much disk space is currently being
consumed by the plugin cache. There are no filtering capabilities yet.
* `pulumi plugin prune` will delete plugins from the cache. By
default, when run with no arguments, it will delete everything.
It may be run with additional arguments, KIND, NAME, and VERSION,
each one getting more specific about what it will delete. For
instance, `pulumi plugin prune resource aws` will delete all AWS
plugin versions, while `pulumi plugin prune resource aws <0.9`
will delete all AWS plugins before version 0.9. Unless --yes is
passed, the command will confirm the deletion with a count of how
many plugins will be affected by the command.
We do not yet actually download plugins on demand yet. That will
come in a subsequent change.
This change includes a handful of stack-related CLI formatting
improvements that I've been noodling on in the background for a while,
based on things that tend to trip up demos and the inner loop workflow.
This includes:
* If `pulumi stack select` is run by itself, use an interactive
CLI menu to let the user select an existing stack, or choose to
create a new one. This looks as follows
$ pulumi stack select
Please choose a stack, or choose to create a new one:
abcdef
babblabblabble
> currentlyselected
defcon
<create a new stack>
and is navigated in the usual way (key up, down, enter).
* If a stack name is passed that does not exist, prompt the user
to ask whether s/he wants to create one on-demand. This hooks
interesting moments in time, like `pulumi stack select foo`,
and cuts down on the need to run additional commands.
* If a current stack is required, but none is currently selected,
then pop the same interactive menu shown above to select one.
Depending on the command being run, we may or may not show the
option to create a new stack (e.g., that doesn't make much sense
when you're running `pulumi destroy`, but might when you're
running `pulumi stack`). This again lets you do with a single
command what would have otherwise entailed an error with multiple
commands to recover from it.
* If you run `pulumi stack init` without any additional arguments,
we interactively prompt for the stack name. Before, we would
error and you'd then need to run `pulumi stack init <name>`.
* Colorize some things nicely; for example, now all prompts will
by default become bright white.
This addresses pulumi/pulumi#446: what we used to call "package" is
now called "project". This has gotten more confusing over time, now
that we're doing real package management.
Also fixespulumi/pulumi#426, while in here.
We've seen yarn fail from time to time during our integration
tests (usually timeouts talking to the npm registry) so let's add a
few retries to all yarn commands.
When reading a configuration value from standard in and standard in is
not connected to a terminal, read until EOF and then trim a trailing
newline (if present) to get the value
Fixes#822
Resources in the checkpoint file which are pending-delete represent old versions of resources which are no longer part of the active deployment. For purposes of constructing the active resource tree, we should skip these resources.
As it stands, we currently hammer the service's update logs endpoint in
a tight loop while waiting for a deployment to complete. This is not
necessary, and can indeed be deletrious to the user experience: it
appears that this may be exacerbating some mysterious 500 responses from
API gateway.
These changes add a brief sleep in the relevant loop that waits for 5
seconds if the last call produced new log entries or 15 seconds if it
did not.
Fixes#844.
Today we don't send any version information with API requests to the service, so we cannot make breaking changes between versions of the backend API while preserving backwards compatibility.
This PR adds a `User-Agent` header with REST requests that sends a CLI version number of "1". If the service were to make a breaking change, it could use this header to determine which response handler to use. (e.g. return a different response for "" or "1" and another for "2".) Obviously we want to avoid being in this situation, but in the event that we need to make a breaking change, we'll need this value.
We send the Pulumi version as well, though the SDK will probably rev much more quickly than the backend API client version.
Fixes#848
We were pretty careful to keep the test directory around if the test ever
exited early due to a panic or error return. But if the test ran to
completion and failed -- for example, if ExtraRuntimeValidation caused the
test to fail -- we would end up deleting the test directory.
Fixes#868
Previously, when uploading a projectm to the service, we would only
upload the folder rooted by the Pulumi.yaml for that project. This
worked well, but it meant that customers needed to structure their
code in a way such that Pulumi.yaml was always as the root of their
project, and if they wanted to share common files between two projects
there was no good solution for doing this.
This change introduces an optional piece of metadata, named context,
that can be added to Pulumi.yaml, which allows controlling the root
folder used for computing the root folder to archive from. When it is
set, it is combined with the location of the Pulumi.yaml file for the
project we are uploading and that folder is uses as the root of what
we upload to the service.
Fixes: #574
The existing logic would flow colorization information into the
engine, so depending on the settings in the CLI, the engine may or may
not have emitted colorized events. This coupling is not great and we
want to start moving to a world where the presentation happens
exclusively at the CLI level.
With this change, the engine will always produce strings that have the
colorization formatting directives (i.e. the directives that
reconquest/loreley understands) and the CLI will apply
colorization (which could mean either running loreley to turn the
directives into ANSI escape codes, or drop them or retain them, for
debuging purposes).
Fixes#742
If we do not do this, the engine will assume that it should elide
unknown input/output properties when interacting with the language
and resource providers and we will produce unexpected results.
This PR adds a new `pulumi history` command, which prints the update history for a stack.
The local backend stores the update history in a JSON file on disk, next to the checkpoint file. The cloud backend simply provides the update metadata, and expects to receive all the data from a (NYI) `/history` REST endpoint.
`pkg/backend/updates.go` defines the data that is being persisted. The way the data is wired through the system is adding a new `backend.UpdateMetadata` parameter to a Stack/Backend's `Update` and `Destroy` methods.
I use `tests/integration/stack_outputs/` as the simple app for the related tests, hence the addition to the `.gitignore` and fixing the name in the `Pulumi.yaml`.
Fixes#636.
Previously, the `pulumi` tool did not show any indication of progress
when doing a deployment. Combined with the fact that we do not create
resources in parallel it meant that sometime `pulumi` would appear to
hang, when really it was just waiting on some resource to be created
in AWS. In addition, some AWS resources take a long time to create and
CI systems like travis will kill the job if there is no output. This
causes us (and our customers) to have to do crazy dances where we
launch shell scripts that write a dot to the console every once in a
while so we don't get killed. While we plan to overhaul the output
logic (see #617), we take a first step towards interactivity by simply
having a nice little spinner (in the interactive case) and when run
non interactive have `pulumi` print a message that it is still
working.
Fixes#794
This PR exports the aggregate resource changes for update and destroy operations. We'll use this information in #636 when summarizing previous updates.
I initially started with a new struct that had fields like `Created`, `Deleted`, `Unchanged`, etc. But it became cumbersome with the seven different type of resource operations we perform. So instead went with the more flexible `map[deploy.StepOp]int`.
Surprisingly `pulumi login -c https://google.com` would succeed. This was because we were too lax in our way of validating credentials. We take the provided cloud URL and call the "GetCurrentUserHandler" method. But we were only checking that it returned a successful response, not that it was actually valid JSON.
So in the "https://google.com" case, Google returned HTML describing a 404 error, but since the sever response was 200, the Pulumi CLI assumed things were on the up and up.
We now parse the response as JSON, and confirm the response has a `name` property that is non-nil. This heuristic covers the majority of false-positive cases, but without us needing to move all of the service's API shape for users, which includes organizations, which includes Clouds, etc. into `pulumi`.
Fixes https://github.com/pulumi/pulumi-service/issues/457. As an added bonus, we now return a much more useful error message.
