These changes enable tracing of Pulumi API calls.
The span with which to associate an API call is passed via a
`context.Context` parameter. This required plumbing a
`context.Context` parameter through a rather large number of APIs,
especially in the backend.
In general, all API calls are associated with a new root span that
exists for essentially the entire lifetime of an invocation of the
Pulumi CLI. There were a few places where the plumbing got a bit hairier
than I was willing to address with these changes; I've used
`context.Background()` in these instances. API calls that receive this
context will create new root spans, but will still be traced.
As it stands, we recompute the workspace for the current directory
potentially many times during some CLI operations, most notably
`stack ls`. These changes add a simple cache based on the complete path
passed to `NewFrom`, and eliminate some lagginess in `stack ls` when
there are multiple stacks.
Another option is to calculate the current workspace once in the CLI and
then fetch it as necessary.
* Initialize a new stack as part of `pulumi new`
* Prompt for values with defaults preselected
* Install dependencies
* Prompt for default config values
Our logic for converting npm style versions to PEP-440 style versions
was not correct in some cases. This change fixes this.
As part of this change we no longer produce a NPM version that would
be just X.Y.Z-dev, instead for development versions we always include
both the timestamp of the commit and the commit hash.
Instead of trying to use a bunch of sed logic to do our conversions,
we now have a small go program that uses a newly added library in
pkg/util. A side effect of this is that we can more easily write tests
to ensure the conversion works as expected.
Fixes#1243
If the file contains a zero byte within the first 8000 bytes (or the entire length of the file if shorter), treat the file as binary and do not do any textual transformations. This is the same approach git uses to determine if a file is binary.
This changes the CLI interface in a few ways:
* `pulumi preview` is back! The alternative of saying
`pulumi update --preview` just felt awkward, and it's a common
operation to want to perform. Let's just make it work.
* There are two flags consistent across all update commands,
`update`, `refresh`, and `destroy`:
- `--skip-preview` will skip the preview step. Note that this
does *not* skip the prompt to confirm that you'd like to proceed.
Indeed, it will still prompt, with a little warning text about
the fact that the preview has been skipped.
* `--yes` will auto-approve the updates.
This lands us in a simpler and more intuitive spot for common scenarios.
I found the flag --force to be a strange name for skipping a preview,
since that name is usually reserved for operations that might be harmful
and yet you're coercing a tool to do it anyway, knowing there's a chance
you're going to shoot yourself in the foot.
I also found that what I almost always want in the situation where
--force was being used is to actually just run a preview and have the
confirmation auto-accepted. Going straight to --force isn't the right
thing in a CI scenario, where you actually want to run a preview first,
just to ensure there aren't any issues, before doing the update.
In a sense, there are four options here:
1. Run a preview, ask for confirmation, then do an update (the default).
2. Run a preview, auto-accept, and then do an update (the CI scenario).
3. Just run a preview with neither a confirmation nor an update (dry run).
4. Just do an update, without performing a preview beforehand (rare).
This change enables all four workflows in our CLI.
Rather than have an explosion of flags, we have a single flag,
--preview, which can specify the mode that we're operating in. The
following are the values which correlate to the above four modes:
1. "": default (no --preview specified)
2. "auto": auto-accept preview confirmation
3. "only": only run a preview, don't confirm or update
4. "skip": skip the preview altogether
As part of this change, I redid a bit of how the preview modes
were specified. Rather than booleans, which had some illegal
combinations, this change introduces a new enum type. Furthermore,
because the engine is wholly ignorant of these flags -- and only the
backend understands them -- it was confusing to me that
engine.UpdateOptions stored this flag, especially given that all
interesting engine options _also_ accepted a dryRun boolean. As of
this change, the backend.PreviewBehavior controls the preview options.
As of this change, the engine will run all Configure calls in parallel.
This improves startup performance, since otherwise, we would block
waiting for all plugins to be configured before proceeding to run a
program. Emperically, this is about 1.5-2s for AWS programs, and
manifests as a delay between the purple "Previewing update of stack"
being printed, and the corresponding grey "Previewing update" message.
This is done simply by using a Goroutine for Configure, and making sure
to synchronize on all actual CRUD operations. I toyed with using double
checked locking to eliminate lock acquisitions -- something we may want
to consider as we add more fine-grained parallelism -- however, I've
kept it simple to avoid all the otherwise implied memory model woes.
I made the judgment call that GetPluginInfo may proceed before
Configure has settled. (Otherwise, we'd immediately call it and block
after loading the plugin, obviating the parallelism benefits.) I also
made the judgment call to do this in the engine, after flip flopping
several times about possibly making it a provider's own decision.
The `go-git` implementation of `git status` is outrageously expensive,
as it performs a hash-based comparision of the working tree against the
committed state with no caching. In some example runs, this takes
upwards of 15 seconds. Because this is on the startup path for updates,
this results in a rather poor user experience.
These changes replace the `go-git` implementation with a call to `git
status --porcelain -z`, which only writes data to stdout if the working
tree is dirty.
Note that these changes also make all git-related update metadata
best-effort.
These changes add a new flag to the CLI, `--profiling`, that enables
CPU and heap profiling as well as execution tracing of the CLI itself.
The argument to this flag serves as a prefix for the profile outputs;
the CPU and heap profiles and execution trace are written to
`[filename].[pid].{cpu,mem,trace}`, respectively.
These changes also fix an issue with `cmdutil.RunFunc` wherein any error
would prevent the command's post-run hooks from executing.
The service has no good way to preview a destroy operation for a stack
managed by a PPC. Until it does, just behave as if --force was passed
to the CLI in this case (i.e. skip the preview).
Fixes#1301
Instead of using a shell script to jump from the language host into
node, just invoke node directly. This makes our start-up path a little
simpler to understand and indirectly fixespulumi/home#156, where we
would fail on Windows if the `-exec` script was in a folder that had
spaces in it (due to a subtle interaction between how go launches cmd
files and how cmd.exe parses arguments).
When linking back to the service, use the newer, simpler, URLs for a
stack: `https://pulumi.com/<owner>/<stack-name>` instead of
`https://pulumi.com/<owner>/-/-/<stack-name>`
Rather than filtering out the `id` and `urn` properties when serializing
the inputs to an invoke, pass these properties along. This enables the
use of invoke endpoints that accepts these as inputs (e.g. the endpoint
that backs `aws.ec2.getSubnet`).
Refactoring to support "preview" and "update" as part of the same
operation interacted poorly with deploying into a PPC via the service.
Per Pat, this is the quickest fix that gets us off the floor.
Part of #1301
This change eliminates our dependencies on the SDK repo. Now that
SDKs are comprised solely of pulumi/pulumi artifacts, a separate repo
isn't required. This allows us to simplify some of the distribution.
The install.sh script is modified slightly, to permit overriding the
default install location using $PULUMI_INSTALL_PATH.
This changes three minor UX things in the CLI's update flow:
1. Use bright blue for column headers, since the dark blue is nearly
invisible on a black background.
2. Move the operation symbol to the far left, as a sort of "bullet
point" for each line of output. This was how they initial symbols
were designed and this helps to glance at the summary to see what's
going on.
3. Shorten some of the column headers that didn't add extra clarity.
We already have a great history viewing experience on
Pulumi.com. `pulumi history` in the CLI today is basically unuseable,
and instead of working on trying to improve it, let's just remove it
for now.
Fixes#965
This change adds two new (hidden) CLI commands:
* `gen-bash-completion`: This command generates a bash completion
script for the CLI, storing it in the file specified by the 1st arg.
This fixespulumi/pulumi#1172.
* `gen-markdown`: This command generates a directory of Markdown files,
one per command, documenting the CLI commands and their usage.
I originally did these as separate scripts that we can use in our
build processes, but it was actually even easier to make `pulumi` able
to generate them for itself. The nice part about this is that we don't
even need to bundle additional assets in order to distribute e.g. the
bash completion scripts, we can simply tell people to run
$ pulumi gen-bash-completion /etc/bash_completion.d/pulumi
This can also be used in our upcoming Brew installer.
When running `pulumi logs -f` we'd also see these messages printed to
standard out:
Getting more logs for /aws/lambda/urlshortenereeb67ce9-d8fa6fa...
This could be useful diagnostics information, but we should be
glog'ing it not unconditionally writing it to the terminal.
* Improve the error message when npm/yarn install hasn't been run
* Same thing, but for Python
* Use PULUMI_RUN in batch script
* Use -e, -f doesn't work for symlinked paths (e.g. yarn link)
Pat ran into a weird error when trying to do some development agains
the testing cloud:
```
$ pulumi logout
$ pulumi login --cloud-url [test-cloud-url]
Logged into [test-cloud-url]
$ pulumi stack ls
Enter your Pulumi access token from https://pulumi.com/account:
```
In his case, we did not have `current` set in our credentials.json
file (likely due to him calling `pulumi logout` at some point) but we
did have stored credentials for that cloud. When he logged in the CLI
noticed we could reuse the stored credentials but did not update the
the current setting to set the current cloud.
While investigating, I also noticed that `logout` did not always do
the right thing when you were logged into a different backend than
pulumi.com
