After #2088, we began calling `Diff` on providers that are not configured
due to unknown configuration values. This hit an assertion intended to
detect exactly this scenario, which was previously unexpected.
These changes adjust `Diff` to indicate that a Diff is unavailable and
return an error message that describes why. The step generator then
interprets the diff as indicating a normal update and issues the error
message to the diagnostic stream.
Fixes#2223.
Configuration keys are simple namespace/name pairs, delimited by
":". For compatability, we also allow
"<namespace>:config:<name>", but we always record the "nice" name in
`Pulumi.<stack-name>.yaml`.
While `pulumi config` and friends would block setting a key like
`a🅱️c` (where the "name" has a colon in it), it would allow
`a:config:b:c`. However, this would be recorded as `a🅱️c` in
`Pulumi.<stack-name>.yaml`, which meant we'd error when parsing the
configuration file later.
To work around this, disallow ":" in the "name" part of a
configuration key. With this change the following all work:
```
keyName
my-project:keyName
my-project:config:keyName
```
However, both
`my-project:keyName:subKey`
`my-project:config:keyName:subKey`
are now disallowed.
I considered allowing colons in subkeys, but I think it adds more
confusion (due to the interaction with how we allow you elide the
project name in the default case) than is worthwhile at this point.
Fixes#2171
When launching plugins today, `pulumi` looks in two places:
1. It looks to see if the plugin in on the $PATH and if so, uses
it. This makes it easy to force a specific version of a resource
provider to be used and is what happens at development time (since
resource providers make their way onto $PATH via GOBIN).
2. If the above fails, it looks in the "plugin cache" in
`~/.pulumi/plugins`. This is the location that `pulumi plugin
install` places plugins.
Unlike resource provider plugins, we don't yet deliver language
plugins via `pulumi plugin install` so the language provider plugins
must be on the `$PATH` to be found. This is okay, because when we ship
the SDK, we include the executables next to `pulumi` itself.
However, if a user chooses to not put `pulumi` on their $PATH, or they
do but it is a symlink to the real `pulumi` binary installed
somewhere, we'd fail to find the language plugins, since they would
not be on the `$PATH`
To address this, when probing for language plugins, also consider
binaries next to the currently running `pulumi` process.
Fixes#1956
While the lifecycle tests wrote a `.yarnrc` file to ensure that copies
of `yarn` did not race with one another, the more barebones testing
framework did not.
This should address some of the yarn issues we've been seeing in CI
recently
Under our old versioning system, when we started a new point release,
we'd tag the HEAD commit of master with a tag like `v0.16.6-dev` and
our scripts would use this to generate a new version number. This
required a great deal of gymnastics when producing a release and
caused us to litter these -dev tags everywhere.
To improve this, we change version number generation to the following
strategy:
1. If the commit we are building has a tag applied to it, use that tag
as the version (appending the dirty bit metadata to the version, if
needed).
2. If the commit we are bulding does not have a tag applied to it,
take the version from the next reachable tag, increment the patch
version and then append the `-dev` pre-release tag. As part of this,
we also make a slight tweek to our semver generation such that instead
of `-dev<TIMESTAMP>` we use `-dev.<TIMESTAMP>` which is more in line
with what semver recommends.
The issue is related to this code:
https://github.com/pulumi/pulumi/blob/v0.16.4/pkg/workspace/plugins.go#L155-L195
Note that we use `defer` to ensure we close our handle to the file we
are unpacking when we encounter a file in the tarball. However, the
defers don't run until the containing function ends, so when we go to
do the rename, or process still has a bunch of open file handles, which
prevents the directory from being renamed because it is "in use".
By doing all of the work in an anonymous function, we ensure that the
defer statements run before we go to rename the directory
Fixes#2217
This is code that should have been part of #2211 but was accidently
dropped during a rebase when responding to CR feedback.
When two installs for the same plugin are racing, the second one will
see the destination directory already exists and fail. We can safely
ignore this error.
These changes add a new resource to the Pulumi SDK,
`pulumi.StackReference`, that represents a reference to another stack.
This resource has an output property, `outputs`, that contains the
complete set of outputs for the referenced stack. The Pulumi account
performing the deployment that creates a `StackReference` must have
access to the referenced stack or the call will fail.
This resource is implemented by a builtin provider managed by the engine.
This provider will be used for any custom resources and invokes inside
the `pulumi:pulumi` module. Currently this provider supports only the
`pulumi:pulumi:StackReference` resource.
Fixes#109.
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
The lockfile is not super interesting when running the lifecycle
tests, since the test project is not a long lived artifact. There was
no real harm in writing it, exepct that in some cases `pipenv` gets
very confused when you have dependencies on pre-release
versions (which is the case right now as we are brining up Python 3).
The packages are still installed correctly, even when we don't write a
lock file.
* Fix Python support in integration test framework
Update the integratino test framework to use pipenv to bootstrap new
virtual environments for tests and use those virtual environments to run
pulumi update and pulumi preview.
Fixespulumi/pulumi#2138
* Install packages via 'Dependencies' field
* Remove code for installing packages from Dependencies
In preparation for some workspace restructuring, I decided to scratch a
few itches of my own in the code:
* Change project's RuntimeInfo field to just Runtime, to match the
serialized name in JSON/YAML.
* Eliminate the no-longer-used Context and NoDefaultIgnores fields on
project, and all of the associated legacy PPC-related code.
* Eliminate the no-longer-used IgnoreFile constant.
* Remove a bunch of "// nolint: lll" annotations, and simply format
the structures with comments on dedicated lines, to avoid overly
lengthy lines and lint suppressions.
* Mark Dependencies and InitErrors as `omitempty` in the JSON
serialization directives for CheckpointV2 files. This was done for
the YAML directives, but (presumably accidentally) omitted for JSON.
Go 1.10 made some breaking changes to the headers in archive/tar [1] and
archive/zip [2], breaking the expected values in tests. In order to keep
tests passing with both, wherever a hardcoded hash is expect we switch
on `runtime.Version()` to select whether we want the Go 1.9 (currently
supported Go version) or later version of the hash.
Eventually these switches should be removed in favour of using the later
version only, so they are liberally commented to explain the reasoning.
[1]: https://golang.org/doc/go1.10#archive/tar
[2]: https://golang.org/doc/go1.10#archive/zip
Downlevel versions of the Pulumi Node SDK assumed that a parallelism
level of zero implied serial execution, which current CLIs use to signal
unbounded parallelism. This commit works around the downlevel issue by
using math.MaxInt32 to signal unbounded parallelism.
In the past, we had a mode where the CLI would upload the Pulumi
program, as well as its contents and do the execution remotely.
We've since stopped supporting that, but all the supporting code has
been left in the CLI.
This change removes the code we had to support the above case,
including the `pulumi archive` command, which was a debugging tool to
generate the archive we would have uploaded (which was helpful in the
past to understand why behavior differed between local execution and
remote execution.)
If you took the time to type out `--skip-preview`, we should have high
confidence that you don't want a preview and you're okay with the
operation just happening without a prompt.
Fixes#1448
* Remove TODO for issue since fixed in PPCs.
* Update issue reference to source
* Update comment wording
* Remove --ppc arg of stack init
* Remove PPC references in int. testing fx
* Remove vestigial PPC API types
* Introduce new metadata keys `vcs.repo`, `vcs.kind` and `vcs.owner` to keep the keys generic for any vcs. Expanded the git SSH regex to account for bitbucket's .org domain.
* Introduce new stack tags keys with the same theme of detecting the vcs.
* Enable gzip compression on the wire
This change allows the Pulumi API client to gzip requests sent to the
Pulumi service if requested using the 'GzipCompress' http option.
This change also sets the Accept-Encoding: gzip header for all requests
originating from the CLI, indicating to the service that it is free to
gzip responses. The 'readBody' function is used in the API client to
read a response's body, regardless of how it is encoded.
Finally, this change sets GzipCompress: true on the
'PatchUpdateCheckpoint' API call, since JSON payloads in that call tend
to be large and it has become a performance bottleneck.
* spelling
* CR feedback:
1. Clarify and edit comments
2. Close the gzip.Reader when reading bodies
3. Log the payload size when logging compression ratios
Providers with unknown properties are currently considered to require
replacement. This was intended to indicate that we could not be sure
whether or not replacement was reqiuired. Unfortunately, this was not a
good user experience, as replacement would never be required at runtime.
This caused quite a bit of confusion--never proposing replacement seems
to be the better option.
The provider registry was checking for a `nil` provider instance before
checking for a non-nil error. This caused the CLI to fail to report
important errors during the plugin load process (e.g. invalid checkpoint
errors) and instead report a failure to find a matching plugin.
Some providers (namely Kubernetes) require unbounded parallelism in
order to function correctly. This commit enables the engine to operate
in a mode with unbounded parallelism and switches to that mode by
default.
* Add new 'pulumi state' command for editing state
This commit adds 'pulumi state unprotect' and 'pulumi state delete', two
commands that can be used to unprotect and delete resources from a
stack's state, respectively.
* Simplify LocateResource
* CR: Print yellow 'warning' before editing state
* Lots of CR feedback
* CR: Only delete protected resources when asked with --force
Last-minute events coming through the engine could cause the goroutine
iterating over engineEvents to write to displayEvents after it has
already been closed by the main goroutine.
Whenever an update fails partially, it gives the engine back some state
bag of outputs that should be persisted to the snapshot. When saving
this state, we shouldn't save the inputs that triggered the update that
failed, since the resource that exists will never have been updated
successfully with those inputs.
Instead of saving the new inputs on partial failed updates, this commit
saves the old inputs and the new outputs. The new outputs will likely
need to be refreshed to be useful, but the old inputs will be correct
from the perpsective of the Pulumi program that generated the last
successful update.
Fixespulumi/pulumi#2011
This adds an option, --suppress-outputs, to many commands, to
avoid printing stack outputs for stacks that might contain sensitive
information in their outputs (like key material, and whatnot).
Fixespulumi/pulumi#2028.
This change adds GitLab CI support, by sniffing out the right
variables (equivalent to what we already do for Travis).
I've also restructured the code to share more logic with our
existing CI detection code, now moved to the pkg/util/ciutil
package, and will be fleshing this out more in the days to come.
There is a seldom-used capability in our CLI, the ability to pass
-m to specify an update message, which we will then show prominently.
At the same time, we already scrape some interesting information from
the Git repo from which an update is performed, like the SHA hash,
committer, and author information. We explicitly didn't want to scrape
the entire message just in case someone put sensitive info inside of it.
It seems safe -- indeed, appealing -- to use just the title portion
as the default update message when no other has been provided (the
majority case). We'll work on displaying it in a better way, but this
strengthens our GitOps/CI/CD story.
Fixespulumi/pulumi#2008.
It's not good practice to dirty up the stdout stream with messages
like this. In fact, I questioned whether we should be emitting
*anything* here, but given that this is often used in unattended
environments, coupled with the fact that it's easy to accidentally set
this and then wonder why `pulumi login` is silently returning, led
me to keep it in (for now, at least).
The preview will proceed as if the operations had not been issued (i.e.
we will not speculate on a new state for the stack). This is consistent
with our behavior prior to the changes that added pending operations to
the checkpoint.
This change stops attempting to pop a web browser in non-interactive
sessions. Instead, the PULUMI_ACCESS_TOKEN environment variable must
be set. Otherwise, any attempt to use the CLI will yield
$ pulumi --non-interactive preview
error: PULUMI_ACCESS_TOKEN must be set for login during non-interactive CLI sessions
This is the behavior we want for Docker-based invocations of the CLI,
and so is part of pulumi/pulumi#1991.
The diff display code was not expecting that it would be possible for
resource properties to transition from being an archive to being an
asset, or the other way around. This commit prints out a reasonable diff
if this situation occurs instead of crashing.
* Process deletions conservatively in parallel
This commit allows the engine to conservatively delete resources in
parallel when it is sure that it is legal to do so. In the absence of a
true data-flow oriented step scheduler, this approach provides a
significant improvement over the existing serial deletion mechanism.
Instead of processing deletes serially, this commit will partition the
set of condemned resources into sets of resources that are known to be
legally deletable in parallel. The step executor will then execute those
independent lists of steps one-by-one until all steps are complete.
* CR: Make ResourceSet a normal map
* Only use the dependency graph if we can trust it
* Reverse polarity of pendingDeletesAreReplaces
* CR: un-export a few types
* CR: simplify control flow in step generator when scheduling
* CR: parents are dependencies, fix loop index
* CR: Remove ParentOf, add new test for parent dependencies
API calls agains the Pulumi service may start setting a new header,
`X-Pulumi-Warning`. The value of this header should be presented to
the user as a warning.
The Service will use this to provide additional information to the
user without having the CLI have to know about every specific warning
path.
Since I was digging around over the weekend after the change to move
away from light black, and the impact it had on less important
information showing more prominently than it used to, I took a step
back and did a deeper tidying up of things. Another side goal of this
exercise was to be a little more respectful of terminal width; when
we could say things with fewer words, I did so.
* Stylize the preview/update summary differently, so that it stands
out as a section. Also highlight the total changes with bold -- it
turns out this has a similar effect to the bright white colorization,
just without the negative effects on e.g. white terminals.
* Eliminate some verbosity in the phrasing of change summaries.
* Make all heading sections stylized consistently. This includes
the color (bright magenta) and the vertical spacing (always a newline
separating headings). We were previously inconsistent on this (e.g.,
outputs were under "---outputs---"). Now the headings are:
Previewing (etc), Diagnostics, Outputs, Resources, Duration, and Permalink.
* Fix an issue where we'd parent things to "global" until the stack
object later showed up. Now we'll simply mock up a stack resource.
* Don't show messages like "no change" or "unchanged". Prior to the
light black removal, these faded into the background of the terminal.
Now they just clutter up the display. Similar to the elision of "*"
for OpSames in a prior commit, just leave these out. Now anything
that's written is actually a meaningful status for the user to note.
* Don't show the "3 info messages," etc. summaries in the Info column
while an update is ongoing. Instead, just show the latest line. This
is more respectful of width -- I often find that the important
messages scroll off the right of my screen before this change.
For discussion:
- I actually wonder if we should eliminate the summary
altogether and always just show the latest line. Or even
blank it out. The summary feels better suited for the
Diagnostics section, and the Status concisely tells us
how a resource's update ended up (failed, succeeded, etc).
- Similarly, I question the idea of showing only the "worst"
message. I'd vote for always showing the latest, and again
leaving it to the Status column for concisely telling the
user about the final state a resource ended up in.
* Stop prepending "info: " to every stdout/stderr message. It adds
no value, clutters up the display, and worsens horizontal usage.
* Lessen the verbosity of update headline messages, so we now instead
of e.g. "Previewing update of stack 'x':", we just say
"Previewing update (x):".
* Eliminate vertical whitespace in the Diagnostics section. Every
independent console.out previously was separated by an entire newline,
which made the section look cluttered to my eyes. These are just
streams of logs, there's no reason for the extra newlines.
* Colorize the resource headers in the Diagnostic section light blue.
Note that this will change various test baselines, which I will
update next. I didn't want those in the same commit.
Now that we're showing SpecUnimportant as regular text, the extra
"Previewing"/"Updating" line that we show really stands out as being
superfluous. For example, we previously said:
Updating stack 'docker-images'
Performing changes:
...
This change eliminates that second line, so we just have:
Updating stack 'docker-images':
...
Recently, we eliminated bright black text, which IMHO makes the
"same" lines really stand out more than we want them to. This is
partly just due to the heavyweight nature of the "*" character,
which we precede every line with. This has the effect of making it
toughter to scan the update to see what's going to happen. The goal
of SpecUnimportant (bright black) was that we wanted to draw less
attention to certain elements of the CLI text -- and have them fade
into the background (apparently it was too successful at this ;-))
So, this change eliminates the "*" prefix for same operations
altogether. It reads better to my eyes and keeps the original intent.
- Attempting to read an archive with an unknown type now returns an
error
- Attempting to read a path archive that is neither a known archive
format or a directory now returns an error
Fixes#1529.
Fixes#1953.
Add two command-line options to the test framework, "-dirs" and
"list-dirs". The former accepts a regex that is used to select which
integration tests to run. The latter lists available integration tests
without running them.
* Protobuf changes
* Move management of root resource state to engine
This commit fixes a persistent side-by-side issue in the NodeJS SDK by
moving the management of root resource state to the engine. Doing so
adds two new endpoints to the Engine gRPC service: 1) GetRootResource
and 2) SetRootResource, which get and set the root resource
respectively.
* Rebase against master, regenerate proto
* Have backend.ListStacks return a new StackSummary interface
* Update filestake backend to use new type
* Update httpstate backend to use new type
* Update commands to use new type
* lint
* Address PR feedback
* Lint
In order to present a message with a link to a user's access tokens in
the service, we have to convert the API server's URL to the URL for
the console. We understand how to do this for servers that look like
our servers (i.e. their host name is `api.<whatever>`), but in general
we can't do this.
Rewrite the code such that we only print a message about how to find
your access tokens (and offer browser based login) only in cases where
we are able to construct a URL to the console.
Fixes#1930
* Revert "Don't show stack outputs when update fails (#1916)"
This reverts commit e3f89e82aa.
* Be more precise about printing outputs
This commit prints outputs only if they are known to be complete. This
avoids massive red diffs during previews and when component resources
fail to call registerResourceOutputs.
* CR: Clean up large boolean expression and comment
* CR: boolean compromise
* Retire pending deletions at start of plan
Instead of letting pending deletions pile up to be retired at the end of
a plan, this commit eagerly disposes of any pending deletions that were
pending at the end of the previous plan. This is a nice usability win
and also reclaims an invariant that at most one resource with a given
URN is live and at most one is pending deletion at any point in time.
* Rebase against master
* Fix a test issue arising from shared snapshots
* CR feedback
* plan -> replacement
* Use ephemeral statuses to communicate deletions
* Close cancellation source before closing events
The cancellation source logs cancellation messages to the engine event
channel, so we must first close the cancellation source before closing
the channel.
* CR: Fix race in shutdown of signal goroutine
* Don't show stack outputs when update fails
It is basically guaranteed that stack outputs are going to be
meaningless if a plan fails, so this commmit doesn't display them if an
error has occured.
* CR: expand on comment
We signal provider cancellation by hangning a goroutine off of the plan
executor's parent context. To ensure clean shutdown, this goroutine also
listens on a channel that closes once the plan has finished executing.
Unfortunately, we were closing this channel too early, and the close was
racing with the cancellation signal. These changes ensure that the
channel closes after the plan has fully completed.
Fixes#1906.
Fixespulumi/pulumi-kubernetes#185.
* Validate type tokens before using them
When registering or reading a resource, we take the type token given to
us from the language host and assume that it's valid, which resulted in
assertion failures in various places in the engine. This commit
validates the format of type tokens given to us from the language host
and issues an appropriate error if it's not valid.
Along the way, this commit also improves the way that fatal exceptions
are rendered in the Node language host.
* Pre-allocate an exception for ReadResource
* Fix integration test
* CR Feedback
This commit is a lower-impact change that fixes the bugs associated with
invalid types on component resources and only checks that a type is
valid on custom resources.
* CR Take 2: Fix up IsProviderType instead of fixing call sites
* Please gometalinter
It is possible for the inputs of a "same" resource to have changed even
if the contents of the input bags are different if the resource's
provider deems the physical change to be semantically irrelevant.
Previously `new` was operating under the assumption that it was always
going to be creating a new project/stack, and would always prompt for
these values. However, we want to be able to use `new` to pull down the
source for an existing stack. This change adds a `--stack` flag to `new`
that can be used to specify an existing stack. If the specified stack
already exists, we won't prompt for the project name/description, and
instead just use the existing stack's values. If `--stack` is specified,
but doesn't already exist, it will just use that as the stack name
(instead of prompting) when creating the stack. `new` also now handles
configuration like `up <url>`: if the stack is a preconfigured empty
stack (i.e. it was created/configured in the Pulumi Console via Pulumi
Button or New Project), we will use the existing stack's config without
prompting. Otherwise we will prompt for config, and just like `up
<url>`, we'll use the existing stack's config values as defaults when
prompting, or if the stack didn't exist, use the defaults from the
template.
Previously `up <url>`'s handling of the project name/description wasn't
correct: it would always automatically use the values from the template
without prompting. Now, just like `new`:
- When updating an existing stack, it will not prompt, and just use the
existing stack's values.
- When creating a new stack, it will prompt for the project
name/description, using the defaults from the template.
This PR consolidates some of the `new`/`up` implementation so it shares
code for this functionality. There's definitely opportunities for a lot
more code reuse, but that cleanup can happen down the line if/when we
have the cycles.
* Don't close eventChannel when panicking
The state of the system is completely unknown when panicking and in
general it's not safe to infer whether or not it is safe to close a
channel when in this tate.
* CR feedback
* Spelling
* Introduce Result type to engine
The Result type can be used to signal the failure of a computation due
to both internal and non-internal reasons. If a computation failed due
to an internal error, the Result type carries that error with it and
provides it when the 'Error' method on a Result is called. If a
computation failed gracefully, but wished to bail instead of continue a
doomed plan, the 'Error' method provides a value of null.
* CR feedback
We generally want examples and apps to be authored such that they are
clonable/deployable as-is without using new/up (and want to
encourage this). That means no longer using the ${PROJECT} and
${DESCRIPTION} replacement strings in Pulumi.yaml and other text files.
Instead, good default project names and descriptions should be specified
in Pulumi.yaml and elsewhere.
We'll use the specified values as defaults when prompting the user, and
then directly serialize/save the values to Pulumi.yaml when configuring
the user's project. This does mean that name in package.json (for nodejs
projects) won't be updated if it isn't using ${PROJECT}, but that's OK.
Our templates in the pulumi/templates repo will still use
${PROJECT}/${DESCRIPTION} for now, to continue to work well with v0.15
of the CLI. After that version is no longer in use, we can update the
templates to no longer use the replacement strings and delete the code
in the CLI that deals with it.
This change implements the same preview behavior we have for
cloud stacks, in pkg/backend/httpbe, for local stacks, in
pkg/backend/filebe. This mostly required just refactoring bits
and pieces so that we can share more of the code, although it
does still entail quite a bit of redundancy. In particular, the
apply functions for both backends are now so close to being
unified, but still require enough custom logic that it warrants
keeping them separate (for now...)
This simply refactors all the display logic out of the
pkg/backend/filestate package. This helps to gear us up to better unify
this logic between the filestate and httpstate backends.
Furthermore, this really ought to be in its own non-backend,
CLI-specific package, but I'm taking one step at a time here.
This change alters the login prompt slightly, so that it is more
obvious that alternative methods exist.
Before this change, we would say:
$ pulumi login
We need your Pulumi account to identify you.
Enter your access token from https://app.pulumi.com/account
or hit <ENTER> to log in using your browser :
After this change, we say this instead:
$ pulumi login
Manage your Pulumi stacks by logging in.
Run `pulumi login --help` for alternative login options.
Enter your access token from https://app.pulumi.com/account
or hit <ENTER> to log in using your browser :
Also updated the help text to advertise this a bit more prominently.
This renames the backend packages to more closely align with the
new direction for them. Namely, pkg/backend/cloud becomes
pkg/backend/httpstate and pkg/backend/local becomes
pkg/backend/filestate. This also helps to clarify that these are meant
to be around state management and so the upcoming refactoring required
to split out (e.g.) the display logic (amongst other things) will make
more sense, and we'll need better package names for those too.
As part of making the local backend more prominent, this changes a few
aspects of how you use it:
* Simplify how you log into a specific cloud; rather than
`pulumi login --cloud-url <url>`, just say `pulumi login <url>`.
* Use a proper URL scheme to denote local backend usage. We have chosen
file://, since the REST API backend is of course always https://.
This means that you can say `pulumi login file://~` to use the local
backend, with state files stored in your home directory. Similarly,
we support `pulumi login file://.` for the current directory.
* Add a --local flag to the login command, to make local logins a
bit easier in the common case of using your home directory. Just say
`pulumi login --local` and it is sugar for `pulumi login file://~`.
* Print the URL for the backend after logging in; for the cloud,
this is just the user's stacks page, and for the local backend,
this is the path to the user's stacks directory on disk.
* Tidy up the documentation for login a bit to be clearer about this.
This is part of pulumi/pulumi#1818.
The existing message put the URL to visit and some explanation text on
the same line, which makes it a little harder to copy only the URL
into a browser. If this extra text ends up being copied as well as the
URL it can lead to failures later, when we try to decode the query
string as part of the OAuth flow.
It's easy enough to fix by just putting the URL on its own line, split
off from the text itself.
Fixes#1832
This changes two things:
1) Per PR feedback, change the text to "N changes found during refresh."
2) Colorize the parenthetical "No resources will be modified" message;
it looked a little odd being plain colored.
This commit reverts most of #1853 and replaces it with functionally
identical logic, using the notion of status message-specific sinks.
In other words, where the original commit implemented ephemeral status
messages by adding an `isStatus` parameter to most of the logging
methdos in pulumi/pulumi, this implements ephemeral status messages as a
parallel logging sink, which emits _only_ ephemeral status messages.
The original commit message in that PR was:
> Allow log events to be marked "status" events
>
> This commit will introduce a field, IsStatus to LogRequest. A "status"
> logging event will be displayed in the Info column of the main
> display, but will not be printed out at the end, when resource
> operations complete.
>
> For example, for complex resource initialization, we'd like to display
> a series of intermediate results: [1/4] Service object created, for
> example. We'd like these to appear in the Info column, but not at the
> end, where they are not helpful to the user.
Previously, we only supported config keys that included a ':' delimiter
in config keys specified in the template manifest and in `-c` flags to
`new` and `up`. This prevented the use of project keys in the template
manifest and made it more difficult to pass such keys with `-c`,
effectively preventing the use of `new pulumi.Config()` in project code.
This change fixes this by allowing config keys that don't have a
delimiter in the template manifest and `-c` flags. In such cases, the
project name is automatically prepended behind the scenes, the same as
what `pulumi config set` does.
The pattern we are using here is generally prone to error, so I hope we find a way to move away from this more generally - but for now we need to be able to configure more of these in places we are using `With`.
Also remove some `+""` that are tripping up the linter for me locally.
- Create all refresh steps before issuing any. This is important as the
state update loop expects all steps to exist.
- Check for cancellation later in the refresher.
This also fixes races in the SnapshotManager and the test journal that
could cause panics during cancellation.
The wording for refresh doesn't accurately convey that the operations
aren't actually mutating your resources, but instead are simply changing
your checkpoint state. This change (hopefully) helps in two ways:
First, put text just before the prompt:
Do you want to perform this refresh?
No resources will be modified as part of this refresh; just your stack's state will be.
Second, alter the summary ever-so-slightly, from:
info: 2 changes performed:
~ 2 resources updated
3 resources unchanged
to:
info: 2 changes refreshed:
~ 2 resources updated
3 resources unchanged
This reads just slightly better, and removes any sense of panic I might
have otherwise had that my refresh just did something wrong.
As I was in here, since I had to pass UpdateKind information to new
places, I cleaned up the situation where we had three mostly-similar
enums (but which actually diverged) and several areas where we were
using untyped strings for this same information. Now there's just one.
This fixespulumi/pulumi#1551.
This commit will greatly improve the experience of dealing with partial
failures by simply re-trying to initialize the relevant resources on
every subsequent `pulumi up`, instead of printing a list of reasons the
resource had previously failed to initialize.
As motivation, consider our behavior in the following common, painful
scenario:
* The user creates a `Service` and a `Deployment`.
* The `Pod`s in the `Deployment` fail to become live. This causes the
`Service` to fail, since it does not target any live `Pod`s.
* The user fixes the `Deployment`. A run of `pulumi up` sees the
`Pod`s successfully initialize.
* Users will expect that the `Service` is now in a state of success,
as the `Pod`s it targets are alive. But, because we don't update the
`Service` by default, it perpetually exists in a state of error.
* The user is now required to change some trivial feature of the
`Service` just to trigger an update, so that we can see it succeed.
There are many situations like this. Another very common one is waiting
for test `Pod`s that are meant to successfully complete when some object
becomes live.
By triggering an empty update step for all resources that have any
initialization errors, we avoid all problems like this.
This commit will implement this empty-update semantics for partial
failures, as well as fix the display UX to correctly render the diff in
these cases.
Often when the test fails, you'll want to use the web console to
investigate the stack, looking at detailed log messages, for
example. But since we always delete the stack today, you often can't.
Stop running `pulumi stack rm` when the test fails, however do run a
`pulumi destroy` so we at least clean up any resources the stack may
have allocated during testing.
Fixes#1722
Replace the Source-based implementation of refresh with a phase that
runs as the first part of plan execution and rewrites the snapshot in-memory.
In order to fit neatly within the existing framework for resource operations,
these changes introduce a new kind of step, RefreshStep, to represent
refreshes. RefreshSteps operate similar to ReadSteps but do not imply that
the resource being read is not managed by Pulumi.
In addition to the refresh reimplementation, these changes incorporate those
from #1394 to run refresh in the integration test framework.
Fixes#1598.
Fixespulumi/pulumi-terraform#165.
Contributes to #1449.
The fact that we include the `:config:` portion of a configuration
name when sending to the service is an artifact of history. It was
needed back when we needed to support deploying using a new CLI
against a PPC that had older packages embeded in it.
We don't have that sort of problem anymore, so we can stop sending
this data.
* Show a better error message when decrypting fails
It is most often the case that failing to decrypt a secret implies that
the secret was transferred from one stack to another via copying the
configuration. This commit introduces a better error message for this
case and instructs users to explicitly re-encrypt their encrypted keys
in the context of the new stack.
* Spelling
* CR: Grammar fixes
These changes simplify a couple aspects of plan execution in the hopes of
clarifying some responsibilities and preparing the code for changes to the
implementation of refresh.
1. All aspects of plan execution are now managed by the plan executor,
which is no longer exported. Instead, it is abstracted behind
`Plan.Execute`.
2. The plan executor's error-handling and reporting have been unified
and simplified somewhat.
* Log errors coming from the language host
Similar to pulumi/pulumi#1762, fixespulumi/pulumi#1775. The language
host can fail without issuing any diagnostics and it is very unclear
what happens if the engine does not log the error.
* CR feedback
A checkpoint write is unnecessary if it does not change the semantics of
the data currently stored in the checkpoint. We currently perform
unnecessary checkpoint writes in two cases:
- Same steps where no aspect of the resource's state has changed
- Replace steps, which exist solely for display purposes
The former case is particularly bothersome, as it is rather common to
run updates--especially in CI--that consist largely/entirely of these
same steps.
These changes eliminate the checkpoint writes we perform in these two
cases. Some care is needed to ensure that we continue to write the
checkpoint in the case of same steps that do represent meaningful
changes (e.g. changes to a resource's output properties or
dependencies).
Fixes#1769.
The glog package force the use of golang's underyling flag package,
which Cobra does not use. To work around this, we had a complicated
dance around defining flags in multiple places, calling flag.Parse
explicitly and then stomping values in the flag package with values we
got from Cobra.
Because we ended up parsing parts of the command line twice, each with
a different set of semantics, we ended up with bad UX in some
cases. For example:
`$ pulumi -v=10 --logflow update`
Would fail with an error message that looked nothing like normal CLI
errors, where as:
`$ pulumi -v=10 update --logflow`
Would behave as you expect. To address this, we now do two things:
- We never call flag.Parse() anymore. Wacking the flags with values we
got from Cobra is sufficent for what we care about.
- We use a forked copy of glog which does not complain when
flag.Parse() is not called before logging.
Fixes#301Fixes#710Fixes#968
1. 'readID' was never assigned to and was always the default value,
leading the refresh source to believe a resource was deleted
2. The refresh source could hang when a resource is deleted.
