We previously logged the number of replaces and changes returned from a call to Diff, but not the actual properties that were forcing replace. Several times we've had to debug issues with unexpected replaces being proposed, and this information is very useful to have access to.
Changes the verbose logging to include the property names for both replaces and changes instead of just the count.
* Enable delete parallelism for Python
* Add CHANGELOG.md entry
* Expand changelog message - upgrade to Python 3
* Rework stack rm test
The service now allows removing a stack if it just contains the top
level `pulumi:pulumi:Stack` resource, so we need to actually create
another resource before `stack rm` fails telling you to pass
`--force`.
Fixes#2444
When `pulumi stack rm` is run against a stack with resources, the
service will respond with an error if `--force` is not
passed. Previously we would just dump the contents of this error and
it looked something like:
`error: [400] Bad Request: Stack still has resources.`
We now handle this case more gracefully, showing our usual "this stack
still has resources" error like we would for the local backend.
Fixes#2431
These changes add a new flag to the various `ResourceOptions` types that
indicates that a resource should be deleted before it is replaced, even
if the provider does not require this behavior. The usual
delete-before-replace cascade semantics apply.
Fixes#1620.
In general, a "delete" in Pulumi is destroying an actual physical
resource. In the case of a read resource, however, the delete is
merely removing the resource from the stack; the physical resource
is not affected. These changes attempt to clarify this situation by
using the term "discard" rather than "delete".
Fixes#2015.
Because of the change to include a stack's project as part of its
identity in the service, the names passed to StackReference now
require the project name as well.
Improve the error message when they do not include them.
- Add support for per-property dependencies to the Go SDK
- Add tests for first-class secret rejection in the checkpoint and RPC
layers and language SDKs
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
The service also does this filtering on requests, because we'll need
to support older clients, but it would be nice if the CLI itself also
cleaned things up.
This change starts to use a stack's project name as part of it's
identity when talking to the cloud backend, which the Pulumi Service
now supports.
When displaying or parsing stack names for the cloud backend, we now
support the following schemes:
`<stack-name>`
`<owner-name>/<stack-name>`
`<owner-name>/<project-name>/<stack-name>`
When the owner is not specificed, we assume the currently logged in
user (as we did before). When the project name is not specificed, we
use the current project (and fail if we can't find a `Pulumi.yaml`)
Fixes#2039
Resources gain two new fields: `PropertyDependencies` and
`PendingReplacement`. The former maps an input property's name to the
dependencies that may affect the value of that property. The latter is
used to track resources that have been deleted as part of a
delete-before-replace operation but have not yet been recreated.
In addition to the new fields, resource properties may now contain
encrypted first-class secret values. These values are of type `SecretV1`,
where the `Sig` field is set to `resource.SecretSig`.
Finally, the deployment type gains a new field, `SecretsProviders`,
which contains any configuration necessary to handle secrets that may be
present in resource properties.
* Use both a in-proc and out-of-proc pipenv lock
Turns out that flock alone is not sufficient to guarantee exclusive
access to a resource within a single process. To remedy this, a few
FileMutex type wraps both an in-proc mutex and an out-of-proc
file-backed mutex to achieve the goal of exclusive access to a resource
in both in-proc and out-of-proc scenarios.
This commit also uses this lock globally in the integration test
framework in order to globally serialize invocations of pipenv install.
* Remove merge markers
* Use a file lock for serializing pipenv installs
A in-process mutex is not sufficient for serializing pipenv installs
because the 1) go test runner occasionally will split test executions
into multiple processes and 2) each test gets an instance of a
programTester and we'd need to share the mutex globally if we wanted to
successfully serialize access to the pipenv install command.
* Please linter
When constructing an Archive based off a directory path, we would
ignore any symlinks that we saw while walking the file system
collecting files to include in the archive.
A user reported an issue where they were unable to use the
[sharp](https://www.npmjs.com/package/sharp) library from NPM with a
lambda deployed via Pulumi. The problem was that the library includes
native components, and these native components include a bunch of
`*.so` files. As is common, there's a regular file with a name like
`foo.so.1.2.3` and then symlinks to that file with the names
`foo.so.1.2`, `foo.so.1` and `foo.so`. Consumers of these SOs will
try to load the shorter names, i.e. `foo.so` and expect the symlink to
resolve to the actual library.
When these links are not present, upstack code fails to load.
This changes modifies our logic such that if we have a symlink and it
points to a regular file, we include it in the archive. At this time,
we don't try to add it to the archive as a symlink, instead we just
add it as another copy of the regular file. We could explore trying to
include these things as symlinks in archive formats that allow
them (While zip does support this, I'm less sure doing this for zip
files will be a great idea, given the set of tricks we already play to
ensure our zip files are usable across many cloud vendors serverless
offerings, it feels like throwing symlinks into the mix may end up
causing even more downstream weirdness).
We continue to ignore symlinks which point at directories. In
practice, these seem fairly uncommon and doing so lets us not worry
about trying to deal with ensuring we don't end up chasing our tail in
cases where there are circular references.
While this change is in pulumi/pulumi, the downstream resource
providers will need to update their vendored dependencies in order to
pick this up and have it work end to end.
Fixes#2077
setuptools's "develop" action is not safe to run concurrently when
targeting the same source tree. In order to work around this, this
commit explicitly serializes package installations.
- Ensure new projects have a project name in line with what we'd like
to enforce going forward
- Do more aggresive validation during the interactive prompts during
`pulumi new`
- Fix an issue where the interactive prompt rendered weridly when
there was a validation error
Contributes to #1988Fixes#1441
We continue to do so when `--debug` has been passed, similar to how
these events are elided from the local display when you are not in a
debug context.
* Initial stack history command
* Adding use of color pkg, adding background colors to color pkg, and removing extra stack output
* gofmt-ed colors file
* Fixing format and removing JSON output
* Fixing nits, changing output for environment, and adding some tests
* fixing failing history test
This value was never used before, but it had a shorter name. In other
API Types we are using `projectName` which we all prefer. Since we are
going to start using this value going forward, let's adopt the good
name now when it won't break anyone.
This will allow the service to include information about what project
a stack is assocated with when listing all stacks a user has access
to.
This was not previously needed because the project did not play into
the stack identity, but it will shortly.
This test had been intermittently failing due to a race condition. Its
implementation of `plugin.Provider.Read` was intended to ensure that
the cancellation of a refresh operation occurred. As written, it was
only able to ensure that the cancellation was requested.
These changes ensure that cancellation has been acknowledged by the engine by
implementing providing an implementation for `plugin.Provider.Cancel`
that closes a channel on which the implementation of `Read` waits.
We're changing /account to redirect to /account/profile instead of
/account/tokens as the user profile settings are a more natural place
to land when going into account settings.
This commit changes the CLI to link
directly to /account/tokens, avoiding having to click on
"Access Tokens" to go to the tokens page to get an access token when
coming from the URL outputted by the CLI.
When returning immediately from the loop, we are closing the `done` channel early. This signals that we have finished processing every engine event, however that isn't true. Since some events may still be in-flight in `recordEngineEvent`. (This could potentially lead to a race condition in the `diag.Sink` passed to the API client used to record the call.)
This ensures that the gRPC server is properly shut down. This fixes an
issue in which a resource plugin that is still configuring could report
log messages to the plugin host, which would in turn attempt to send
diagnostic packets over a closed channel, causing a panic.
Fixes#2170.
The event rendering goroutine in the remote backend was not properly
synchronizing with the goroutine that created it, and could continue
executing after its creator finished. I believe that this is the root
cause of #1850.
When reading values like access keys or secrets from the terminal, we
would use the `terminal.ReadPassword` function to ensure characters
the user typed were not echo'd back to the console, as a convience.
When standard input was not connected to a tty (which would happen in
some cases like in docker when -t was not passed or in CI), this would
fail with an error about an bad ioctl. Update our logic such that
when standard in is not connected to a terminal, we just read input
normally.
While I was in the area, I unified the code for Windows and *NIX for
these functions.
Fixes#2017
This option allows the user to override the file used to fetch and store
configuration information for a stack. It is available for the config,
destroy, logs, preview, refresh, and up commands.
Note that this option is not persistent: if it is not specified, the
stack's default configuration will be used. If an alternate config file
is used exclusively for a stack, it must be specified to all commands
that interact with that stack.
This option can be used to share plaintext configuration across multiple
stacks. It cannot be used to share secret configuration, as secrets are
associated with a particular stack and cannot be decryptex by other
stacks.
When the install fails, we end up printing the entire contents of
yarn's stdout to stdout. This output can often be quite long and will
cause Travis to fail in some cases.
The regular error output should be sufficent for us to diagnose any
issues we'll face.
Add a new property to ProgramTestOptions, `Overrides` that allows a
test to request a different version of a package is used instead of
what would be listed in the package.json file.
This will be used by our nightly automation to run everything "at head"
Semver allows you to attach "build metadata" to a version by appending
the version with `+` and then metadata. In #2216 we started to take
advantage of this as the place to put the git commit information,
instead of including it as part of the "version". This is more in line
with what Semver expects to be done, because git commit information
isn't orderable.
Because of this, we started to publish plugins with versions like
`v0.16.5-dev.1542649729+g07d8224`. However, our logic for discovering
plugins in the cache did an initial filtering based on folder names in
the cache and the regex did not allow a + in the "version" field.
This meant that from the point of view of the cache, the plugin was
not present. This would lead to very confusing behavior where
something like `pulumi plugin install resource azure
v0.16.5-dev.1542649729+g07d8224` would download the plugin, but
`pulumi plugin ls` would not see it and attempting to do an update
with it would fail with an error saying the plugin was not installed.
This change relaxes the regular expression to allow it to match these
sorts of paths. We still use the `semver` library to ensure that the
version we've extracted from the directory name is a valid semver.
