We can't correctly print simple messages for prelude events when doing progress based display in a terminal, as it would lead to resetting the display of the table rendering.
This does mean that `--show-config` no longer works in the default terminal display mode - but it's not clear it *can* work correctly (at least as currently implemented) since it doesn't cleanly participate in the table rendering.
For cases where `--show-config` is not set (the norm) -nothing would have been printed anyway, so the changes here just avoid resetting the table rendering unnecessarily.
Fixes#3469.
Adds a new experimental `pulumi watch` CLI command which can be used for inner loop development on a Pulumi stack. This command is only available currently via `PULUMI_EXPERIMENTAL=true` while in active development.
The `watch` command does the following:
1. Watches the workspace (the tree rooted at the `Pulumi.yaml` file) for changes
2. Triggers an `update` to the stack whenever there is a change
3. Streams output containing summaries of key update events as well as logs from any resources under management into a combined CLI output
Part of https://github.com/pulumi/pulumi/issues/3448.
The PULUMI_EXPERIMENTAL flag also makes`query` and `policy` available.
* Add the ability to log all engine events to a file.
The path to the file can be specified using the `--event-log` flag to
the CLI. The file will be truncated if it exists. Events are written as
a list of JSON values using the schema described by `pkg/apitype`.
* Expose update engine events to ExtraRuntimeValidation.
Just what it says on the tin. Events from previews are not exposed.
This package's flags conflict with those in google/glog. Replace all
references to this package with references to
pulumi/pulumi/pkg/util/logging, and change that package to explicitly
call `flag.CommandLine.Parse` with an empty slice.
This should make it much easier to consume these packages in downstream
repos that have direct or indirect dependencies on google/glog.
These changes add support for passing `ignoreChanges` paths to resource
providers. This is intended to accommodate providers that perform diffs
between resource inputs and resource state (e.g. all Terraform-based
providers, the k8s provider when using API server dry-runs). These paths
are specified using the same syntax as the paths used in detailed diffs.
In addition to passing these paths to providers, the existing support
for `ignoreChanges` in inputs has been extended to accept paths rather
than top-level keys. It is an error to specify a path that is missing
one or more component in the old or new inputs.
Fixes#2936, #2663.
This removes the need for a replace directive in every downstream `go.mod`,
however it does not protect against the case of a `go.mod` being added upstream
with a different import path in future. This seems unlikely given the cadence of
work upstream, however.
The current CLI update view attributes all policy violation errors to
the root Stack resource. This commit will attribute them to the resource
that violated the policy.
The reason for this mis-attribution is a simple bookkeeping error:
* Resource policies intercept and prevent RegisterResource requests for
when the resource in question violates some policy.
* The CLI "tree" view of resources "hides" rows for resources that have
not been registered. Thus, if a policy violation occurs for a
resource, it becomes "orphaned" and is attributed to the stack,
because there is no row for the resource that violates the policy.
The solution, thus, is to simply set the "hidden" flag to false when we
encounter a policy violation.
Fixespulumi/pulumi-policy#25
* Plumbing the custom timeouts from the engine to the providers
* Plumbing the CustomTimeouts through to the engine and adding test to show this
* Change the provider proto to include individual timeouts
* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface
* Change how the CustomTimeouts are sent across RPC
These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest
```
req=&pulumirpc.RegisterResourceRequest{
Type: "aws:s3/bucket:Bucket",
Name: "my-bucket",
Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
Custom: true,
Object: &structpb.Struct{},
Protect: false,
Dependencies: nil,
Provider: "",
PropertyDependencies: {},
DeleteBeforeReplace: false,
Version: "",
IgnoreChanges: nil,
AcceptSecrets: true,
AdditionalSecretOutputs: nil,
Aliases: nil,
CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{
Create: 300,
Update: 400,
Delete: 500,
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
},
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
}
```
* Changing the design to use strings
* CHANGELOG entry to include the CustomTimeouts work
* Changing custom timeouts to be passed around the engine as converted value
We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
A resource can be imported by setting the `import` property in the
resource options bag when instantiating a resource. In order to
successfully import a resource, its desired configuration (i.e. its
inputs) must not differ from its actual configuration (i.e. its state)
as calculated by the resource's provider.
There are a few interesting state transitions hiding here when importing
a resource:
1. No prior resource exists in the checkpoint file. In this case, the
resource is simply imported.
2. An external resource exists in the checkpoint file. In this case, the
resource is imported and the old external state is discarded.
3. A non-external resource exists in the checkpoint file and its ID is
different from the ID to import. In this case, the new resource is
imported and the old resource is deleted.
4. A non-external resource exists in the checkpoint file, but the ID is
the same as the ID to import. In this case, the import ID is ignored
and the resource is treated as it would be in all cases except for
changes that would replace the resource. In that case, the step
generator issues an error that indicates that the import ID should be
removed: were we to move forward with the replace, the new state of
the stack would fall under case (3), which is almost certainly not
what the user intends.
Fixes#1662.
Instead of simply converting a detailed diff entry that indicates an
update to an entire composite value as a simple old/new value diff,
compute the nested diff. This alllows us to render a per-element diff
for the nested object rather than simply displaying the new and the old
composite values.
This is necessary in order to improve diff rendering once
pulumi/pulumi-terraform#403 has been rolled out.
Thse changes make a subtle but critical adjustment to the process the
Pulumi engine uses to determine whether or not a difference exists
between a resource's actual and desired states, and adjusts the way this
difference is calculated and displayed accordingly.
Today, the Pulumi engine get the first chance to decide whether or not
there is a difference between a resource's actual and desired states. It
does this by comparing the current set of inputs for a resource (i.e.
the inputs from the running Pulumi program) with the last set of inputs
used to update the resource. If there is no difference between the old
and new inputs, the engine decides that no change is necessary without
consulting the resource's provider. Only if there are changes does the
engine consult the resource's provider for more information about the
difference. This can be problematic for a number of reasons:
- Not all providers do input-input comparison; some do input-state
comparison
- Not all providers are able to update the last deployed set of inputs
when performing a refresh
- Some providers--either intentionally or due to bugs--may see changes
in resources whose inputs have not changed
All of these situations are confusing at the very least, and the first
is problematic with respect to correctness. Furthermore, the display
code only renders diffs it observes rather than rendering the diffs
observed by the provider, which can obscure the actual changes detected
at runtime.
These changes address both of these issues:
- Rather than comparing the current inputs against the last inputs
before calling a resource provider's Diff function, the engine calls
the Diff function in all cases.
- Providers may now return a list of properties that differ between the
requested and actual state and the way in which they differ. This
information will then be used by the CLI to render the diff
appropriately. A provider may also indicate that a particular diff is
between old and new inputs rather than old state and new inputs.
Fixes#2453.
Adds a new resource option `aliases` which can be used to rename a resource. When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource.
There are two key places this change is implemented.
The first is the step generator in the engine. When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource. That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order). This can allow the resource to be matched as a (potential) update to an existing resource with a different URN.
The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs. This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases. It is similar to how many other resource options are "inherited" implicitly from the parent.
Four specific scenarios are explicitly tested as part of this PR:
1. Renaming a resource
2. Adopting a resource into a component (as the owner of both component and consumption codebases)
3. Renaming a component instance (as the owner of the consumption codebase without changes to the component)
4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase)
4. Combining (1) and (3) to make both changes to a resource at the same time
If --suppress-outputs is passed to `pulumi preview --json`, we
should not emit the stack outputs. This change fixespulumi/pulumi#2765.
Also adds a test case for this plus some variants of updates.
We were not actually calling our colorization routines, which lead to
printing this very confusing text:
```
<{%reset%}> --outputs:--<{%reset%}>
```
When running updates with `--diff` or when drilling into details of a
proposed operation, like a refresh.
We were dropping new and old states on the floor instead of including
them as part of the previewed operation due to a logic error (we want
to append them when there are no errors from serialization, vs when
there are errors).
When constructing a Deployment (which is a plaintext representation of
a Snapshot), ensure that we encrypt secret values. To do so, we
introduce a new type `secrets.Manager` which is able to encrypt and
decrypt values. In addition, it is able to reflect information about
itself that can be stored in the deployment such that we can
deserialize the deployment into a snapshot (decrypting the values in
the process) without external knowledge about how it was encrypted.
The ability to do this is import for allowing stack references to
work, since two stacks may not use the same manager (or they will use
the same type of manager, but have different state).
The state value is stored in plaintext in the deployment, so it **must
not** contain sensitive data.
A sample manager, which just base64 encodes and decodes strings is
provided, as it useful for testing. We will allow it to be varried
soon.
The Pulumi CLI currently has two "display modes" -- one for rendering
diffs and one for rendering program updates. Neither of these is
particularly well-suited to `pulumi query`, which essentially needs to
render stdout from the query program verbatim.
This commit will add a separate display mode for this purpose:
* In interactive mode, `pulumi query` will use the display spinner to
say "running query". In non-interactive mode, this will be omitted.
* Query mode will print stdout from the query program verbatim, but
ignore `diag.Infoerr` so that they're not double-printed when they're
emitted again as error events.
* Query mode also does not double-print newlines at the end of diag
events.
This change adds a --json flag to the preview command, enabling
basic JSON serialization of preview plans. This effectively flattens
the engine event stream into a preview structure that contains a list
of steps, diagnostics, and summary information. Each step contains
the deep serialization of resource state, in addition to metadata about
the step, such as what kind of operation it entails.
This is a partial implementation of pulumi/pulumi#2390. In particular,
we only support --json on the `preview` command itself, and not `up`,
meaning that it isn't possible to serialize the result of an actual
deployment yet (thereby limiting what you can do with outputs, etc).
This commit switches from dep to Go 1.12 modules for tracking Pulumi
dependencies. Rather than _building_ using Go modules, we instead use the `go
mod vendor` command to populate a vendor tree in the same way as `dep ensure`
was previously doing.
In order to prevent checksum mismatches, it was necessary to also update CI to
use Go 1.12 instead of 1.11 - which also necessitated fixing some linting errors
which appeared with the upgraded golangci-lint for 1.12.
If a provider returns information about the top-level properties that
differ, use those keys to filter the diffs that are rendered to the
user.
Fixes#2453.
In general, a "delete" in Pulumi is destroying an actual physical
resource. In the case of a read resource, however, the delete is
merely removing the resource from the stack; the physical resource
is not affected. These changes attempt to clarify this situation by
using the term "discard" rather than "delete".
Fixes#2015.
The event rendering goroutine in the remote backend was not properly
synchronizing with the goroutine that created it, and could continue
executing after its creator finished. I believe that this is the root
cause of #1850.
This adds an option, --suppress-outputs, to many commands, to
avoid printing stack outputs for stacks that might contain sensitive
information in their outputs (like key material, and whatnot).
Fixespulumi/pulumi#2028.
