This change adds a `RegisterInputType` function (similar to the existing `RegisterOutputType`) that is used to register an input interface's type and its associated input type, and adds registrations for the built-in input types.
This will be used when copying inputs to an args struct for multi-lang components. When a field is typed as the input interface (e.g. `StringMapInput`) and doesn't need to be an `Output`, we can use this to lookup the non-Output type that implements the interface (e.g. `StringMap`) so it can be instantiated.
A subsequent change will update the Go SDK codegen to produce input type registrations for a provider's input types.
* Thread replaceOnChanges through the Go SDK
* Add replaceOnChanges to the .NET SDK
* Update CHANGELOG_PENDING.md
* Fix null error
* Update CHANGELOG_PENDING.md
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
This change expands the definition of `resource.Output` in the Go SDK with additional information about the output, i.e. dependencies and secretness, and adds support in the core Go RPC code for (un)marshaling output values.
Output values are marshaled as special objects ala archives, assets, and resource refs and are unmarshaled as `resource.Output` values.
Subsequent PRs will add:
- A monitor feature for output values, which will initially be disabled by default but available to turn on via an envvar
- Support for (un)marshaling output values in each language SDKs
- A way for providers to indicate support for receiving output values
- E2E tests
- Turn the monitor feature on by default (w/ env var to disable) (Note: the current plan is to initially scope this to only be used when marshaling inputs to a multi-language component)
Currently whenever an issue occurs in `UnmarshalProperties` and
`MarshalProperties` the offending property is hidden and very difficult
to track down.
This commit changes the behavior. For `Assets` and `Archives` the error
message now includes the URI and for other properties it includes the
key of the `PropertyMap`.
Developer documentation is written in Markdown and can be built into
HTML, PDF, etc. using Sphinx. Diagrams are written in PlantUML and
rendered as SVGs. All developer docs live in the `developer-docs` folder
under the root of the repository.
A few identically-typed variables got confused with the changes in #7737.
The confusion caused empty property values to be included in resources
that had any dependencies on other resources, which confused the
unmarshaling code for Go multi-language components. These changes fix
the typo and restore the original behavior, which is to omit empty
property values.
Co-authored-by: Emiliza Gutierrez <emiliza@pulumi.com>
* Work in progress, first passing tests
* Test unknown dep prop similar to Python
* Test fixes
* Fix lint
* Nit fix
* CHANGELOG
* Add ResourceInputArray and simplify the API
* Adopt urnSet
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.
Part of #7542.
Do not return the inputs as the state for update previews that use an
unconfigured provider. Returning the inputs as the state allows the
language SDKs to incorrectly treat unknown properties as known (because
we can't call `Update` on an unconfigured provider, we can't know which
properties are unknown). Users can re-enable the existing behavior by
setting the `PULUMI_LEGACY_PROVIDER_PREVIEW` environment variable to a
truthy value (e.g. `1`, `true`, etc.).
Most users will be unaffected by these changes. The most common programs
that may be affected are those that combine the creation of a managed
Kubernetes cluster with the deployment of applications to that cluster. These
programs generally need to configure a k8s provider instance by constructing
a kubeconfig from the output of the managed k8s cluster. Any changes to the
cluster that cause the kubeconfig to be unknown then cause the provider to
go unconfigured at runtime. Prior to these changes, resources managed by the
k8s provider would have some known outputs in this scenario, as the engine
would treat the resource's input values as its output values. After these changes,
the resource's outputs will be treated as unknown. The most frequent affect
that this has is that applies/stack outputs that depend on the outputs of
a k8s resource managed by a provider with an unknown kubeconfig will not
run/be displayed as `output`s during previews, respectively.
We might be able to improve on this by taking advantage of schema
information and filling in unknown values for properties that do not
exist in the inputs.
Fixes#7521.
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Luke Hoban <luke@pulumi.com>
Fix `cloudSourceControlSSHRegex` so that it will match git remotes with periods or hyphens in the hostname. However, `azureSourceControlSSHRegex` _does_ match hostnames with a period. As a result, `TryGetVCSInfo` will treat these sorts of remotes as Azure source control remotes and drop the first group. This causes updates to have an incorrect `"vcs.kind"` field. For example, an update with a Git remote of `github.foo.acme.com` will have a `"vcs.kind"` field of `foo.acme.com`. This occurs if a user is using a self-hosted GH enterprise instance.
* Experiment with gotestsum and test timings
* Fix to locating the helper script
* Fix the code for installing gotestsum
* Try alternative installation method
* Use go to compute test stats; Python fails parsing time values
* Try version without v
* Try with fixed gorelaser config
* Fix test time correlation
* Try a stable test stat sort finally
* Use more accurate test duration aggregation
* Include python and auto-api tests in the Go timing counts
* Bring back TESTPARALLELISM
* Fix test compilation
* Only top 100 slow tests
* Try to fracture build matrix to fan out tests
* Do not run Publish Test Results on unsuppored Mac
* Auto-create test-results-dir
* Fix new flaky test by polling for logs
* Try to move native tests to their own config
* Actually skip
* Do not fail on empty test-results folder
* Try again
* Try once more
* Integration test config is the crit path - make it smaller
* Squash underutilized test configurations
* Remove the test result summary box from PR - counts now incorrec
* Remove debugging step
Previously, any provider resource passed to multi-lang components would be instantiated as a `DependencyProviderResource` inside `Construct`, which prevents the component from being able to easily access the provider's state as an instance of of the provider (e.g. `*aws.Provider`).
This change attempts to rehydrate the provider resource in the same way that resource references are rehydrated, if it's been registered, s.t. the specific provider resource type is instantiated with its state. Otherwise falling back to returning `DependencyProviderResource`.
The two more strongly-typed Pulumi SDKs curently fail with an error
during unmarshaling when attempting to marshal a value that is not an
asset into an asset-typed location (e.g. an asset-typed resource
output property). While this behavior is reasonable on its face, it
gives rise to practical challenges when dealing with TF-provider-backed
resources that have asset-typed properties. When such a resource is
refreshed, the values of its asset-typed properties are replaced with
non-asset values, as the TF bridge can't currently create a resonable
stand-in asset value.
For example, consider an S3 bucket object:
```
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
const bucket = new aws.s3.Bucket("my-bucket");
new aws.s3.BucketObject("my-object", {
source: new pulumi.FileAsset("some/file"),
});
```
Upon creation, the value of the input property `source` will be a file
asset backed by the path `some/file`. The bridge will propagate this
value to the `source` output property; this propagation is safe because
the resource was just created and so the output property must have the
value that was passed by the program.
Now, let some actor apply out-of-band changes to the contents of the
bucket object s.t. the `source` property changes when the object is
refreshed. In that case, the `source` property will be a string value
which the bridge is unable to interpret as an asset. The next time the
Pulumi program is run, the Go or .NET SDK will attempt to deserialize
the string into an asset-typed property and will fail.
With these changes, the deserialization would not fail, and would
instead create an asset or archive value that will fail to marshal if
passed to another resource. Users can avoid these errors by not passing
asset or archive outputs to other resources/stack outputs.
These changes unblock users who are hitting
https://github.com/pulumi/pulumi-aws/issues/1521.
This fixes#7504, and is the other half of the interim fix corresponding
to #7359. The case listed in #7509 no longer reproduces after this fix
is applied.
Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.
Fixes#6753.
Co-authored-by: Levi Blackstone <levi@pulumi.com>
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
These changes contain a preliminary fix for #7359 in the Go SDK. The fix
handles input values that are nested one level deep within maps and
arrays, but does not handle other cases of nested input types.
Rotating a passphrase requires that the old passphrase is available via
one of the `PULUMI_CONFIG_PASSPHRASE` or `PULUMI_CONFIG_PASSPHRASE_FILE`
environment variables. This confuses `readPassphrase` when reading a new
passphrase, since that function checks the aforementioned environment
variables prior to reading from the console. The overall effect is that
it is impossible to rotate the passphrase for a stack using the
passphrase provider. These changes fix this by always reading from the
console when rotating a passphrase.
This commit moves from simply asserting on the assignability of types to
using `Assertf` to print the types in question. This provides more
information to a user whose code is panicking because of
non-assignability.
Ultimately it would likely be better to surface this via error messages
instead of via panic, but this at least improves the debuggability in
the meantime.
This command converts an appdash trace into a pprof file for use with
`go tool pprof`. Spans are converted into stacks by sampling each root
span at a given rate and recording the stack of subspans at each sample.
These changes also replace the conditional addition of experimental and
debug commands with conditional visibility. Experimental and debug
commands will always be available, but will be hidden unless the
appropraite environment variables are set.
Co-authored-by: Levi Blackstone <levi@pulumi.com>
* Add trace proxying to fix sub-process trace collection when tracing to files
* Better func naming in test
* Avoid dealing with Windows path nightmare
* On Windows it is go.exe of course
* Rename operation to component to better align with existing trace output
This change is a simple perf optimization to speed up the process of listing plugins by excluding some metadata like size by default.
Our strategy for finding a plugin is to first look on the path, and then to iterate through all plugins in the plugin cache (a directory). We do this for each plugin that is loaded when NewProvider is called. Unfortunately, the codepath that gets all plugins is shared by pulumi plugin ls that needs to do things like display the total size of all plugins, the size of each plugin, and when the plugin was last installed/last used.
This means that any time a plugin is loaded, we are computing the size of all plugins by recursively enumerating all folder (including all of the node_modules directories of any installed node multi-lang plugins!). For my 5 gb of node plugins this translated to 10s of overhead each time a plugin was loaded.
This change is a very simple fix. pulumi plugin ls is the only code path that uses size, so we create a dedicated code path GetPluginsWithMetadata that populates that info, excluding from the result of GetPlugins by default.
* Send plugin install output to stderr
We currently send plugin install output to stdout. This interferes
with --json (#5747), automation API scenarios, and in general is bad
CLI hygiene. This change sends plugin output to stdout instead.
* Add a changelog entry
Temporarily disable the new config secret warning to avoid unactionable warnings from provider `config` modules. We'll re-enable the warning when we've addressed that issue.
The Pulumi Go SDK does not currently await all outstanding asynchronous
work associated with a Pulumi program. Because all relevant asynchronous
work is created via the Pulumi SDK, we can track this asynchronous work
and ensure that it has all completed prior to returning from
`Context.Run`.
This is complicated by the fact that many of the existing APIs that are
able to create `Output`s--`NewOutput`, `ToOutput`, `Any`, `ToSecret`,
and `All`--do not have a `*Context` parameter, and so have no
straightforward way to associate themselves with a `*Context`. To address
this, these changes add new versions of each of these APIs as methods on
`*Context`.
Despite these new methods, most Pulumi programs should work without
changes: the bulk of `Output`s are created by the SDK itself as part of
resource registration, and for `Any` and `All`, we can pick up the
context from any `Output`s present in the arguments. The only programs
that should require changes are those that create outputs from whole
cloth using `NewOutput`, `ToOutput`, or `ToSecret` and create unawaited
async work rooted at those outputs.
On an implementation level, these changes track asynchronous work using
a `sync.WaitGroup` associated with each `*Context`. This `WaitGroup` is
passed to each output associated with the context. The SDK increments
this `WaitGroup`'s count prior to starting any asynchronous work and
decrements it once the work (including any callbacks triggered by the
work) is complete.
This fixes the Go portion of #3991.
* Propagate workspace.Project metadata to plugin init
* Get to a working fix
* Propagate Root via plugin context
* Propagate root instead of yaml path
* Revert out unnecessary parameter propagation
* Root is now always absolute at this point; simplify code and docs
* Drop python conditional and propagate unused -root to all lang hosts
* Add tests that fail before and pass after
* Lint
* Add changelog entry
Config values that are objects are represented in memory as JSON strings. When a config map is being saved to a file, object values are first unmarshaled from JSON to `interface{}` and then the entire config map is marshaled to YAML (or JSON) and saved to disk. When an object value is unmarshaled from JSON, any numbers in the JSON string were being implicitly unmarshaled as `float64`, which resulted in some numbers in the nested objects being emitted in YAML using exponential notation (e.g. a number `12321123131` in an object value was being saved in the YAML as `1.2321123131e+10`). To address this, when unmarshaling the JSON for an object value, first try to unmarshal any numbers as `int64`, falling back to `float64`.
This commit makes it possible to opt out of loading plugins from PATH by
setting PULUMI_IGNORE_AMBIENT_PLUGINS to any non-empty value. This is
useful when automatic IDE tooling may build remote component plugins
into GOBIN unbeknownst to the user, and a resulting stale version of the
plugin is loaded in place of newer versions - even those , explicitly
installed.
When calling l.GetEnvVars() to evaluate whether version checking should
be skipped because of variables passed into the workspace, they have not
yet been set. This commit modifies the logic to look at the environment
variables in the options instead of in the workspace.
Since the original functionality has not yet been released, I do not
believe it warrants a CHANGELOG entry.
#6636 inadvertently changed the minimum Go version
requirement to 1.16 since ReadFile was moved in that
version. Switch back to ioutil.ReadFile to avoid forcing
an upgrade at this time.
