* Enable output values by default
Enable output values by default in the resource monitor and change the polarity of the envvar from `PULUMI_ENABLE_OUTPUT_VALUES` to `PULUMI_DISABLE_OUTPUT_VALUES`.
* Marshal unknown as unknown string when `!KeepOutputValues`
Marshal all unknown output values as `resource.MakeComputed(resource.NewStringProperty(""))` when not keeping output values, which is consistent with what the SDKs do.
Otherwise, when `v.OutputValue().Element` is nil, `resource.MakeComputed(v.OutputValue().Element)` will be marshaled as a null value rather than as an unknown sentinel.
* Add MarshalOptions.DontSkipOutputs and use where needed
Before we expanded the meaning of `resource.Output`, `MarshalProperties` always skipped output values:
```go
if v.IsOutput() {
logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```
As part of expanding the meaning of `resource.Output`, I'd adjusted `MarshalProperties` to only skip output values when the value was unknown and when not keeping output values:
```go
if v.IsOutput() && !v.OutputValue().Known && !opts.KeepOutputValues {
logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```
However, this doesn't work the way we want when marshaling properties that include unknown output values to a provider that does not accept outputs. In that case, `opts.KeepOutputValues` will be `false` because we want the marshaler to fall back to returning non-output-values (e.g. unknown sentinel value for unknown output values), but instead of getting the intended fallback values, the unknown output values are skipped (not what we want).
I suspect we may be able to delete the output value skipping in `MarshalProperties` altogether (it's odd that it is skipping `resource.Output` but not `resource.Computed`), but to avoid any unintended side effects of doing that, instead, this commit introduces a new `MarshalOptions.DontSkipOutputs` option that can be set to `true` to opt-in to not skipping output values when marshaling. The check in `MarshalProperties` now looks like this:
```go
if !opts.DontSkipOutputs && v.IsOutput() && !v.OutputValue().Known {
logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```
`opts.DontSkipOutputs` is set to `true` when marshaling properties for calls to a provider's `Construct` and `Call`.
* [sdk/nodejs] Deserialize output values
This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK.
* [sdk/python] Deserialize output values
This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK.
This schema can be used to validate the contents of a Pulumi deployment.
If a deployment validates against this schema, it should be considered
syntactically valid, though it may contain certain classes of semantic
errors (e.g. references to unknown resources in dependency lists,
dependency cycles, etc.).
This schema is not yet used for validation in practice and may not be
entirely accurate.
These changes also add this schema (and the schemas on which it depends)
to the developer documentation. jsonschema2md.go has been updated to
support multi-file schemas.
This change adds a `RegisterInputType` function (similar to the existing `RegisterOutputType`) that is used to register an input interface's type and its associated input type, and adds registrations for the built-in input types.
This will be used when copying inputs to an args struct for multi-lang components. When a field is typed as the input interface (e.g. `StringMapInput`) and doesn't need to be an `Output`, we can use this to lookup the non-Output type that implements the interface (e.g. `StringMap`) so it can be instantiated.
A subsequent change will update the Go SDK codegen to produce input type registrations for a provider's input types.
* Thread replaceOnChanges through the Go SDK
* Add replaceOnChanges to the .NET SDK
* Update CHANGELOG_PENDING.md
* Fix null error
* Update CHANGELOG_PENDING.md
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
This change expands the definition of `resource.Output` in the Go SDK with additional information about the output, i.e. dependencies and secretness, and adds support in the core Go RPC code for (un)marshaling output values.
Output values are marshaled as special objects ala archives, assets, and resource refs and are unmarshaled as `resource.Output` values.
Subsequent PRs will add:
- A monitor feature for output values, which will initially be disabled by default but available to turn on via an envvar
- Support for (un)marshaling output values in each language SDKs
- A way for providers to indicate support for receiving output values
- E2E tests
- Turn the monitor feature on by default (w/ env var to disable) (Note: the current plan is to initially scope this to only be used when marshaling inputs to a multi-language component)
Currently whenever an issue occurs in `UnmarshalProperties` and
`MarshalProperties` the offending property is hidden and very difficult
to track down.
This commit changes the behavior. For `Assets` and `Archives` the error
message now includes the URI and for other properties it includes the
key of the `PropertyMap`.
Developer documentation is written in Markdown and can be built into
HTML, PDF, etc. using Sphinx. Diagrams are written in PlantUML and
rendered as SVGs. All developer docs live in the `developer-docs` folder
under the root of the repository.
A few identically-typed variables got confused with the changes in #7737.
The confusion caused empty property values to be included in resources
that had any dependencies on other resources, which confused the
unmarshaling code for Go multi-language components. These changes fix
the typo and restore the original behavior, which is to omit empty
property values.
Co-authored-by: Emiliza Gutierrez <emiliza@pulumi.com>
* Work in progress, first passing tests
* Test unknown dep prop similar to Python
* Test fixes
* Fix lint
* Nit fix
* CHANGELOG
* Add ResourceInputArray and simplify the API
* Adopt urnSet
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.
Part of #7542.
Do not return the inputs as the state for update previews that use an
unconfigured provider. Returning the inputs as the state allows the
language SDKs to incorrectly treat unknown properties as known (because
we can't call `Update` on an unconfigured provider, we can't know which
properties are unknown). Users can re-enable the existing behavior by
setting the `PULUMI_LEGACY_PROVIDER_PREVIEW` environment variable to a
truthy value (e.g. `1`, `true`, etc.).
Most users will be unaffected by these changes. The most common programs
that may be affected are those that combine the creation of a managed
Kubernetes cluster with the deployment of applications to that cluster. These
programs generally need to configure a k8s provider instance by constructing
a kubeconfig from the output of the managed k8s cluster. Any changes to the
cluster that cause the kubeconfig to be unknown then cause the provider to
go unconfigured at runtime. Prior to these changes, resources managed by the
k8s provider would have some known outputs in this scenario, as the engine
would treat the resource's input values as its output values. After these changes,
the resource's outputs will be treated as unknown. The most frequent affect
that this has is that applies/stack outputs that depend on the outputs of
a k8s resource managed by a provider with an unknown kubeconfig will not
run/be displayed as `output`s during previews, respectively.
We might be able to improve on this by taking advantage of schema
information and filling in unknown values for properties that do not
exist in the inputs.
Fixes#7521.
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Luke Hoban <luke@pulumi.com>
Fix `cloudSourceControlSSHRegex` so that it will match git remotes with periods or hyphens in the hostname. However, `azureSourceControlSSHRegex` _does_ match hostnames with a period. As a result, `TryGetVCSInfo` will treat these sorts of remotes as Azure source control remotes and drop the first group. This causes updates to have an incorrect `"vcs.kind"` field. For example, an update with a Git remote of `github.foo.acme.com` will have a `"vcs.kind"` field of `foo.acme.com`. This occurs if a user is using a self-hosted GH enterprise instance.
* Experiment with gotestsum and test timings
* Fix to locating the helper script
* Fix the code for installing gotestsum
* Try alternative installation method
* Use go to compute test stats; Python fails parsing time values
* Try version without v
* Try with fixed gorelaser config
* Fix test time correlation
* Try a stable test stat sort finally
* Use more accurate test duration aggregation
* Include python and auto-api tests in the Go timing counts
* Bring back TESTPARALLELISM
* Fix test compilation
* Only top 100 slow tests
* Try to fracture build matrix to fan out tests
* Do not run Publish Test Results on unsuppored Mac
* Auto-create test-results-dir
* Fix new flaky test by polling for logs
* Try to move native tests to their own config
* Actually skip
* Do not fail on empty test-results folder
* Try again
* Try once more
* Integration test config is the crit path - make it smaller
* Squash underutilized test configurations
* Remove the test result summary box from PR - counts now incorrec
* Remove debugging step
Previously, any provider resource passed to multi-lang components would be instantiated as a `DependencyProviderResource` inside `Construct`, which prevents the component from being able to easily access the provider's state as an instance of of the provider (e.g. `*aws.Provider`).
This change attempts to rehydrate the provider resource in the same way that resource references are rehydrated, if it's been registered, s.t. the specific provider resource type is instantiated with its state. Otherwise falling back to returning `DependencyProviderResource`.
The two more strongly-typed Pulumi SDKs curently fail with an error
during unmarshaling when attempting to marshal a value that is not an
asset into an asset-typed location (e.g. an asset-typed resource
output property). While this behavior is reasonable on its face, it
gives rise to practical challenges when dealing with TF-provider-backed
resources that have asset-typed properties. When such a resource is
refreshed, the values of its asset-typed properties are replaced with
non-asset values, as the TF bridge can't currently create a resonable
stand-in asset value.
For example, consider an S3 bucket object:
```
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
const bucket = new aws.s3.Bucket("my-bucket");
new aws.s3.BucketObject("my-object", {
source: new pulumi.FileAsset("some/file"),
});
```
Upon creation, the value of the input property `source` will be a file
asset backed by the path `some/file`. The bridge will propagate this
value to the `source` output property; this propagation is safe because
the resource was just created and so the output property must have the
value that was passed by the program.
Now, let some actor apply out-of-band changes to the contents of the
bucket object s.t. the `source` property changes when the object is
refreshed. In that case, the `source` property will be a string value
which the bridge is unable to interpret as an asset. The next time the
Pulumi program is run, the Go or .NET SDK will attempt to deserialize
the string into an asset-typed property and will fail.
With these changes, the deserialization would not fail, and would
instead create an asset or archive value that will fail to marshal if
passed to another resource. Users can avoid these errors by not passing
asset or archive outputs to other resources/stack outputs.
These changes unblock users who are hitting
https://github.com/pulumi/pulumi-aws/issues/1521.
This fixes#7504, and is the other half of the interim fix corresponding
to #7359. The case listed in #7509 no longer reproduces after this fix
is applied.
