The underlying library `dill` that we use for serializing dynamic providers into Pulumi state for Python dynamic providers serializes classes differently depending on whether they are in `__main__` or in another module. We need the by-value serialization to be applied in all cases.
https://github.com/uqfoundation/dill/issues/424 is tracking adding the ability into `dill` to specify this by-value serialization explicitly, but until then, we will temporarily re-write the `__module__` of a provder class prior to serialization, so that `dill` behaves as we need for the dynamic provider use case.
Fixes#7453.
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.
Part of #7542.
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.
Part of #7542.
Do not return the inputs as the state for update previews that use an
unconfigured provider. Returning the inputs as the state allows the
language SDKs to incorrectly treat unknown properties as known (because
we can't call `Update` on an unconfigured provider, we can't know which
properties are unknown). Users can re-enable the existing behavior by
setting the `PULUMI_LEGACY_PROVIDER_PREVIEW` environment variable to a
truthy value (e.g. `1`, `true`, etc.).
Most users will be unaffected by these changes. The most common programs
that may be affected are those that combine the creation of a managed
Kubernetes cluster with the deployment of applications to that cluster. These
programs generally need to configure a k8s provider instance by constructing
a kubeconfig from the output of the managed k8s cluster. Any changes to the
cluster that cause the kubeconfig to be unknown then cause the provider to
go unconfigured at runtime. Prior to these changes, resources managed by the
k8s provider would have some known outputs in this scenario, as the engine
would treat the resource's input values as its output values. After these changes,
the resource's outputs will be treated as unknown. The most frequent affect
that this has is that applies/stack outputs that depend on the outputs of
a k8s resource managed by a provider with an unknown kubeconfig will not
run/be displayed as `output`s during previews, respectively.
We might be able to improve on this by taking advantage of schema
information and filling in unknown values for properties that do not
exist in the inputs.
Fixes#7521.
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Luke Hoban <luke@pulumi.com>
Similar to #5282, but for core SDK types. The tl;dr is that because
Sequence[T] is covariant, constructing resources becomes much more
ergonomic.
Fix#7693.
Fix `cloudSourceControlSSHRegex` so that it will match git remotes with periods or hyphens in the hostname. However, `azureSourceControlSSHRegex` _does_ match hostnames with a period. As a result, `TryGetVCSInfo` will treat these sorts of remotes as Azure source control remotes and drop the first group. This causes updates to have an incorrect `"vcs.kind"` field. For example, an update with a Git remote of `github.foo.acme.com` will have a `"vcs.kind"` field of `foo.acme.com`. This occurs if a user is using a self-hosted GH enterprise instance.
* Inputty depends_on support for Python
* Fix circular import
* Add a test, fix bugs found by test
* Fix typo
* Minimize loss of dep information in presence of unks to match Node behavior
* Add CHANGELOG entr
* Update sdk/python/lib/pulumi/output.py
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Update sdk/python/lib/test/test_resource.py
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Fix _is_prompt
* Enhance tests, found out a gap against Node impl
* Add non-listy case
* Internal functional combinators
* Do not use all/deep await when merging
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* [backend/filestate] Allow `pulumi stack ls` to see all stacks regardless of passphrase
The information exposed via `pulumi stack ls` does not require being able to decrypt state files, but the existing logic for `pulumi stack ls` with the filestate backend was to fully decrypt the state file anyway, silently skipping any stacks that could not be decrypted. This led to surprising results from `pulumi stack ls`.
After these changes, `pulumi stack ls` with the filestate backend will list *all* stacks that are available. Notably, because there is no notion of "project" scoping in the fielstate backend (yet), `pulumi stack ls` will list all stacks independent of the project name.
Fixes#4798.
When initializing `DependencyProviderResource`, pass the package to the base constructor instead of an empty string s.t. the provider is usable when its package is read.
Previously, any provider resource passed to multi-lang components would be instantiated as a `DependencyProviderResource` inside `Construct`, which prevents the component from being able to easily access the provider's state as an instance of of the provider (e.g. `*aws.Provider`).
This change attempts to rehydrate the provider resource in the same way that resource references are rehydrated, if it's been registered, s.t. the specific provider resource type is instantiated with its state. Otherwise falling back to returning `DependencyProviderResource`.
* Reproduce the issue in a failing test
* Fix
* Tentative fix
* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Update sdk/dotnet/Pulumi/Deployment/Deployment.Runner.cs
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Do not allocate TaskCompletionSource when not needed
* Update sdk/dotnet/Pulumi/Deployment/Deployment.Runner.cs
Co-authored-by: Josh Studt <32800478+orionstudt@users.noreply.github.com>
* Fix warning
* Cache delegate
* Simplify with named tuples
* Test early exception termination
* Test logging
* Remove the smelly method of suppressing engine exceptions
* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs
Co-authored-by: Josh Studt <32800478+orionstudt@users.noreply.github.com>
* Fix typo; check in xml docs
* Try CI again
* Add CHANGELOG entry
* Dedup exceptions before reporting
* Lock access to _exceptions list
* Fix typos
* Version of HandleExceptionsAsync that accepts N exceptions
* Do not aggregate exceptions prematurely
* Rename private members
* Formatting
* Summary markers
* Short-circuit return
* Stylistic fixes
* Strengthen test
* Check that we have only 1 exception
* Remove defensive clause about AggregateException from the test
* Simplify TerminatesEarly test
* Remove EmptyStack
* Notes on the regression nature of the WorksUnderStress test
* Remove race condition repro as it is a poor repro, impossible to trigger from user code
* Brace style
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Josh Studt <32800478+orionstudt@users.noreply.github.com>
The two more strongly-typed Pulumi SDKs curently fail with an error
during unmarshaling when attempting to marshal a value that is not an
asset into an asset-typed location (e.g. an asset-typed resource
output property). While this behavior is reasonable on its face, it
gives rise to practical challenges when dealing with TF-provider-backed
resources that have asset-typed properties. When such a resource is
refreshed, the values of its asset-typed properties are replaced with
non-asset values, as the TF bridge can't currently create a resonable
stand-in asset value.
For example, consider an S3 bucket object:
```
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
const bucket = new aws.s3.Bucket("my-bucket");
new aws.s3.BucketObject("my-object", {
source: new pulumi.FileAsset("some/file"),
});
```
Upon creation, the value of the input property `source` will be a file
asset backed by the path `some/file`. The bridge will propagate this
value to the `source` output property; this propagation is safe because
the resource was just created and so the output property must have the
value that was passed by the program.
Now, let some actor apply out-of-band changes to the contents of the
bucket object s.t. the `source` property changes when the object is
refreshed. In that case, the `source` property will be a string value
which the bridge is unable to interpret as an asset. The next time the
Pulumi program is run, the Go or .NET SDK will attempt to deserialize
the string into an asset-typed property and will fail.
With these changes, the deserialization would not fail, and would
instead create an asset or archive value that will fail to marshal if
passed to another resource. Users can avoid these errors by not passing
asset or archive outputs to other resources/stack outputs.
These changes unblock users who are hitting
https://github.com/pulumi/pulumi-aws/issues/1521.
In short, this allows subtyping to correctly "propagate" through an Output;
if Cow is a subtype of Animal, this commit makes it so Output[Cow] is
treated as a subtype of Output[Animal].
Without this change, users of the Python SDK occasionally contend with a
confusing error message that is resolved by adding a call to
`.apply(lambda x: x)` to satisfy mypy.
Touches #3767.
Fix#6843.
These changes add support for unmarshaling and marshaling package
schemas using YAML instead of JSON. Language-specific data is
canonically JSON. Users of the `*Spec` types will need to update the
types of the the their `Language` values to use the new
`schema.RawMessage` type instead of `json.RawMessage`: the former has
support for YAML while the latter does not.
Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.
Fixes#6753.
Co-authored-by: Levi Blackstone <levi@pulumi.com>
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
These changes contain a preliminary fix for #7359 in the Go SDK. The fix
handles input values that are nested one level deep within maps and
arrays, but does not handle other cases of nested input types.
Rotating a passphrase requires that the old passphrase is available via
one of the `PULUMI_CONFIG_PASSPHRASE` or `PULUMI_CONFIG_PASSPHRASE_FILE`
environment variables. This confuses `readPassphrase` when reading a new
passphrase, since that function checks the aforementioned environment
variables prior to reading from the console. The overall effect is that
it is impossible to rotate the passphrase for a stack using the
passphrase provider. These changes fix this by always reading from the
console when rotating a passphrase.
* resolve issue with exit code not bubbling up and inflight tasks swallowing exceptions
* update changelog
* refacter to reduce and centralize inline program exception flow complexity
* quick comments on new pulumi function return type
Although `Output` objects can never correct support iteration, Python will see the implementation of `__getitem__` and try to iterate the object, leading to an infinite loop. To prevent this, we need to explicitly implement `__iter__` and make it return a `TypeError` to prevent iteration (and offer a useful error message).
Fixes#5028.
`log.debug` messages are part of an update and get sent to the service. Reducing the number of `log.debug` calls can significantly improve the performance of a program. This commit changes the debug logging in the Python SDK to be similar to the debug logging in the Node.js SDK.
- An `excessive_debug_output` variable has been added (that can be set to `True` during debugging), but otherwise will avoid a large number of `log.debug` calls.
- Some unhelpful/redundant `log.debug` calls have been deleted.
Every time a resource is created (without an explicit provider version specified), `_utilities.get_version()` is called to determine the package version. This call is expensive and can measurably impact performance, especially for programs with many resources. This change caches the value so it is only determined once.
Python resource constructor overloads were recently added that accept a
`<Resource>Args` class for input properties, as an alternative to the
other constructor overload that accepts keyword arguments. The name of
the new args class is the name of the resource concatenated with an
`Args` suffix.
Some providers (e.g. Kubernetes, Azure Native, and Google Native) have
input types with the same name as resources in the same module, which
results in two different `<Resource>Args` classes in the same module.
When you try to use the new args class with the constructor, e.g.:
```python
pulumi_kubernetes.storage.v1.StorageClass(
resource_name='string',
args=pulumi_kubernetes.storage.v1.StorageClassArgs(...),
opts=pulumi.ResourceOptions(...),
)
```
You run into an error, because
`pulumi_kubernetes.storage.v1.StorageClassArgs` is actually referring to
the existing input type rather than the intended `StorageClassArgs`
class for the constructor arguments.
Having the duplicate classes hasn't broken existing usage of the input
type because we "export" all the input types for a module _after_ all
the resources and resource args classes are exported, so the input type
just ends up "overwriting" the duplicate resource args class.
Other languages don't have this problem because the input type is either
in it's own module/namespace (e.g. Node.js and .NET) or a different name
is used for the input type (Go). But with Python, the input types and
resources are all available in the same module.
To address this for Python, when there is an input type in the same
module with the same name as the resource, the args class for the
resource will be emitted as `<Resource>InitArgs` instead of
`<Resource>Args`.
This commit modifies the work in #7058 to permit properties which do not
pass the test of being strings directly, but which have an underlying
type of string.
When applied to `pulumi-aws`, this results in the following diff:
```
diff --git a/sdk/go/aws/provider.go b/sdk/go/aws/provider.go
index c32ad2367..8b4c9fd0a 100644
--- a/sdk/go/aws/provider.go
+++ b/sdk/go/aws/provider.go
@@ -21,6 +21,8 @@ type Provider struct {
AccessKey pulumi.StringPtrOutput `pulumi:"accessKey"`
// The profile for API operations. If not set, the default profile created with `aws configure` will be used.
Profile pulumi.StringPtrOutput `pulumi:"profile"`
+ // The region where AWS operations will take place. Examples are us-east-1, us-west-2, etc.
+ Region pulumi.StringPtrOutput `pulumi:"region"`
// The secret key for API operations. You can retrieve this from the 'Security & Credentials' section of the AWS console.
SecretKey pulumi.StringPtrOutput `pulumi:"secretKey"`
// The path to the shared credentials file. If not set this defaults to ~/.aws/credentials.
```
The primary purpose this is desirable is to expose Region from instances
of the AWS provider.
* Add get value async to output utilities
asdasd
* Update changelog
* Move public apis from unshipped to shipped
Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
Fix the generated C# code for plain properties:
- Value types should not be initialized with `= null!`
- Arrays and maps should be `List<T>` and `Dictionary<string, TValue>`
These mutually recursive functions unintentionally had exponential complexity in nesting depth of objects, arg types and most likely arrays.
Remove the exponential complexity by avoiding direct recursion of from_input on itself, and relying on mutual recursion with all alone to reduce nested substructure.
Also simplify the implementation to aid readability.
Fixes pulumi/pulumi-kubernetes#1597.
Fixes pulumi/pulumi-kubernetes#1425.
Fixes pulumi/pulumi-kubernetes#1372.
Fixes#3987.
* Send plugin install output to stderr
We currently send plugin install output to stdout. This interferes
with --json (#5747), automation API scenarios, and in general is bad
CLI hygiene. This change sends plugin output to stdout instead.
* Add a changelog entry
Temporarily disable the new config secret warning to avoid unactionable warnings from provider `config` modules. We'll re-enable the warning when we've addressed that issue.
This commit adds a fallback for the README definition in the generated
setup.py files for Python SDKs, thus allowing editable installs of
packages which not yet been built.
Co-authored-by: Luke Hoban <luke@pulumi.com>
