We have seen cases where a lot of errors like this are reported:
```
UnboundLocalError: local variable 'resources' referenced before assignment
```
This change prevents this failure mode, which might be a symptom of some
other issue, but currently obscures it in the error path.
* .NET & python SDKs parity for bad pulumi versions
They handle invalid Pulumi CLI version gracefully.
* Make python version property lazy
* Clarify .NET logic
* Add python test for validate_pulumi_version
* Add tests for invalid versions
* Fix python test
* Fix typo
* Fix tests
* Have _validate_pulumi_version handle parsing
* Modify python and .NET to parseAndValidate
* Modify typescript and go to parseAndValidate
* fix name
* Enable output values by default
Enable output values by default in the resource monitor and change the polarity of the envvar from `PULUMI_ENABLE_OUTPUT_VALUES` to `PULUMI_DISABLE_OUTPUT_VALUES`.
* Marshal unknown as unknown string when `!KeepOutputValues`
Marshal all unknown output values as `resource.MakeComputed(resource.NewStringProperty(""))` when not keeping output values, which is consistent with what the SDKs do.
Otherwise, when `v.OutputValue().Element` is nil, `resource.MakeComputed(v.OutputValue().Element)` will be marshaled as a null value rather than as an unknown sentinel.
* Add MarshalOptions.DontSkipOutputs and use where needed
Before we expanded the meaning of `resource.Output`, `MarshalProperties` always skipped output values:
```go
if v.IsOutput() {
logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```
As part of expanding the meaning of `resource.Output`, I'd adjusted `MarshalProperties` to only skip output values when the value was unknown and when not keeping output values:
```go
if v.IsOutput() && !v.OutputValue().Known && !opts.KeepOutputValues {
logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```
However, this doesn't work the way we want when marshaling properties that include unknown output values to a provider that does not accept outputs. In that case, `opts.KeepOutputValues` will be `false` because we want the marshaler to fall back to returning non-output-values (e.g. unknown sentinel value for unknown output values), but instead of getting the intended fallback values, the unknown output values are skipped (not what we want).
I suspect we may be able to delete the output value skipping in `MarshalProperties` altogether (it's odd that it is skipping `resource.Output` but not `resource.Computed`), but to avoid any unintended side effects of doing that, instead, this commit introduces a new `MarshalOptions.DontSkipOutputs` option that can be set to `true` to opt-in to not skipping output values when marshaling. The check in `MarshalProperties` now looks like this:
```go
if !opts.DontSkipOutputs && v.IsOutput() && !v.OutputValue().Known {
logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```
`opts.DontSkipOutputs` is set to `true` when marshaling properties for calls to a provider's `Construct` and `Call`.
* [sdk/nodejs] Deserialize output values
This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK.
* [sdk/python] Deserialize output values
This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK.
v3.13.0 introduces support for serializing outputs in inputs as special output value objects in the Node.js and Python SDKs when serializing inputs for remote components and method calls. This functionality is currently disabled by default in the engine (setting the `PULUMI_ENABLE_OUTPUT_VALUES` envvar to a truthy value enables it).
However, unit testing remote components with mocks results in errors being raised in v3.13.0, related to the new output value support. This is due to the mock monitor implementation saying it supports all features (which now includes output values), so the SDK serializers are serializing outputs as output values, which the mock monitor can't handle correctly.
This change addresses the issue by updating the mock monitor implementation in the Node.js and Python SDKs to indicate the specific features that are supported, excluding support for output values. New tests with mocks fail before the change and pass after.
This reverts commit ebb0e6aaed.
The changes in #7755 introduced a regression tracked in #7795. It is not yet clear how to retain the desired behaviour introduced in #7755 while avoiding this regression, so for now we will revert those changes, and re-open #7453 to track a deeper fix. That fix may require making changes to upstream `dill` to properly support these serialization requirements.
Fixes#7795.
* Factor out URN parsing in Python
* More code sharing to satisfy lint dup code detector
* Fix lint R1735 recommendation of empty dict literal
* Fix lint issue insisting on known encoding during open()
* Fix parsing URNs without urn_name; test; exception wrapper
Now there is not possible to change a name of dynamic provider resource without copying a code of the `pulumi.sdk.python.lib.pulumi.dynamic.dynamic.Resource` and changing the hard-coded name `"pulumi-python:dynamic:Resource"`. I successfully uses this proposal to make it possible.
Usage:
```python
class CustomResource(
Resource, name="my-custom-provider:CustomResource"
):
...
```
Co-authored-by: Pat Gavlin <pgavlin@gmail.com>
The underlying library `dill` that we use for serializing dynamic providers into Pulumi state for Python dynamic providers serializes classes differently depending on whether they are in `__main__` or in another module. We need the by-value serialization to be applied in all cases.
https://github.com/uqfoundation/dill/issues/424 is tracking adding the ability into `dill` to specify this by-value serialization explicitly, but until then, we will temporarily re-write the `__module__` of a provder class prior to serialization, so that `dill` behaves as we need for the dynamic provider use case.
Fixes#7453.
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.
Part of #7542.
Similar to #5282, but for core SDK types. The tl;dr is that because
Sequence[T] is covariant, constructing resources becomes much more
ergonomic.
Fix#7693.
* Inputty depends_on support for Python
* Fix circular import
* Add a test, fix bugs found by test
* Fix typo
* Minimize loss of dep information in presence of unks to match Node behavior
* Add CHANGELOG entr
* Update sdk/python/lib/pulumi/output.py
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Update sdk/python/lib/test/test_resource.py
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Fix _is_prompt
* Enhance tests, found out a gap against Node impl
* Add non-listy case
* Internal functional combinators
* Do not use all/deep await when merging
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
When initializing `DependencyProviderResource`, pass the package to the base constructor instead of an empty string s.t. the provider is usable when its package is read.
Previously, any provider resource passed to multi-lang components would be instantiated as a `DependencyProviderResource` inside `Construct`, which prevents the component from being able to easily access the provider's state as an instance of of the provider (e.g. `*aws.Provider`).
This change attempts to rehydrate the provider resource in the same way that resource references are rehydrated, if it's been registered, s.t. the specific provider resource type is instantiated with its state. Otherwise falling back to returning `DependencyProviderResource`.
In short, this allows subtyping to correctly "propagate" through an Output;
if Cow is a subtype of Animal, this commit makes it so Output[Cow] is
treated as a subtype of Output[Animal].
Without this change, users of the Python SDK occasionally contend with a
confusing error message that is resolved by adding a call to
`.apply(lambda x: x)` to satisfy mypy.
Touches #3767.
Fix#6843.
The map of property dependencies the Python SDK receives from the
engine uses `camelCase` property keys, but the SDK performs lookups
using `snake_case` property keys. Fix this by translating the map's keys
from `camelCase` to `snake_case` prior to performing any lookups.
Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.
Fixes#6753.
Co-authored-by: Levi Blackstone <levi@pulumi.com>
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
* Do not hang but propagate exception when it happens in resolve_outputs
* Add an integration test for the issue
* Better error message
* Add CHANGELOG_PENDING entry
* Update sdk/python/lib/pulumi/runtime/rpc.py
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Address PR feedback and tighten path param typing
* Given Windows builder is failing, allow 2x time for the test
* Give some more time to the Windows runner
* Attempt to solve differently
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Although `Output` objects can never correct support iteration, Python will see the implementation of `__getitem__` and try to iterate the object, leading to an infinite loop. To prevent this, we need to explicitly implement `__iter__` and make it return a `TypeError` to prevent iteration (and offer a useful error message).
Fixes#5028.
`log.debug` messages are part of an update and get sent to the service. Reducing the number of `log.debug` calls can significantly improve the performance of a program. This commit changes the debug logging in the Python SDK to be similar to the debug logging in the Node.js SDK.
- An `excessive_debug_output` variable has been added (that can be set to `True` during debugging), but otherwise will avoid a large number of `log.debug` calls.
- Some unhelpful/redundant `log.debug` calls have been deleted.
These mutually recursive functions unintentionally had exponential complexity in nesting depth of objects, arg types and most likely arrays.
Remove the exponential complexity by avoiding direct recursion of from_input on itself, and relying on mutual recursion with all alone to reduce nested substructure.
Also simplify the implementation to aid readability.
Fixes pulumi/pulumi-kubernetes#1597.
Fixes pulumi/pulumi-kubernetes#1425.
Fixes pulumi/pulumi-kubernetes#1372.
Fixes#3987.
Temporarily disable the new config secret warning to avoid unactionable warnings from provider `config` modules. We'll re-enable the warning when we've addressed that issue.
