Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.
Fixes#6753.
Co-authored-by: Levi Blackstone <levi@pulumi.com>
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
* Do not hang but propagate exception when it happens in resolve_outputs
* Add an integration test for the issue
* Better error message
* Add CHANGELOG_PENDING entry
* Update sdk/python/lib/pulumi/runtime/rpc.py
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Address PR feedback and tighten path param typing
* Given Windows builder is failing, allow 2x time for the test
* Give some more time to the Windows runner
* Attempt to solve differently
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Although `Output` objects can never correct support iteration, Python will see the implementation of `__getitem__` and try to iterate the object, leading to an infinite loop. To prevent this, we need to explicitly implement `__iter__` and make it return a `TypeError` to prevent iteration (and offer a useful error message).
Fixes#5028.
`log.debug` messages are part of an update and get sent to the service. Reducing the number of `log.debug` calls can significantly improve the performance of a program. This commit changes the debug logging in the Python SDK to be similar to the debug logging in the Node.js SDK.
- An `excessive_debug_output` variable has been added (that can be set to `True` during debugging), but otherwise will avoid a large number of `log.debug` calls.
- Some unhelpful/redundant `log.debug` calls have been deleted.
These mutually recursive functions unintentionally had exponential complexity in nesting depth of objects, arg types and most likely arrays.
Remove the exponential complexity by avoiding direct recursion of from_input on itself, and relying on mutual recursion with all alone to reduce nested substructure.
Also simplify the implementation to aid readability.
Fixes pulumi/pulumi-kubernetes#1597.
Fixes pulumi/pulumi-kubernetes#1425.
Fixes pulumi/pulumi-kubernetes#1372.
Fixes#3987.
Temporarily disable the new config secret warning to avoid unactionable warnings from provider `config` modules. We'll re-enable the warning when we've addressed that issue.
* Propagate workspace.Project metadata to plugin init
* Get to a working fix
* Propagate Root via plugin context
* Propagate root instead of yaml path
* Revert out unnecessary parameter propagation
* Root is now always absolute at this point; simplify code and docs
* Drop python conditional and propagate unused -root to all lang hosts
* Add tests that fail before and pass after
* Lint
* Add changelog entry
Pulumi 3.0 raises an error when a dict value is passed as an input but the type annotation does not accept a dict. Unfortunately, this prevents historical cases where a dict value is allowed but the type annotation doesn't match. We need to fix the type annotations, but in the meantime, we should not raise an error in the SDK for such cases as it breaks existing programs.
Pulumi 3.0 uses type annotations for input values to determine whether dict keys should be translated from snake_case to camelCase or not. This additional inspection of types did not work correctly on Python 3.6 due to some missing functionality on that version of Python which we need to provide an implementation for. Specifically, when inspecting `Union` args to determine whether or not a value is intended to be an input class or user-defined dict. To address the issue, this change improves how we get the args for `Union` types when running on Python 3.6 to behave the same way as later versions of Python (for our purposes). Existing tests fail on Python 3.6 before this change, and pass after.
We were only looking at the current resource class's type/name metadata for camelCase <=> snake_case property name translations which prevented it from working correctly when using a subclass of a resource. This change addresses this by looking at metadata of the current class and any base classes.
Additionally, to help resolve forward references when getting type hints, we'd pass along the current resource class's globals, which doesn't work correctly when using a subclass of a resource. This change also addresses this, by using the globals of the current class and any base classes.
See #6200 for a complete description of the issue. In short, we generate
inconsistent names for object types depending on whether or not they are
transitively reachable from resources or functions, which risks
unintentional breaking changes due to schema updates.
1. Name "input" types differently: `TArgs` for a type that is used in
resource inputs, having `Input<T>` properties, and `T` for a type
that is used in invoke inputs. The same schema type can produce both.
2. Always keep the name `T` for output types, avoid appending `Result` to
the name.
3. As needed, introduce a flag in the existing providers' schemas to avoid
breaking changes. Consider removing it on a major version bump.
Fixes#6200.
This change addresses Python dictionary key translation issues. When the
type of `props` passed to the resource is decorated with `@input_type`,
the type's and resource's property name metadata will be used for dict
key translations instead of the resource's `translate_input_property`
and `translate_output_property` methods.
The generated provider SDKs will be updated to opt-in to this new
behavior:
- FIX: Keys in user-defined dicts will no longer be unintentionally
translated/modified.
- BREAKING: Dictionary keys in nested output classes are now
consistently snake_case. If accessing camelCase keys from such output
classes, move to accessing the values via the snake_case property
getters (or snake_case keys). Generated SDKs will log a warning
when accessing camelCase keys.
When serializing inputs:
- If a value is a dict and the associated type is an input type, the
dict's keys will be translated based on the input type's property
name metadata.
- If a value is a dict and the associated type is a dict (or Mapping),
the dict's keys will _not_ be translated.
When resolving outputs:
- If a value is a dict and the associated type is an output type, the
dict's keys will be translated based on the output type's property
name metadata.
- If a value is a dict and the associated type is a dict (or Mapping),
the dict's keys will _not_ be translated.
The current logic lets unhandled errors in the RPC invocation
unahandled in the async loop, which crashes the process due to
the way we await completion of RPCs before exiting the process
in Python. Instead of doing that, we can just marshal them back
to the synchronous awaiter as part of the calling convention,
and have that awaiter (which is called by the invoke methods)
re-raise the exception. This should fixpulumi/pulumi#3611.
* Avoid double-quailfying venv folder path
* Replace `path` with `filepath`
* Add a Python integration test to cover venv auto-creation
* Merged
* Fix spelling
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Make AbsPath and RelPath test variants
* Fix issue on Windows backslash paths
* Debug windows test failure: more logging and aggressive YAML escaping
* Use filepath.IsAbs instead of path.IsAbs
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
The Project and Stack save routines were erroneously
dumping the Python objects rather than the __dict__
property, which resulted in some extra annotations
in the resulting YAML files. Some parsers don't handle
these annotations correctly, and consider the resulting
YAML file to be invalid.
Fixes:#6565
As part of #6460, the logic for determing the version of the build was
moved to be a dependency on pulumictl.
Unfortunately, the homebrew installs use the "make dist" command to
build + install Pulumi to the user maching and as that would have a
dependency on pulumictl and it not existing on the user machine, it
would pass an empty version to the ldflag
This then manifested to the user as:
```
▶ pulumi version
warning: A new version of Pulumi is available. To upgrade from version '0.0.0' to '2.22.0', run
$ brew upgrade pulumi
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
```
We are able to mitigate this behaviour by bringing back the get-version
script and using that script as part of the make brew installation
We can see that the versions are the same between the 2 different
installation techniques
```
make dist <------- uses pulumict
DIST:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=2.24.0-alpha.1616029310+787eb70a" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
DIST:
BUILD:
```
```
make brew <----- uses the legacy script
▶ make brew
BREW:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=v2.24.0-alpha.1616029310+g787eb70a2" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
BREW:
```
A full post mortem will be carried out to ensure we mitigate these
types of errors going forward and that we are able to better test
these types of situations
dotnet, nodejs and python automation APIs did not specify exec-kind for
refresh or destroy operations. This is now added following the same
logic from the go automation API.
This change avoids `RuntimeError: There is no current event loop in thread '<thread_name>'` errors when passing a resource as an input multiple times when using mocks.
The problem is that when using mocks, we deserialize the gRPC inputs before passing them to the user's mock methods. Deserializing inputs doesn't typically require an event loop, however, during deserialization of resource references, we end up creating some instances of `Future`, which does require an event loop to be present for the current thread. If this is done multiple times for a resource, it's possible that `deserialize_properties` will be called on an asyncio thread that doesn't yet have an event loop, resulting in the error being raised.
The error does not occur when only passing the resource reference once because typically the thread (e.g. `asyncio_0`) used in that case will have already had an event loop created for it due to the use of the internal `_syncawait` when _serializing_ the source resource's properties, which ensures an event loop is set for the thread.
The fix is to ensure an event loop is created for the thread in the mocks implementation before calling `deserialize_properties`.
We previously looked for `python3` and fallback to `python` on all systems. However, our Windows CI images include a `python3.exe` symlink to `python.exe` which does not work with `venv`. So on Windows, just look for `python` first, falling back to `python3`. (The default python.org Windows installation only includes `python.exe`).
Adds a `--limit` flag to `pulumi stack history. This allows limiting to the last few entries rather than fetching the entirety of a stack's update history (which can be quite slow for stacks with lots of updates). Example: `pulumi stack history --limit 1` fetches the last history entry only.
`stack.up` and related operations in the Automation API have been updated to consume this change, drastically reducing overhead.
`Output.from_input` deeply unwraps nested output values in dicts and lists, but doesn't currently do that for the more recently added "input types" (i.e. args classes). This leads to errors when using args classes with output values with `Provider` resources, which uses `Output.from_input` on each input property and then serializes the value to JSON in an `apply`. This changes fixes `Output.from_input` to recurse into values within args classes to properly unwrap any nested outputs.
