* Propagate workspace.Project metadata to plugin init
* Get to a working fix
* Propagate Root via plugin context
* Propagate root instead of yaml path
* Revert out unnecessary parameter propagation
* Root is now always absolute at this point; simplify code and docs
* Drop python conditional and propagate unused -root to all lang hosts
* Add tests that fail before and pass after
* Lint
* Add changelog entry
Fixes: #6286
When a user is using the passphrase provider and unsets the environment
variables that let them interact with the secrets provider, then would
get an error as follows:
```
▶ pulumi up -y -f
error: decrypting secret value: failed to decrypt: incorrect passphrase, please set PULUMI_CONFIG_PASSPHRASE to the correct passphrase
```
We are oging to change this error message to make it more obvious
to a user what the error is and how they need to fix it
```
▶ pulumi up -y -f
error: constructing secrets manager of type "passphrase": unable to find either `PULUMI_CONFIG_PASSPHRASE` nor `PULUMI_CONFIG_PASSPHRASE_FILE` when trying to access the Passphrase Secrets Manager. Please ensure one of these values are set to allow the operation to continue
```
Ideally, we would like to prompt the user for the passphrase at this
point rather than error, but the CLI could be in the middle of an
update operation as the same codepath is used for reading stackreference
values
This change addresses Python dictionary key translation issues. When the
type of `props` passed to the resource is decorated with `@input_type`,
the type's and resource's property name metadata will be used for dict
key translations instead of the resource's `translate_input_property`
and `translate_output_property` methods.
The generated provider SDKs will be updated to opt-in to this new
behavior:
- FIX: Keys in user-defined dicts will no longer be unintentionally
translated/modified.
- BREAKING: Dictionary keys in nested output classes are now
consistently snake_case. If accessing camelCase keys from such output
classes, move to accessing the values via the snake_case property
getters (or snake_case keys). Generated SDKs will log a warning
when accessing camelCase keys.
When serializing inputs:
- If a value is a dict and the associated type is an input type, the
dict's keys will be translated based on the input type's property
name metadata.
- If a value is a dict and the associated type is a dict (or Mapping),
the dict's keys will _not_ be translated.
When resolving outputs:
- If a value is a dict and the associated type is an output type, the
dict's keys will be translated based on the output type's property
name metadata.
- If a value is a dict and the associated type is a dict (or Mapping),
the dict's keys will _not_ be translated.
This change fixes the provider implementation of `Construct` for multi-lang components written in Node.js to wait for any in-flight RPCs to finish before returning the results, s.t. all registered child resources are created.
In additional, invocations of `construct` are now serialized so that each call runs one after another, avoiding concurrent runs, since `construct` modifies global state. We'll follow-up with a more general concurrency fix to allow nested `construct` calls within the same provider.
Add support for creating instances of resources in Python using a
`<Resource>Args` class. This capability aligns with how args are passed
to resources in all the other language SDKs and the separate object bag
allows the properties to be manipulated/validated/passed-around before
creating the resource.
* Avoid double-quailfying venv folder path
* Replace `path` with `filepath`
* Add a Python integration test to cover venv auto-creation
* Merged
* Fix spelling
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
* Make AbsPath and RelPath test variants
* Fix issue on Windows backslash paths
* Debug windows test failure: more logging and aggressive YAML escaping
* Use filepath.IsAbs instead of path.IsAbs
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
When using the filestate backend (local files and cloud buckets) there is no protection to prevent two processes from managing the same stack simultaneously.
This PR creates a locks directory in the management directory that stores lock files for a stack. Each backend implementation gets its own UUID that is joined with the stack name. The feature is currently available behind the `PULUMI_SELF_MANAGED_STATE_LOCKING=1` environment variable flag.
When passing a package source as part of a `dotnet add package` in
our acceptance testing framework, dotnet was then trying to use that
package source for the restoration of other packages in the csproj
file.
We have removed passing the source to dotnet add package add
and replaced it with adding a machine level package source via
dotnet nuget add source command
this is the more correct way to work and will allow us to be able
to search multiple locations as part of the dotnet restore command
This way, the tests use the built-in virtual environment support by
default, which is what most customers will be using. A new `UsePipenv`
option is available to go back to using pipenv for tests.
We previously looked for `python3` and fallback to `python` on all systems. However, our Windows CI images include a `python3.exe` symlink to `python.exe` which does not work with `venv`. So on Windows, just look for `python` first, falling back to `python3`. (The default python.org Windows installation only includes `python.exe`).
Update Python tests to use `UseAutomaticVirtualEnv` and set `PythonBin` to `python` on Windows because `python3` doesn't work correctly in the GitHub Action runners.
Implement GetRequiredPlugins for Python, which determines the plugins
required by the program.
Also, if the `virtualenv` runtime option is set, and the specified
virtual directory is missing or empty, automatically create it and
install dependencies into it.
* Correctly rename stack files during a rename
This fixespulumi/pulumi#4463, by renaming a stack's configuration
file based on its stack-part, and ignoring the owner-part. Our
workspace system doesn't recognize configuration files with fully
qualified names. That, by the way, causes problems if we have
multiple stacks in different organizations that share a stack-part.
The fix here is simple: propagate the new StackReference from the
Rename operation and rely on the backend's normalization to a
simple name, and then use that the same way we are using a
StackReference to determine the path for the origin stack.
An alternative fix is to recognize fully qualified config files,
however, there's a fair bit of cleanup we will be doing as part of
https://github.com/pulumi/pulumi/issues/2522 and
https://github.com/pulumi/pulumi/issues/4605, so figured it is best
to make this work the way the system expects first, and revisit it
as part of those overall workstreams. I also suspect we may want to
consider changing the default behavior here as part of
https://github.com/pulumi/pulumi/issues/5731.
Tests TBD; need some advice on how best to test this since it
only happens with our HTTP state backend -- all integration tests
appear to use the local filestate backend at the moment.
* Add a changelog entry for bug fix
* Add some stack rename tests
* Fix a typo
* Address CR feedback
* Make some logic clearer
Use "parsedName" instead of "qn", add a comment explaining why
we're doing this, and also explicitly ignore the error rather
than implicitly doing so with _.
The PULUMI_BACKEND_URL env var allows specifying the backend to use instead of deferring to the project or the ~/.pulumi/credentials.json file to decide on the "current" backend. This allows for using Pulumi without a dependence on this piece of global filesystem state, so that each `pulumi` invocation can control the exact backend it want's to operate on, without having to do stateful `pulumi login`/`pulumi logout` operations.
This is especially useful for automation scenarios like Automation API generally (and effectively solves https://github.com/pulumi/pulumi/issues/5591), or https://github.com/pulumi/pulumi-kubernetes-operator/issues/83 specifically.
This also makes things like efe7a599e6/dist/actions/entrypoint.sh (L10) less necessary, and possible to accomplish for any containerized `pulumi` execution without the need for this logic to be embedded in bash scripts wrapping the CLI.
Just what it says on the tin.
The SDK code generator will be updated to use the new `urn`
resource option inside of each module's implementation of
`ResourceModule.construct`.
Part of #2430.
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Resources are serialized as their URN, ID, and package version. Each
Pulumi package is expected to register itself with the SDK. The package
will be invoked to construct appropriate instances of rehydrated
resources. Packages are distinguished by their name and their version.
This is the foundation of cross-process resources.
Related to #2430.
Co-authored-by: Mikhail Shilkov <github@mikhail.io>
Co-authored-by: Luke Hoban <luke@pulumi.com>
Co-authored-by: Levi Blackstone <levi@pulumi.com>
A recent change to output deserialization resulted in secrets being returned unwrapped. This change addresses the regression, ensuring any unwrapped secret values are rewrapped before being returned.
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.
Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.
The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.
These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).
An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.
This is the core of #2430.
We make several calls to `os/user`, which uses CGO and means
cross-compilation is not possible. This replaces `os/user` with the
`luser` package, which is a drop-in replacement which does not use `CGO`
In particular, use the parser to filter and extract examples. This also
sets up support for entity references in documentation that can be used
in order to render language-specific names for resources, functions,
types, and properties.
Related to #4632 and #4159.
Pylint currently reports `E1101: Instance of 'Bucket' has no 'id' member (no-member)` on lines in Pulumi Python programs like:
```python
pulumi.export('bucket_name', bucket.id)
```
Here's a description of this message from http://pylint-messages.wikidot.com/messages:e1101:
> Used when an object (variable, function, …) is accessed for a non-existent member.
>
> False positives: This message may report object members that are created dynamically, but exist at the time they are accessed.
This appears to be a false positive case: `id` isn't set in the constructor (it's set later in `register_resource`) and Pylint isn't able to figure this out statically. `urn` has the same problem. (Oddly, Pylint doesn't complain when accessing other resource output properties).
This change refactors `register_resource` so that `id` and `urn` can be assigned in the resource's constructor, so that Pylint can see it being assigned. The change also does the same with `read_resource`.
When running `pulumi up`, after the preview, we prompt asking whether to proceed with the update. If you type to filter the options and then hit an arrow key a couple times, the CLI panics. This is a bug in the `survey` library we depend on. The issue has been fixed in the library upstream; this change updates our dependency.
Automatically create a virtual environment and install dependencies in it with `pulumi new` and `pulumi policy new` for Python templates.
This will save a new `virtualenv` runtime option in `Pulumi.yaml` (`PulumiPolicy.yaml` for policy packs):
```yaml
runtime:
name: python
options:
virtualenv: venv
```
`virtualenv` is the path to a virtual environment that Pulumi will use when running `python` commands.
Existing projects are unaffected and can opt-in to using this by setting `virtualenv`, otherwise, they'll continue to work as-is.
This class was available in the pulumi.resource module, but was not exported from the core `pulumi` module as intended for all public APIs at this level.
- Move the implementation of loadPackageSchema into a method on
PackageCache
- Protect the cache with synchronization primitives to enable
concurrency in downstream consumers
- Use jsoniter to deserialize schemas
# This is the 1st commit message:
Use patched azure gocloud library
Fixes#4642
# The commit message #2 will be skipped:
# sync with pulumi-master branch
* Enable container runtime tests for cron jobs
* Use VERSION var from Makefile
Co-authored-by: Paul Stack <public@paulstack.co.uk>
Co-authored-by: Paul Stack <public@paulstack.co.uk>
Some of the apply rewriter's assumptions were broken by the richer
expressions available in HCL2. These changes fix those broken
assumptions, in particular the assumption that only scope traversal
expressions are sources of eventual values.
* Ensure yes is passed when creating a new template in a container
I fixed this locally when creating the new docker containers so it's
time to push this back to master
* Update tests/containers/containers_test.go
Co-Authored-By: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
When setting structured config values using `--path`, we automatically
treat values that can be converted into an integer via `strconv.Atoi` as
an integer, rather than as a string.
However, this ends up converting values like "0123456" into the integer
123456, stripping the leading 0, which isn't desirable for values like
commit SHAs, etc., where you want to keep the 0 (and keep it a string).
This change makes it so that values starting with 0 are not implicitly
converted to an integer; instead such values will remain a string.
* Make `async:true` the default for `invoke` calls (#3750)
* Switch away from native grpc impl. (#3728)
* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)
* Only retry as long as we get unavailable back. Anything else continues. (#3769)
* Handle all errors for now. (#3781)
* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)
* Upgrade all paths for sdk and pkg to v2
* Backport C# invoke classes and other recent gen changes (#4288)
Adjust C# generation
* Replace IDeployment with a sealed class (#4318)
Replace IDeployment with a sealed class
* .NET: default to args subtype rather than Args.Empty (#4320)
* Adding system namespace for Dotnet code gen
This is required for using Obsolute attributes for deprecations
```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```
* Fix the nullability of config type properties in C# codegen (#4379)
These were causing *very* old versions of `@pulumi/pulumi` to get pulled in, and were seeing failures on Node 13 due to missnig gRPC native modules.
Aside - we've tried to not have any dependencies in `@pulumi/pulumi` on `@pulumi/aws` and other higher-level libraries, to avoid layering violation issues. Would love to see if we can reasonably simplify the testing at this layer to not have this dependency.
* Make Python StackReference test similar to others (with two steps)
* Include new Python StackReference integration test that uses multiple stacks
* Expose various life cycle methods for ProgramTester
