The logic for validating prompted values in 'new' wasn't quite right,
leading to the possibility of creating Pulumi.yaml files with blank
project names.
This manifests in various ways and I've hit it a number of times
over the past few months because of the way we handle project/stack
name conflicts in 'new' -- which itself is a bit annoying too:
https://github.com/pulumi/pulumi/blob/master/pkg/cmd/pulumi/new.go#L206-L207
Because we substitue a default value of "", and because the prompting
logic assumed default values are always valid, we would skip validation
and therefore accept a blank Pulumi.yaml file.
This generates an invalid project which causes errors elsewhere, such as
error: failed to load Pulumi project located at ".../Pulumi.yaml":
project is missing a 'name' attribute
I hit this all the time with our getting started guide because I've
gone through it so many times and have leftover stacks from prior
run-throughs. I wouldn't be surprised if a lot of people hit this.
The solution here validates all values, including the default.
Note also that we failed to validate the value used by 'new --yes'
which meant you could bypass all validation by passing --yes, leading
to similar outcomes.
I've added a couple new tests for these cases. There is a risk we
depend on illegal default values somewhere which will now be rejected,
but that would seem strange, and assuming the tests pass, I would
assume that's not true. Let me know if that's wrong.
Fixespulumi/pulumi#3255.
A recent change to output deserialization resulted in secrets being returned unwrapped. This change addresses the regression, ensuring any unwrapped secret values are rewrapped before being returned.
We recently made a change to the Python codegen to emit `int` type annotations, instead of `float`, for properties that are typed as `schema.IntType`.
But the number values that come back from protobuf structs are always floats (like JSON), so we need to cast the values intended to be integers to `int`.
* Revise host mode.
The current implementation of host mode uses a `pulumi host` command and
an ad-hoc communication protocol between the engine and client to
connect a language host after the host has begun listening. The most
significant disadvantages of this approach are the communication
protocol (which currently requires the use of stdout), the host-specific
command, and the difficulty of accommodating the typical program-bound
lifetime for an update.
These changes reimplement host mode by adding engine support for
connecting to an existing language runtime service rather than launching
a plugin. This capability is provided via an engine-specific language
runtime, `client`, which accepts the address of the existing languge
runtime service as a runtime option. The CLI exposes this runtime via
the `--client` flag to the `up` and `preview` commands, which similarly
accepts the address of an existing language runtime service as an
argument. These changes also adjust the automation API to consume the
new host mode implementation.
When installing a plugin, if it contains a `PulumiPlugin.yaml` file with a `runtime` value of `nodejs` or `python`, install dependencies for the plugin.
For Node.js, `npm install` is run (or `yarn install` if `PULUMI_PREFER_YARN` is set).
For Python, a virtual environment is created and deps installed into it.
* add initial pull-request workflow
* run SDK test all
* add SDK tests
* fixup make targets
* add dist target
* revert back to 5 updates
* disable test
* add issue for test disabling
Non-string provider inputs must be projected as JSON formatted strings. The current codegen simply calls `json.dumps` for such properties, but this does not work for the new input types, which aren't JSON serializable.
This commit adds a new `pulumi.runtime.to_json` utility function to the core SDK, which is capable of serializing both raw dicts and input types as JSON. The codegen will be updated to make use of this new function rather than `json.dumps`.
We currently emit array types as `List[T]` for Python, but `List[T]` is invariant, which causes type checkers like mypy to produce errors when values like `["foo", "bar"]` are passed as args typed as `List[pulumi.Input[str]]` (since `Input[str]` is an alias for `Union[T, Awaitable[T], Output[T]]`. To address this, we should move to using [`Sequence[T]`](https://docs.python.org/3/library/typing.html#typing.Sequence) which is covariant, and does not have this problem.
We actually already do this for `Dict` vs. `Mapping`, emitting map types as `Mapping[str, T]` rather than `Dict[str, T]` because `Mapping[str, T]` is covariant for the value. This change makes us consistent for array types.
These are the SDK changes necessary to support `Sequence[T]`.
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.
Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.
The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.
These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).
An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.
This is the core of #2430.
We make several calls to `os/user`, which uses CGO and means
cross-compilation is not possible. This replaces `os/user` with the
`luser` package, which is a drop-in replacement which does not use `CGO`
Update pip, setuptools, and wheel in the virtual environment before installing dependencies as recommended by the Python documentation. This should help avoid failures when only source distributions are available for a package and pip attempts to build a wheel locally.
Certain operations in `engine/diff` mutate engine events during display.
This mutation can occur concurrently with the serialization of the event
for persistence, which causes a panic in the CLI. These changes fix the
offending code and add code that copies each engine event before
persisteing it in order to guard against future issues.
Previously, streamInvoke was only supported by
the query command. Copied the implementation
into the resource monitor, which will allow
streaming invoke commands to run during updates.
Also fixed a bug with cancellation of streaming
invokes. The check was comparing against a
hardcoded string, which did not match the actual
error string. Instead, we can rely on the error code.
This function adds a property value to another property at a given path,
creating containing properties as required. If the property cannot be
added because of a mismatch between the value types required by the path
and the values present in the destination, the add will fail. If a value
already exists at the given path, the add will succeed.
Related to #1635.
Several users reported cases where error messages would
cause a panic if they contained accented characters. I wasn't
able to reproduce this failure locally, but tracked down the
panic to logging gRPC calls. The Message field is typed as
a string, which requires all of the characters to be valid UTF-8.
This change runs each log string through the strings.ToValidUTF8
function, which will replace any invalid characters with the
"unknown" character. This should prevent the the logger from
panicking.
Pylint currently reports `E1101: Instance of 'Bucket' has no 'id' member (no-member)` on lines in Pulumi Python programs like:
```python
pulumi.export('bucket_name', bucket.id)
```
Here's a description of this message from http://pylint-messages.wikidot.com/messages:e1101:
> Used when an object (variable, function, …) is accessed for a non-existent member.
>
> False positives: This message may report object members that are created dynamically, but exist at the time they are accessed.
This appears to be a false positive case: `id` isn't set in the constructor (it's set later in `register_resource`) and Pylint isn't able to figure this out statically. `urn` has the same problem. (Oddly, Pylint doesn't complain when accessing other resource output properties).
This change refactors `register_resource` so that `id` and `urn` can be assigned in the resource's constructor, so that Pylint can see it being assigned. The change also does the same with `read_resource`.
Automatically create a virtual environment and install dependencies in it with `pulumi new` and `pulumi policy new` for Python templates.
This will save a new `virtualenv` runtime option in `Pulumi.yaml` (`PulumiPolicy.yaml` for policy packs):
```yaml
runtime:
name: python
options:
virtualenv: venv
```
`virtualenv` is the path to a virtual environment that Pulumi will use when running `python` commands.
Existing projects are unaffected and can opt-in to using this by setting `virtualenv`, otherwise, they'll continue to work as-is.
This class was available in the pulumi.resource module, but was not exported from the core `pulumi` module as intended for all public APIs at this level.
The previous attempt to allow this didn't actually allow it, so this is
take two. As part of the previous attempt, I thought after tweaking the
test I had observed the test failing, and then succeeding after making
the product changes, but I must have been mistaken.
It turns out that our existing mocks tests weren't running at all
because of a missing `__init__.py` file. Once the missing `__init__.py`
is added, the tests run, but other tests ("test mode" tests) fail
because the code that creates the mocks and resources will run during
test discovery, and setting the mocks modifies global state.
To address the test issue, I've moved the mocks tests into their own
`test_with_mocks` package that can be run separately from other tests.
And addressed the original issue, by creating a root Stack resource if
one isn't already present when the mocks are set.
With these changes, a resource struct may tag a field with the empty
string. If such a field is present, any resource outputs that were not
unmarshalled into other fields will be unmarshalled into this field,
which must be a `MapOutput`.
Fixes#4629.
This avoids the "NotSupportedException : Multiple executions of TestAsync must run serially" from calls to `Deployment.TestAsync` in the mocks test, which can happen if a stack test runs before the mock test (.NET unit tests are run in random order).
Despite having the `[assembly: CollectionBehavior(DisableTestParallelization = true)]` attribute, it appears `dotnet test` is still running tests in parallel. To address, use a configuration file to disable parallelization.
Adds support for RegisterResource to accept map-typed implementations if Input as well as the existing struct-typed implementations. Currently these must be fully untyped - but both map[string]pulumi.Input and map[string]interface{} are allowed. In the future, it's plausible that a mode where the data itself is a map, but the ElementType implementation returns a struct could be supported, with the struct used to provide type information over the untyped map.
This change allows importing modules with calls to `pulumi.export` in unit tests. Previously, you'd have to structure the Python program in a way that avoids the `pulumi.export` from being called from unit tests.
* [sdk/python] Improve `ResoruceOptions.merge` type
The implementation correctly handles `None` inputs, so the type should allow these as well.
* Add CHANGELOG
After importing some resources, and running a second update with the
import still applied, an unexpected replace would occur. This wouldn't
happen for the vast majority of resources, but for some it would.
It turns out that the resources that trigger this are ones that use a
different format of identifier for the import input than they do for the
ID property.
Before this change, we would trigger an import-replacement when an
existing resource's ID property didn't match the import property, which
would be the case for the small set of resources where the input
identifier is different than the ID property.
To avoid this, we now store the `importID` in the statefile, and
compare that to the import property instead of comparing the ID.
When setting structured config values using `--path`, we automatically
treat values that can be converted into an integer via `strconv.Atoi` as
an integer, rather than as a string.
However, this ends up converting values like "0123456" into the integer
123456, stripping the leading 0, which isn't desirable for values like
commit SHAs, etc., where you want to keep the 0 (and keep it a string).
This change makes it so that values starting with 0 are not implicitly
converted to an integer; instead such values will remain a string.
* Make `async:true` the default for `invoke` calls (#3750)
* Switch away from native grpc impl. (#3728)
* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)
* Only retry as long as we get unavailable back. Anything else continues. (#3769)
* Handle all errors for now. (#3781)
* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)
* Upgrade all paths for sdk and pkg to v2
* Backport C# invoke classes and other recent gen changes (#4288)
Adjust C# generation
* Replace IDeployment with a sealed class (#4318)
Replace IDeployment with a sealed class
* .NET: default to args subtype rather than Args.Empty (#4320)
* Adding system namespace for Dotnet code gen
This is required for using Obsolute attributes for deprecations
```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```
* Fix the nullability of config type properties in C# codegen (#4379)
