The langauge SDKs will use this function to fetch the state required to
deserialize a resource reference. SDK support will be added as a follow-up
change.
Contributes to #2430.
Resources are serialized as their URN, ID, and package version. Each
Pulumi package is expected to register itself with the SDK. The package
will be invoked to construct appropriate instances of rehydrated
resources. Packages are distinguished by their name and their version.
This is the foundation of cross-process resources.
Related to #2430.
Co-authored-by: Mikhail Shilkov <github@mikhail.io>
Co-authored-by: Luke Hoban <luke@pulumi.com>
Co-authored-by: Levi Blackstone <levi@pulumi.com>
Move these tests to a new package, `lifecycletest`, that also exposes
APIs that allow consumers to implement their own lifecycle tests. This
is intended to ease the burden of testing plugin implementations and to
set the stage for cleaning up the lifecycle tests themselves.
This involves two changes to the public API, only one of which is
strictly necessary:
- The `host` field of `UpdateOptions` is now exported
- The `Journal` type has been moved from test-only code to the package
proper
The former change is necessary, as it is the mechanism by which package
consumers may inject their own plugin loaders. I was reluctant to expose
this field originally because I wanted to ensure that the behavior of
packages that embed Pulumi is consistent with that of the Pulumi CLI
with respect to plugin loading. I now believe that the risk of consumers
changing this behavior outside of test scenarios is low enough that we
can expose this field. This may also be useful for future scenarios,
e.g. statically linking providers and Pulumi programs.
The latter change is not necessary, but fleshes out the engine package
into a more complete toolkit. Downstream consumers may use the Journal
type to conveniently implement snapshotting.
Fixes: #5509
When changing from a passphrase provider to a cloud secrets provider,
the encryptionsalt is not required, so we should ensure this is removed
These changes add support for provider-side previews of create and
update operations, which allows resource providers to supply output
property values for resources that are being created or updated during a
preview.
If a plugin supports provider-side preview, its create/update methods
will be invoked during previews with the `preview` property set to true.
It is the responsibility of the provider to fill in any output
properties that are known before returning. It is a best practice for
providers to only fill in property values that are guaranteed to be
identical if the preview were instead an update (i.e. only those output
properties whose values can be conclusively determined without
actually performing the create/update operation should be populated).
Providers that support previews must accept unknown values in their
create and update methods.
If a plugin does not support provider-side preview, the inputs to a
create or update operation will be propagated to the outputs as they are
today.
Fixes#4992.
* fix pulumi bot token and env var
* use commit hash non-marketplace actions
* fix goreleaser binaries
foo
* fix goreleaser archive format
* update workflows to use repo-dispatch
* fixing tag naming
* add support for beta tags
* add prerelease workflow
* remove windows builds
* only use travis for tags
* remove prerelease steps
* fix container build indentation
* check the policy tempdir exists
Fixes: https://github.com/pulumi/docs/issues/4340
The deprecated message sits below the examples so it is not clear
to the user that the resource / datasource is deprecated
The logic for validating prompted values in 'new' wasn't quite right,
leading to the possibility of creating Pulumi.yaml files with blank
project names.
This manifests in various ways and I've hit it a number of times
over the past few months because of the way we handle project/stack
name conflicts in 'new' -- which itself is a bit annoying too:
https://github.com/pulumi/pulumi/blob/master/pkg/cmd/pulumi/new.go#L206-L207
Because we substitue a default value of "", and because the prompting
logic assumed default values are always valid, we would skip validation
and therefore accept a blank Pulumi.yaml file.
This generates an invalid project which causes errors elsewhere, such as
error: failed to load Pulumi project located at ".../Pulumi.yaml":
project is missing a 'name' attribute
I hit this all the time with our getting started guide because I've
gone through it so many times and have leftover stacks from prior
run-throughs. I wouldn't be surprised if a lot of people hit this.
The solution here validates all values, including the default.
Note also that we failed to validate the value used by 'new --yes'
which meant you could bypass all validation by passing --yes, leading
to similar outcomes.
I've added a couple new tests for these cases. There is a risk we
depend on illegal default values somewhere which will now be rejected,
but that would seem strange, and assuming the tests pass, I would
assume that's not true. Let me know if that's wrong.
Fixespulumi/pulumi#3255.
Instead of simplifying any module that ends with `/<name>`, only simplify
types where `<name>` matches the type name portion after camel-casing.
This continues to simplify tfbridge types like `aws:s3/bucket:Bucket`,
but would not simplify a type like `aws:s3:Bucket`.
We're not going to generate language-specific API docs for the Azure NextGen provider, only resource docs. This change makes it so the resource docs do not emit any links to nonexistent API docs.
* Revise host mode.
The current implementation of host mode uses a `pulumi host` command and
an ad-hoc communication protocol between the engine and client to
connect a language host after the host has begun listening. The most
significant disadvantages of this approach are the communication
protocol (which currently requires the use of stdout), the host-specific
command, and the difficulty of accommodating the typical program-bound
lifetime for an update.
These changes reimplement host mode by adding engine support for
connecting to an existing language runtime service rather than launching
a plugin. This capability is provided via an engine-specific language
runtime, `client`, which accepts the address of the existing languge
runtime service as a runtime option. The CLI exposes this runtime via
the `--client` flag to the `up` and `preview` commands, which similarly
accepts the address of an existing language runtime service as an
argument. These changes also adjust the automation API to consume the
new host mode implementation.
When installing a plugin, if it contains a `PulumiPlugin.yaml` file with a `runtime` value of `nodejs` or `python`, install dependencies for the plugin.
For Node.js, `npm install` is run (or `yarn install` if `PULUMI_PREFER_YARN` is set).
For Python, a virtual environment is created and deps installed into it.
Non-string provider inputs must be projected as JSON formatted strings. The current codegen simply calls `json.dumps` for such properties, but this does not work for the new input types, which aren't JSON serializable.
To address this, make use of the new `pulumi.runtime.to_json` utility function, which is capable of serializing raw dicts and input types as JSON.
We currently emit array types as `List[T]` for Python, but `List[T]` is invariant, which causes type checkers like mypy to produce errors when values like `["foo", "bar"]` are passed as args typed as `List[pulumi.Input[str]]`. Instead, we should move to using `Sequence[T]` which is covariant, and does not have this problem.
We actually already do this for `Dict` vs. `Mapping`, emitting map types as `Mapping[str, T]` rather than `Dict[str, T]` because `Mapping[str, T]` is covariant. This change makes us consistent for array types.
* add initial pull-request workflow
* run SDK test all
* add SDK tests
* fixup make targets
* add dist target
* revert back to 5 updates
* disable test
* add issue for test disabling
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.
Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.
The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.
These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).
An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.
This is the core of #2430.
An extra constructor overload was recently added to pass undefined state from
`get` for resources that do not have any state inputs (notably Kubernetes
resources). This ended up breaking PaC's `validateResourceOfType`, which relies
on type inference in common usage to determine the resource's args type based
on the signature of the constructor.
This constructor overload isn't necessary. Instead, we can remove it and modify
how the constructor is called inside `get`. This also makes it so we're not
exposing details about `get`'s implementation in the public API.
Changed the codegen in 6fd72dc0 but missed a condition that
is causing incorrect code in pulumi-kubernetes. This change
correctly generates inputs in both conditional branches.
If PyName(name) != PyNameLegacy(name), we panic, unless name is in an exclusion list.
Once all providers have been updated to this codegen (and we've added to `useLegacyName` and `excludeFromPanic` as appropriate), we can go back and remove the panic behavior along with the `excludeFromPanic` behavior.
If there's a config subpackage, we had explicitly added it to the map of modules because there *are* config properties, so we know it will be non-empty. This change ensures the config subpackage is imported in the top-level package's __init__.py and that the config subpackage itself has an __init__.py emitted.
Resource doc changes for Python:
- Types are included in constructor/function args
- The property names for input/output types are now always snake_case, regardless of the generated mapping tables, to match the new input/output classes
- Some other minor tweaks to function/constructor signatures (e.g. removed the `__props__` arg, as it's not meant to be used directly; use `@staticmethod` for static `get` methods).
Fixes: #5134
This ensures that `pulumi history` has been deprecated in favor of
the new `pulumi stack history` command. The deprecated command will
be removed in v3.0.0 of Pulumi
Fixes: #5126
In the newServiceSecretsManager, we were loading and saving the project stack
without understanding if any changes had actually been made - e.g. changing the
stack to be a new secrets provider
This has now been guarded against, tests added to understand when it will
actually be saved and we no longer get unnecessary config sorting when interacting
with a default stack AND no longer get an empty config map in our config
file when we initiate a new stack with a default secrets provider
* Use a package map where appropriate
* Fix imports
* handle NoneType for literal value expressions in go program gen
* add operator tests for c#, python, and node
Co-authored-by: evanboyle <evan@pulumi.com>
Fixes: #5104
This reverts a code change that was checking initially for the
SecretsProvider of the currentStack being an empty string. When it
was an empty string, we were checking the backend type and we were
setting a serviceSecretsManager. This wasn't correct, the logic
needed to check for an empty SecretsProvider AND an empty EncryptionSalt
The important part is that it needed to check for the EncryptionSalt
to make sure it wasn't a passphrase secrets manger
We make several calls to `os/user`, which uses CGO and means
cross-compilation is not possible. This replaces `os/user` with the
`luser` package, which is a drop-in replacement which does not use `CGO`
Certain operations in `engine/diff` mutate engine events during display.
This mutation can occur concurrently with the serialization of the event
for persistence, which causes a panic in the CLI. These changes fix the
offending code and add code that copies each engine event before
persisteing it in order to guard against future issues.
Rewrites that should produce nested applies due to functions that return
eventual types were instead producing a single top-level apply. These
changes fix that by considering a function that produces an eventual
value as inspecting eventual values.
Just what it says on the tin. Currently it's not possible to create a
valid value of this type because the `bytes` field is unexported. This
constructor fixes that.
The apply rewrite for relative traversals did not consider whether or
not the receiver was eventually-typed, and did not properly check
whether or not the relative traversal itself was eventually-typed. These
changes correct those mistakes.
In general, a package/module name in these targets is derived from the
module portion of a type token. If the type token is not already in an
expected form--namely, all lowercase--the generated package/module names
will also be in unexpected forms. These changes normalize the module
names to lowercase s.t. the generated package/module names conform to
expectations.
- NodeJS: fix get + constructor generation for types with required args
and no state type.
- Go: fix type + import generation for properties in the root module.
- NodeJS: fix get + constructor generation for types with required args
and no state type.
- Go: fix type + import generation for properties in the root module.
Previously, streamInvoke was only supported by
the query command. Copied the implementation
into the resource monitor, which will allow
streaming invoke commands to run during updates.
Also fixed a bug with cancellation of streaming
invokes. The check was comparing against a
hardcoded string, which did not match the actual
error string. Instead, we can rely on the error code.
Add a new package, `codegen/importer`, that can generate definitions for
resource states in PCL or TS/Python/C#/Go. The pipeline is relatively
simple: given a list of resource states, generate a PCL program in
memory, bind it, and pass it to the language-specific code generator.
This builds upon the existing PCL IR, and can be used with the currently
supported code generators.
Related to #1635.
* Use the correct format for package name when module name is empty.
* Add an exclusion for Docker's Image component resource when generating the Python formal params.
* Prefix the type name with the package name when linking to Python function names.
Instead of requiring a plugin host for package loading in the HCL2
binder, define a much narrower interface that exposes the ability to
fetch the schema for a package at a specific version. This interface is
defined in the schema package, which also exposes a caching loader that
is backed by provider plugins.
These changes also add some convenience methods to `*schema.Package` for
fast access to particular resources and functions.
Related to #1635.
In general, each item in an HCL2 body must be followed by a trailing
newline. The printer did not properly insert these newlines for body
items without any associated tokens/trivia, or with trivia that did not
include a trailing new line.
Related to #1635.
- Remove `Info` from `Source`. This method was not used.
- Remove `Stack` from `EvalSource`. This method was not used.
- Remove `Type` and `URN` from `Step`. These values are available via
`Res().URN.Type()` and `Res().URN`, respectively. This removes the
possibility of inconsistencies between the type, URN, and state of the
resource associated with a `Step`.
- Remove URN from StepEventMetadata.
The k8s SDK logs deprecations warnings as part of the
provider logic, and includes a flag to suppress these
warnings. Including them in the SDKs breaks this logic.
The PCL binder has supported resource options for some time, but these
options haven't been used or processed by the various code generators.
These options--particularly the parent and provider options0--are
critical for import codegen. These changes implement the basic set of
options, and add a note about fleshing out the rest as necessary.
One component of these changes is a new rewriter that rewrites property
references into property paths that are understood by the Pulumi engine.
This rewriter is used to preprocess the contents of the `ignoreChanges`
resource option.
These changes also hack around a weakness in the HCL2 type system:
In Go, references to resources should be typed as `hcl2.ResourceType`.
Unfortunately, this breaks the existing collection semantics associated
with resources. Because of this, the Go code generator does not have
enough information to know that it should generate a `[]pulumi.Resource`
for lists of resources. These changes hack around that limitation using
a Go-specific opaque type and some hardcoded comparisons in
`argumentTypeName`.
Fixes#4923.
The docs generator previously assumed that the opts parameter
for every resource was of the CustomResource type. This is
incorrect for the YAML and Helm overlays, which are
ComponentResources. This should be handled more generally
once our schema supports ComponentResources, but this fixes
the docs for now.
For the NodeJS k8s SDK, rather than falling back to default values
using the || operator, use the nullish coalescing operator (??).
This avoid situations where the primary value is set to false,
and then is overridden by the default value.
Use the schema package's Markdown parser and walk its AST to extract
examples.
These changes also rename StripNonRelevantExamples to FilterExamples.
This is preparatory work for #4159 and #4632.
In particular, use the parser to filter and extract examples. This also
sets up support for entity references in documentation that can be used
in order to render language-specific names for resources, functions,
types, and properties.
Related to #4632 and #4159.
Several users reported cases where error messages would
cause a panic if they contained accented characters. I wasn't
able to reproduce this failure locally, but tracked down the
panic to logging gRPC calls. The Message field is typed as
a string, which requires all of the characters to be valid UTF-8.
This change runs each log string through the strings.ToValidUTF8
function, which will replace any invalid characters with the
"unknown" character. This should prevent the the logger from
panicking.
Fixes: #4827
Diff:
```
if encrypted is not None:
- warnings.warn("Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"", DeprecationWarning)
+ warnings.warn("Rename PostgreSQL role resource attribute \"encrypted\" to \"encrypted_password\"", DeprecationWarning)
pulumi.log.warn("encrypted is deprecated: Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"")
```
When running `pulumi up`, after the preview, we prompt asking whether to proceed with the update. If you type to filter the options and then hit an arrow key a couple times, the CLI panics. This is a bug in the `survey` library we depend on. The issue has been fixed in the library upstream; this change updates our dependency.
