* Fix rename stack message when attempting to move organizations
* Include the URL to change stack ownership
Co-authored-by: Lee-Ming Zen <lee@pulumi.com>
Rather than shelling out to the `rm` and `cp` commands,
use Go's os utils to perform these operations. This will
work on any platform rather than just Linux.
This package contains a driver for generating source code from HCL2
expressions. The driver is based on the fmt package's support for custom
formatters.
These changes implement `GetRequiredPlugins` for Go using a registry
mechanism and an alternate entry point for `pulumi.Run`. Packages that
require plugins are expected to register themselves with the Pulumi SDK.
When `pulumi.Run` is used and the `PULUMI_PLUGINS` envvar is truthy, the
program will dump a JSON-encoded description of its required plugins to
stdout. The language host then uses this description to respond to
These changes add a package for type checking and modeling HCL2
configurations. It is made up of three primary components:
1. A static type system
2. A semantic representation of HCL2 expressions and a binder from HCL2
native syntax to this representation
3. A semantic representation of HCL2 structural elements and binders
from HCL2 native syntax to this representation.
The type system is described in the "Extended Types" section of the
specification. The semantic representations of expressions and
structural elements are documented in their implementations.
* Generate the constructor params for Python along with other languages.
* Remove redundant py_function_param nested template. Declare a new type for defining property characteristics rather than using inlining formal params. Generate the Lookup functions for all languages similar to the constructor params with linking enabled.
* Fix bug with generating the input arg type name for Functions in Go.
* Add prefix for args param of a Go-based Resource Function.
* Input args for Go-based Functions use Lookup*Args and not Get*Args.
* Turns out that args for Go-based Functions use a different prefix based on whether the function is a package-level or module-level Function.
* Update the Python list and dictionary type names for the resource doc generator.
* Add a separate function for Python doc helper return a type string representing dictionaries that are simple maps and don't have a known nested element type.
* Also remove the Examples property from the resourceDocArgs type since the resource Comment will contain the multi-lang examples on their surrounded by a shortcode.
The integration test framework currently supports using `dep` for dependency management.
However, `dep` has no native ability to manage "yarn link"-style dependencies on locally available packages.
This is a necessary scenario for testing in most repos though, as (e.g.) examples in the Kubernetes repo need to test against the locally available version of `pulumi-kubernetes`.
The best we can do is a trick of (a) deleting the vendored copy of the locally available dependency (b) copying the locally available dependency into the vendor folder (c) deleting the nested vendor folder in the new copy of the locally available dependency.
* Added a new template for Functions. Implement the genFunction method for generating the docs for Functions.
* Rename type resourceArgs to resourceDocArgs. Minor updates the resource template.
* Generate nested types for Functions.
* Unexport types that don't need to be exported. Create the doc language helper objects in an init function and reuse them rather than recreating them every time. Update genNestedTypes to work with schema functions or resources.
* Fixed bug in nested type generation for Functions. Fixed bug in generating input and output doc links for nested types.
The changes in #4004 caused old provider configuration to be used even when a provider was different between inputs and outputs, in the case that the diff returned DiffUnkown.
To better handle that case, we compute a more accurate (but still conservative) DiffNone or DiffSome so that we can ensure we conservatively update to a new provider when needed, but retain the performance benefit of not creating and configuring a new provider as much as possible.
Part of https://github.com/pulumi/pulumi-aws/issues/814.
Fixes: https://github.com/pulumi/pulumi-terraform-bridge/issues/119
This allows us to specify an overlays block e.g.
```
Overlay: &tfbridge.OverlayInfo{
DestFiles: []string{
"pulumi_docker/docker.py",
"pulumi_docker/image.py",
},
},
```
The overlays files are treated differently to normal module files
as they are not generated. This structure means that we will emit
the correct entries in the __init__.py file
Without this structure (ie. pulumi_pkgname), the generator actually
copies the file (i.e. docker.py) to the root of the Python SDK. This
is because the structure of the Python SDK has a sub-folder than that
of the NodeJS SDK
I tested this using PR https://github.com/pulumi/pulumi-docker/pull/141
and this now works as expected and we can take advantage of the new
Python overlays for Docker
* Update properties.tmpl to render property comment as-is. WIP splitting out properties to lang-specific tables.
* Generate the constructor dynamically from the resource per language.
* Add doc functions in each language generator package for getting doc links for types..and later other functions too.
* Render the constructor params in the Go code and inject into the template.
* Generate nodejs types using the nodejs lang generator.
* Add a templates bundler. Added a new Make target for autogenerating a static bundle for the resource docs generator.
* Generate type links for all languages based on their schema type. Render the property type with a link if the underlying elements have a supporting type. Fix word-breaks for Python type names.
* Various changes including the introduction of an interface type under the codegen package to help with generating some language-specific information for the resource docs generator.
* Add a function to explicitly generate links for input types of nested types. Fix the resource doc link generator for Go. Don't replace the module name from the nodejs language type.
* Fix bug with C# property type html encoding.
* Fix some template formatting. Pass the state inputs for Python to generate the lookup function for it.
* Do not generate the examples section if there are none.
* Generating the property types per language.
* Formatting. Rename function for readability.
* Add comments. Update README.
* Use relative URLs for doc links within the main site
These changes implement `GetRequiredPlugins` for Go using a registry
mechanism and an alternate entry point for `pulumi.Run`. Packages that
require plugins are expected to register themselves with the Pulumi SDK.
When `pulumi.Run` is used and the `PULUMI_PLUGINS` envvar is truthy, the
program will dump a JSON-encoded description of its required plugins to
stdout. The language host then uses this description to respond to
`GetRequiredPlugins`.
* started transformations for go sdk
* added first basic test
* added second test with child
* added RegisterStackTransformation
* added a couple tests to lifecycle_test
* update CHANGELOG and test
* included TODO for #3846
In the very common case where provider configuration does not change, during preview we were calling `Configure` on the cloud provider twice - once for the "old" configuration, and once for the "new" configuration.
This is not necessary, and we can just avoid using the new provider when configuration has not changed, since we will have configured the old provider very early so if we can use that we should.
Note that this technically doesn't prevent the second call to `Configure` from being made, but it prevents us from ever waiting on it. We may want to go further and avoid even calling `Configure` on the provider in this case.
Part of #3671.
These changes add a new method to the resource provider gRPC interface,
`GetSchema`, that allows consumers of these providers to extract
JSON-serialized schema information for the provider's types, resources,
and functions.
These changes add a helper package for parsing HCL2 syntax files. The
helpers are intended to provide direct access to syntax elements that
are abstracted away by the standard HCL2 parser. The bulk of the code
deals with mapping syntax nodes to their relevant tokens in order to
avoid losing information about the comments associated with syntax
nodes.
It appears there are cases where our IsInteractive heuristics return true, but terminal.GetSize returns an error. In these cases, we should assume we do not have an interactive terminal and avoid trying to render interactive progress by default.
Fixes#3935.
* Make Python StackReference test similar to others (with two steps)
* Include new Python StackReference integration test that uses multiple stacks
* Expose various life cycle methods for ProgramTester
* Revert "Use test helper. (#1977)"
This reverts commit e498cab239.
* Avoid duplicate newlines
Ensure that we print exactly one trailing newline per log entry.
- Make go comment generation lint-clean for blank comment lines
- Fix the casing of `Provider.py` in Python to `provider.py`
- Fix a spacing issue in the NodeJS code generator
We can't correctly print simple messages for prelude events when doing progress based display in a terminal, as it would lead to resetting the display of the table rendering.
This does mean that `--show-config` no longer works in the default terminal display mode - but it's not clear it *can* work correctly (at least as currently implemented) since it doesn't cleanly participate in the table rendering.
For cases where `--show-config` is not set (the norm) -nothing would have been printed anyway, so the changes here just avoid resetting the table rendering unnecessarily.
Fixes#3469.
The provider plugin protocol is to write a port number followed by `\n`. We must guarantee we do that even on Windows, so must avoid Python `print` statements which implicitly rewrite newlines to platform specific character sequences.
Fixes#3807.
Replace the various `defaults` maps in the schema with per-property
`default` and `defaultInfo` fields. The former holds the static default
value; the latter holds the envvars and language-specific info.
Also, fix a minor bug in the Python codegen that caused diffs in
property docstrings.
We were seeing that ~all same steps were requiring checkpoint writes due to percieving a difference between `Dependencies` being `nil` and `[]URN{}` - which should be considered the same for this purpose.
The schema format is described in pkg/codegen/schema/schema.go. The code
generators are derived from the code generators contained in
https://github.com/pulumi/pulumi-terraform-bridge/pkg/tfgen, with the
exception of the Go code generator, which is net new.
The redesign is focused around providing better static typings and
improved ease-of-use for the Go SDK. Most of the redesign revolves
around three pivots:
- Strongly-typed inputs, especially for nested types
- Struct-based resource and invoke APIs
- Ease-of-use of Apply
1. Strongly-typed inputs
Input is the type of a generic input value for a Pulumi resource.
This type is used in conjunction with Output to provide polymorphism
over strongly-typed input values.
The intended pattern for nested Pulumi value types is to define an
input interface and a plain, input, and output variant of the value
type that implement the input interface.
For example, given a nested Pulumi value type with the following shape:
```
type Nested struct {
Foo int
Bar string
}
```
We would define the following:
```
var nestedType = reflect.TypeOf((*Nested)(nil)).Elem()
type NestedInput interface {
pulumi.Input
ToNestedOutput() NestedOutput
ToNestedOutputWithContext(context.Context) NestedOutput
}
type Nested struct {
Foo int `pulumi:"foo"`
Bar string `pulumi:"bar"`
}
type NestedInputValue struct {
Foo pulumi.IntInput `pulumi:"foo"`
Bar pulumi.StringInput `pulumi:"bar"`
}
func (NestedInputValue) ElementType() reflect.Type {
return nestedType
}
func (v NestedInputValue) ToNestedOutput() NestedOutput {
return pulumi.ToOutput(v).(NestedOutput)
}
func (v NestedInputValue) ToNestedOutputWithContext(ctx context.Context) NestedOutput {
return pulumi.ToOutputWithContext(ctx, v).(NestedOutput)
}
type NestedOutput struct { *pulumi.OutputState }
func (NestedOutput) ElementType() reflect.Type {
return nestedType
}
func (o NestedOutput) ToNestedOutput() NestedOutput {
return o
}
func (o NestedOutput) ToNestedOutputWithContext(ctx context.Context) NestedOutput {
return o
}
func (o NestedOutput) Foo() pulumi.IntOutput {
return o.Apply(func (v Nested) int {
return v.Foo
}).(pulumi.IntOutput)
}
func (o NestedOutput) Bar() pulumi.StringOutput {
return o.Apply(func (v Nested) string {
return v.Bar
}).(pulumi.StringOutput)
}
```
The SDK provides input and output types for primitives, arrays, and
maps.
2. Struct-based APIs
Instead of providing expected output properties in the input map passed
to {Read,Register}Resource and returning the outputs as a map, the user
now passes a pointer to a struct that implements one of the Resource
interfaces and has appropriately typed and tagged fields that represent
its output properties.
For example, given a custom resource with an int-typed output "foo" and
a string-typed output "bar", we would define the following
CustomResource type:
```
type MyResource struct {
pulumi.CustomResourceState
Foo pulumi.IntOutput `pulumi:"foo"`
Bar pulumi.StringOutput `pulumi:"bar"`
}
```
And invoke RegisterResource like so:
```
var resource MyResource
err := ctx.RegisterResource(tok, name, props, &resource, opts...)
```
Invoke arguments and results are also provided via structs, but use
plain-old Go types for their fields:
```
type MyInvokeArgs struct {
Foo int `pulumi:"foo"`
}
type MyInvokeResult struct {
Bar string `pulumi:"bar"`
}
var result MyInvokeResult
err := ctx.Invoke(tok, MyInvokeArgs{Foo: 42}, &result, opts...)
```
3. Ease-of-use of Apply
All `Apply` methods now accept an interface{} as the callback type.
The provided callback value must have one of the following signatures:
func (v T) U
func (v T) (U, error)
func (ctx context.Context, v T) U
func (ctx context.Context, v T) (U, error)
T must be assignable from the ElementType of the Output. If U is a type
that has a registered Output type, the result of the Apply will be the
corresponding Output type. Otherwise, the result of the Apply will be
AnyOutput.
Fixes https://github.com/pulumi/pulumi/issues/2149.
Fixes https://github.com/pulumi/pulumi/issues/3488.
Fixes https://github.com/pulumi/pulumi/issues/3487.
Fixes https://github.com/pulumi/pulumi-aws/issues/248.
Fixes https://github.com/pulumi/pulumi/issues/3492.
Fixes https://github.com/pulumi/pulumi/issues/3491.
Fixes https://github.com/pulumi/pulumi/issues/3562.
* Add a new metadata property for BuildNumber. Update Travis and GitLab to set both Build ID and Build Number. Add link to env vars doc for Codefresh.
* Update changelog
* Update CI vars detection test.
* Add PR number to changelog.
* Use Merge Request Instance ID instead of the Merge Request ID for GitLab CI.
* Use GitLab Pipeline Instance ID as the BuildID for GitLab CI.
* Update the changelog.
* Update the test for GitLab CI detection.
* Fix logic to determine PRNumber and BuildURL for Az Pipelines.
* Update changelog
* Set the BranchName to the PR source branch if PRNumber is not empty.
* added support for using GOOGLE_CREDENTIALS environment variable for authenticating with gs:// file state
* modified the change to fix#2791 as well
* fixed a small bug
* fixed linter error
* added code comments
* Update pkg/backend/filestate/gcpauth.go
Co-Authored-By: CyrusNajmabadi <cyrus.najmabadi@gmail.com>
* Parse provided backend url to check if scheme is gs://
* Update changelog
A regression was introduced when we added support for non-Node.js Pulumi programs to run Policy Packs. With that change, we now pass the Policy Pack's full path as the plugin's pwd (so that it would load the `@pulumi/pulumi/cmd/run-policy-pack` Node module from the Policy Pack's node_modules rather than the program's node_modules), but we also pass the path to the policy pack as well. If the path is a full rooted path, this would work fine, and that's what our tests do. However, if a relative path is specified, then it will be looking to load the Policy Pack relative to the pwd, which doesn't produce a correct path leading to failures trying to load the Policy Pack.
Since the pwd is the policy pack path, we can simply pass the path as `"."` to the analyzer plugin, and it will load the policy pack in its pwd.
This change adds support for setting `PULUMI_PREFER_YARN` to true to opt-in to preferring `yarn` over `npm` when installing Node.js dependencies (and publishing Policy Packs). If `PULUMI_PREFER_YARN` is truthy, but `yarn` cannot be found on `$PATH`, we fallback to using `npm`. If `npm` can't be found on `$PATH`, we provide a more helpful error message.
Add support for a test option that indicates that failed update steps
should be retried. Currently the maximum number of retries (3) is not
configurable.
The text "Plan applied failed: " is pretty inscrutable given our
current system. While both "plan" and "apply" are concepts inside the
the implementation of the CLI, we usually talk in terms of `preview`
and `update`. I suspect there are some cases where this prefix is not
100% technically correct, and if there's a better short way of saying
something more correct, I would love to adopt that instead, but as is,
I would really love to get rid of the "Plan apply failed" text in our
system, it pains me every time I read it.
Codepaths which could result in a hang will print a message to the console indicating the problem, along with a link to documentation on how to restructure code to best address it.
`StackReference.getOutputSync` and `requireOutputSync` have been deprecated as they may cause hangs on some combinations of Node and certain OS platforms. `StackReference.getOutput` and `requireOutput` should be used instead.
To allow Policy Packs to run against Pulumi programs written in all languages, we now look for the `@pulumi/pulumi/cmd/run-policy-pack` module in the Policy Pack's node_modules (instead of in the Pulumi program's node_modules; which doesn't exist for non-node languages). The `@pulumi/policy` library that a Policy Pack will depend on should already depend on a recent enough version of `@pulumi/pulumi`. When we can't find the module, it's more likely it's due to the dependencies for the Policy Pack not being installed. Provide a more helpful error message in this case.
- If an untargeted create would not affect the inputs of any targeted
resources, do not fail the update. Untargeted creates that are
directly dependend on by targeted resources will still cause failures
that inform the user to add the untargeted resources to the --target
list.
- Users may now pass the `--target-dependents` flag to allow targeted
destroys to automatically target dependents that must be destroyed in
order to destroy an explicitly targeted resource.
