This commit adds a new language option for Python generation to specify
the package name instead of using `pulumi_x` where x is the name defined
in the schema.
A new test is added, and this has also been shown to produce no diff
when run against `pulumi-eks`.
Fixes: #6974
Passphrase Environment variables were set before loading the
secrets provider from state
Unfortunately, it seems that some users are using empty passphrases
and thus this newly introduced logic has broken their usecases
We now check that the environment variables are set - it doesn't
matter if they are set as empty, but the existance of an empty
environment variabe still suggests that it is an intentional
empty passphrase
* Remove leading and trailing whitespace in resource properties
* Make tests pass
* Add PULUMI_ACCEPT support to docs gen tests
* Handle a couple more places
Co-authored-by: Pat Gavlin <pat@pulumi.com>
Fixes: #6934
With this snippet of code:
```
func main() {
pulumi.Run(func(ctx *pulumi.Context) error {
// Create an AWS resource (S3 Bucket)
vpc, err := ec2.NewVpc(ctx, "main", &ec2.VpcArgs{
CidrBlock: pulumi.String("10.0.0.0/16"),
})
if err != nil {
return err
}
rt, err := ec2.NewRouteTable(ctx, "example", &ec2.RouteTableArgs{
VpcId: vpc.ID(),
})
// Export the name of the bucket
ctx.Export("rt", rt)
return nil
})
}
```
the CLI would panic on the diff as follows:
```
panic: fatal: An assertion has failed
goroutine 249 [running]:
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.failfast(...)
/private/tmp/pulumi-20210422-70582-1bpvlru/sdk/go/common/util/contract/failfast.go:23
github.com/pulumi/pulumi/sdk/v3/go/common/util/contract.Assert(...)
/private/tmp/pulumi-20210422-70582-1bpvlru/sdk/go/common/util/contract/assert.go:26
github.com/pulumi/pulumi/pkg/v3/engine.printPropertyValue(0xc0005d41b0, 0x57cce00, 0xc001da9050, 0x0, 0x1, 0x5932853, 0x4, 0x0)
/private/tmp/pulumi-20210422-70582-1bpvlru/pkg/engine/diff.go:511 +0x1485
```
This was due to the entire object being added to the output and
the property being a ResourceReference
On the changing of the code to use a switch statement, we can
now include the ResourceReference and ensure that we catch any
missing case statements with a panic as default
This means the same piece of code now outputs to the CLI as
follows:
```
Outputs:
rt: {
URN: "urn:pulumi:dev::testing-new-engine-diff::aws:ec2/routeTable:RouteTable::example"
ID : "rtb-09b37608ec34f3b49"
PackageVersion: ""
}
Resources:
3 unchanged
Duration: 2s
```
If `rootPackageName` is set, we can look for the version in the
baseImportPath rather than at a location based on the package name -
which currently fails if every component is not named `pulumi-*`. To err
on the side of caution, this method is only used for packages where
`rootPackageName` is set, meaning existing SDKs retain their current
behavior.
The new behavior is confirmed via the test added in #6862.
* Import subpackages lazily
* Tighten up lazy_import impl
* Eagerly register resources, but lazily load their impl
* Add CHANGELOG entry
* Satisfy lint
* Restore mypy behavior so the change is not breaking
* Fix golden tests
When working in a monorepo environment, it can be desirable to generate
Go SDKs into a structure less like the upstream SDKs, and more like
this:
github.com/x/mymonorepo/sdk/go/package-name
Where `package-name` is also the root of a Go module. Since
`package-name` is not a valid package name in Go, it's also desirable to
be able to choose a replacement name and reduce the amount of nesting.
This commit adds a new Go option to the schema, `rootPackageName`, which
can be used to modify the generated root package name (e.g. to
`mypackage` instead of `package-name`, and remove the additional layer
of nesting.
Test coverage is added to ensure that the correct file structure and
package names are generated.
This incorporates some post merge feedback from https://github.com/pulumi/pulumi/pull/6893
This doesn't change the functionality of any of the tests:
```
go test -v ./secrets/passphrase -count=1
=== RUN TestPassphraseManagerIncorrectPassphraseReturnsErrorCrypter
--- PASS: TestPassphraseManagerIncorrectPassphraseReturnsErrorCrypter (0.89s)
=== RUN TestPassphraseManagerIncorrectStateReturnsError
--- PASS: TestPassphraseManagerIncorrectStateReturnsError (0.00s)
=== RUN TestPassphraseManagerCorrectPassphraseReturnsSecretsManager
--- PASS: TestPassphraseManagerCorrectPassphraseReturnsSecretsManager (1.08s)
=== RUN TestPassphraseManagerNoEnvironmentVariablesReturnsError
--- PASS: TestPassphraseManagerNoEnvironmentVariablesReturnsError (0.00s)
PASS
ok github.com/pulumi/pulumi/pkg/v3/secrets/passphrase 2.270s
```
Fixes: #6286
When a user is using the passphrase provider and unsets the environment
variables that let them interact with the secrets provider, then would
get an error as follows:
```
▶ pulumi up -y -f
error: decrypting secret value: failed to decrypt: incorrect passphrase, please set PULUMI_CONFIG_PASSPHRASE to the correct passphrase
```
We are oging to change this error message to make it more obvious
to a user what the error is and how they need to fix it
```
▶ pulumi up -y -f
error: constructing secrets manager of type "passphrase": unable to find either `PULUMI_CONFIG_PASSPHRASE` nor `PULUMI_CONFIG_PASSPHRASE_FILE` when trying to access the Passphrase Secrets Manager. Please ensure one of these values are set to allow the operation to continue
```
Ideally, we would like to prompt the user for the passphrase at this
point rather than error, but the CLI could be in the middle of an
update operation as the same codepath is used for reading stackreference
values
This commit adjusts the way that Go module versions are discovered from
packages when generating Go programs, to account for those on module
version 1. Previously, this function would panic when dereferencing a
nil instance of semver.Version.
These changes fix a regression introduced by #6686 that caused the SDK
code generators for .NET, Python, and Typescript to omit definitions for
plain object types. This regression occurred because #6686 drew a
clearer line between types used as resource arguments and types used
as function arguments, but conflated "resource arguments" with "inputty
types". This caused the code generators to generate inputty types for
any types used as resource arguments, even those that are used for
plainly-typed properties.
Fixes#6796.
Add line breaks and whitespace to avoid long horizontal scrolls for Python constructor/function arguments. Also, include the new ResourceArgs constructor overload.
See #6200 for a complete description of the issue. In short, we generate
inconsistent names for object types depending on whether or not they are
transitively reachable from resources or functions, which risks
unintentional breaking changes due to schema updates.
1. Name "input" types differently: `TArgs` for a type that is used in
resource inputs, having `Input<T>` properties, and `T` for a type
that is used in invoke inputs. The same schema type can produce both.
2. Always keep the name `T` for output types, avoid appending `Result` to
the name.
3. As needed, introduce a flag in the existing providers' schemas to avoid
breaking changes. Consider removing it on a major version bump.
Fixes#6200.
These have been deprecated for a very long time and it's a trivial change to remove them from the generated code. Let's clean this up for the 3.0-based providers.
This change updates the Python SDK codegen to opt-in to the new casing
translation behavior, which will use the passed-in props type's property
name metadata for translations, rather than calling the resource's
`translate_input_property` and `translate_output_property` methods.
- FIX: Keys in user-defined dicts will no longer be unintentionally
translated/modified.
- BREAKING: Dictionary keys in nested output classes are now
consistently snake_case. If accessing camelCase keys from such output
classes, move to accessing the values via the snake_case property
getters (or snake_case keys). A warning will be logged when accessing
camelCase keys.
When serializing inputs:
- If a value is a dict and the associated type is an input type, the
dict's keys will be translated based on the input type's property
name metadata.
- If a value is a dict and the associated type is a dict (or Mapping),
the dict's keys will _not_ be translated.
When resolving outputs:
- If a value is a dict and the associated type is an output type, the
dict's keys will be translated based on the output type's property
name metadata.
- If a value is a dict and the associated type is a dict (or Mapping),
the dict's keys will _not_ be translated.
Update the RegisterResource method in the ResourceMonitor
to unmarshal the providers field added in d297db3 and then resolve
any provider references so that they can be set on the Construct call.
Add support for creating instances of resources in Python using a
`<Resource>Args` class. This capability aligns with how args are passed
to resources in all the other language SDKs and the separate object bag
allows the properties to be manipulated/validated/passed-around before
creating the resource.
We've been emitting calls to `New<Resource>` for resource registrations
in Go, passing `nil` for args. However, some of those `New<Resource>`
functions actually check for `nil` args and return an error if the
resource has required arguments.
At first, I was looking for a way to check inside `New<Resource>` if
the `URN` option was specified and in that case not error on
`nil` args (like we do in other languages), but we don't provide a way
to access the resource option values outside the Go SDK), so I don't
think there is a way to do it this way for Go.
So instead, this change updates the registration code to call
`ctx.RegisterResource` directly instead of `New<Resource>`, where we can
pass a `nil` args.
When using the filestate backend (local files and cloud buckets) there is no protection to prevent two processes from managing the same stack simultaneously.
This PR creates a locks directory in the management directory that stores lock files for a stack. Each backend implementation gets its own UUID that is joined with the stack name. The feature is currently available behind the `PULUMI_SELF_MANAGED_STATE_LOCKING=1` environment variable flag.
This change adds schema and codegen support for plain properties which
are emitted typed as the plain type rather than wrapped as an `Input`.
Plain properties require a prompt value and do not accept a value that
is `Output`.
When passing a package source as part of a `dotnet add package` in
our acceptance testing framework, dotnet was then trying to use that
package source for the restoration of other packages in the csproj
file.
We have removed passing the source to dotnet add package add
and replaced it with adding a machine level package source via
dotnet nuget add source command
this is the more correct way to work and will allow us to be able
to search multiple locations as part of the dotnet restore command
We currently include `version.txt` in the package via `<Content Include="version.txt">`. This places the file in the NuGet package in two locations: the `content` directory and `contentFiles`.
We want the file to be in `content` because this is where the Pulumi .NET Language host looks for the file when determining a program's required plugins.
However, having the file in `contentFiles` is problematic. For packages that reference other Pulumi resource packages (e.g. for multi-lang components), referenced `contentFiles` are included in the package by default. This means another package's `version.txt` will be included and used over the current project's `version.txt`. For example, if a multi-lang component package `Pulumi.Xyz` references `Pulumi.Aws`, the `version.txt` from `Pulumi.Aws` will be used in `Pulumi.Xyz` package rather than the intended `version.txt` for `Pulumi.Xyz`.
To address this, stop including `version.txt` in `contentFiles`. We do this by changing the `<Content Include="version.txt" />` to `<None Include="version.txt" Pack="True" PackagePath="content" />` in the project file. This ensures the file will still be included in the package in the `content` dir (where the Pulumi .NET language host expects to find it), but doesn't include the file in `contentFiles` which would cause it to be used in other packages that reference the package.
Further, when generating `<PackageReference>`s for packages that start with "Pulumi.", include an additional `ExcludeAssets="contentFiles"` attribute to prevent the package's `contentFiles` from being included.
Fixes: #4029Fixes: #3537
Should the user want to get permalinks when using a self-managed backend, they can pass a flag:
```
$ pulumi up --suppress-permalink false
```
Permalinks for these self-managed backends will be suppressed on `update`, `preview`, `destroy`,
`import` and `refresh` operations.
This new optional file can be used to include additional metadata about the provider plugin (such as version and custom server URL), which the Python language host will use when determining a program's required plugins.
This change fixes the codegen to emit the file in the correct location (inside the package dir). Note: Providers need to opt-in to emitting this file via a schema option (because it requires some Makefile changes to insert the version in the file) and we haven't done that with any of our providers yet.
The schema specifies supported environment variables
for Provider inputs, but these are not currently reflected
in the generated docs. This change adds any supported
environment variables to the input property comment
field on Provider resources.
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
This way, the tests use the built-in virtual environment support by
default, which is what most customers will be using. A new `UsePipenv`
option is available to go back to using pipenv for tests.
We previously looked for `python3` and fallback to `python` on all systems. However, our Windows CI images include a `python3.exe` symlink to `python.exe` which does not work with `venv`. So on Windows, just look for `python` first, falling back to `python3`. (The default python.org Windows installation only includes `python.exe`).
