Fixes: #6974
Passphrase Environment variables were set before loading the
secrets provider from state
Unfortunately, it seems that some users are using empty passphrases
and thus this newly introduced logic has broken their usecases
We now check that the environment variables are set - it doesn't
matter if they are set as empty, but the existance of an empty
environment variabe still suggests that it is an intentional
empty passphrase
This incorporates some post merge feedback from https://github.com/pulumi/pulumi/pull/6893
This doesn't change the functionality of any of the tests:
```
go test -v ./secrets/passphrase -count=1
=== RUN TestPassphraseManagerIncorrectPassphraseReturnsErrorCrypter
--- PASS: TestPassphraseManagerIncorrectPassphraseReturnsErrorCrypter (0.89s)
=== RUN TestPassphraseManagerIncorrectStateReturnsError
--- PASS: TestPassphraseManagerIncorrectStateReturnsError (0.00s)
=== RUN TestPassphraseManagerCorrectPassphraseReturnsSecretsManager
--- PASS: TestPassphraseManagerCorrectPassphraseReturnsSecretsManager (1.08s)
=== RUN TestPassphraseManagerNoEnvironmentVariablesReturnsError
--- PASS: TestPassphraseManagerNoEnvironmentVariablesReturnsError (0.00s)
PASS
ok github.com/pulumi/pulumi/pkg/v3/secrets/passphrase 2.270s
```
Fixes: #6286
When a user is using the passphrase provider and unsets the environment
variables that let them interact with the secrets provider, then would
get an error as follows:
```
▶ pulumi up -y -f
error: decrypting secret value: failed to decrypt: incorrect passphrase, please set PULUMI_CONFIG_PASSPHRASE to the correct passphrase
```
We are oging to change this error message to make it more obvious
to a user what the error is and how they need to fix it
```
▶ pulumi up -y -f
error: constructing secrets manager of type "passphrase": unable to find either `PULUMI_CONFIG_PASSPHRASE` nor `PULUMI_CONFIG_PASSPHRASE_FILE` when trying to access the Passphrase Secrets Manager. Please ensure one of these values are set to allow the operation to continue
```
Ideally, we would like to prompt the user for the passphrase at this
point rather than error, but the CLI could be in the middle of an
update operation as the same codepath is used for reading stackreference
values
* Make `async:true` the default for `invoke` calls (#3750)
* Switch away from native grpc impl. (#3728)
* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)
* Only retry as long as we get unavailable back. Anything else continues. (#3769)
* Handle all errors for now. (#3781)
* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)
* Upgrade all paths for sdk and pkg to v2
* Backport C# invoke classes and other recent gen changes (#4288)
Adjust C# generation
* Replace IDeployment with a sealed class (#4318)
Replace IDeployment with a sealed class
* .NET: default to args subtype rather than Args.Empty (#4320)
* Adding system namespace for Dotnet code gen
This is required for using Obsolute attributes for deprecations
```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```
* Fix the nullability of config type properties in C# codegen (#4379)
- Cache the username and last verified time associated with each logged-in
backend
- In the HTTP backend, verify the access token explicitly at most once
per hour
This trades off a little bit of usability for improved inner-loop
latency: if a user's API token becomes invalid less than an hour after
it was last verified, the user will see 4xx errors when attempting stack
operations rather than seeing the login prompt.
This caching is enabled by wrapping the `secrets.Manager` returned by
`DefaultSecretsProvider.OfType` in an outer `secrets.Manager` that
cooperates with `stack.{Serialize,Deserialize}PropertyValue`. Ciphertext
is cached on a per-secret-instance basis (i.e. not a per-plaintext-value
basis). Cached ciphertext is only reused if the plaintext for the secret
value has not changed. Entries are inserted into the cache upon both
encryption and decryption so that values that originated from ciphertext
and that have not changed can aoid re-encryption.
Contributes to #3178.
Adds support for additional cloud secrets providers (AWS KMS, Azure KeyVault, Google Cloud KMS, and HashiCorp Vault) as the encryption backend for Pulumi secrets. This augments the previous choice between using the app.pulumi.com-managed secrets encryption or a fully-client-side local passphrase encryption.
This is implemented using the Go Cloud Development Kit support for pluggable secrets providers.
Like our cloud storage backend support which also uses Go Cloud Development Kit, this PR also bleeds through to users the URI scheme's that the Go CDK defines for specifying each of secrets providers - like `awskms://alias/LukeTesting?region=us-west-2` or `azurekeyvault://mykeyvaultname.vault.azure.net/keys/mykeyname`.
Also like our cloud storage backend support, this PR doesn't solve for how to configure the cloud provider client used to resolve the URIs above - the standard ambient credentials are used in both cases. Eventually, we will likely need to provide ways for both of these features to be configured independently of each other and of the providers used for resource provisioning.
The change does two things:
- Reorders some calls in the CLI to prevent trying to create a secrets
manager twice (which would end up prompting twice).
- Adds a cache inside the passphrase secrets manager such that when
decrypting a deployment, we can re-use the one created earlier in
the update. This is sort of a hack, but is needed because otherwise
we would fail to decrypt the deployment, meaning that if you had a
secret value in your deployment *and* you were using local
passphrase encryption *and* you had not set PULUMI_CONFIG_PASSPHRASE
you would get an error asking you to do so.
Fixes#2729
This is helpful some round trip cases where we many not be able to
build the encrypter or decrypter but we will end up not needing
them. When we fail to load the manager, we return a manager that has
the correct state, but will error when it tries to preform any
operations. However, if there are no secrets in the deployment, these
methods will never be called and we'll be able to correctly roundtrip
checkpoints even without having access to the password (since there
were no secret values to decrypt or encrypt).
This change allows using the passphrase secrets manager when creating
a stack managed by the Pulumi service. `pulumi stack init`, `pulumi
new` and `pulumi up` all learned a new optional argument
`--secrets-provider` which can be set to "passphrase" to force the
passphrase based secrets provider to be used. When unset the default
secrets provider is used based on the backend (for local stacks this
is passphrase, for remote stacks, it is the key managed by the pulumi
service).
As part of this change, we also initialize the secrets manager when a
stack is created, instead of waiting for the first time a secret
config value is stored. We do this so that if an update is run using
`pulumi.secret` before any secret configuration values are used, we
already have the correct encryption method selected for a stack.
We move the implementations of our secrets managers in to
`pkg/secrets` (which is where the base64 one lives) and wire their use
up during deserialization.
It's a little unfortunate that for the passphrase based secrets
manager, we have to require `PULUMI_CONFIG_PASSPHRASE` when
constructing it from state, but we can make more progress with the
changes as they are now, and I think we can come up with some ways to
mitigate this problem a bit (at least make it only a problem for cases
where you are trying to take a stack reference to another stack that
is managed with local encryption).
When constructing a Deployment (which is a plaintext representation of
a Snapshot), ensure that we encrypt secret values. To do so, we
introduce a new type `secrets.Manager` which is able to encrypt and
decrypt values. In addition, it is able to reflect information about
itself that can be stored in the deployment such that we can
deserialize the deployment into a snapshot (decrypting the values in
the process) without external knowledge about how it was encrypted.
The ability to do this is import for allowing stack references to
work, since two stacks may not use the same manager (or they will use
the same type of manager, but have different state).
The state value is stored in plaintext in the deployment, so it **must
not** contain sensitive data.
A sample manager, which just base64 encodes and decodes strings is
provided, as it useful for testing. We will allow it to be varried
soon.
