Commit graph

1857 commits

Author SHA1 Message Date
Mikhail Shilkov 273ade2ac0 Parameterize the progress function with a message 2019-08-23 14:45:29 +02:00
Mikhail Shilkov 370047bbb6 Show a progress bar for plugin installation during pulumi up 2019-08-23 14:45:29 +02:00
Mikhail Shilkov c270204387 Check project existence during pulumi new 2019-08-23 09:09:54 +02:00
Chris Smith eb0934970c
Add support for filtering stacks by organization, tag (#3108)
* Add support for filtering stacks by organization, tag

* Update CHANGELOG.md

* Address PR feedback

* Address even more PR feedback

* Support empty-string filters
2019-08-22 13:56:43 -07:00
Pat Gavlin 42fc75fffe
Fail read steps with missing resources. (#3123)
Just what it says on the tin.

Fixes #262.
2019-08-21 10:09:02 -07:00
Pat Gavlin 8745440c1b
Allow users to explicitly disable delete-before-replace. (#3118)
With these changes, a user may explicitly set `deleteBeforeReplace` to
`false` in order to disable DBR behavior for a particular resource. This
is the SDK + CLI escape hatch for cases where the changes in
https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable
behavior.
2019-08-20 15:51:02 -07:00
Paul Stack 2c0510a23e
Update the JSON representation of customTimeouts in state (#3101)
Omit them when empty and change the casing of Create, Update and
Delete
2019-08-21 01:01:27 +03:00
Fraser Waters a46357a0b3 Fix comment about QName regex (#3121)
The comment on QName explained that QNames match the Name regex, but
the regex written was slightly different to the actual Name regex
(missed that '.' is a valid char in each group).
2019-08-20 14:43:48 -07:00
Paul Stack cfd3b8ce09
Allow Acceptance Tests to pass a flag that runs an upgrade test (#3110)
This will allow us to install a latest version from PyPi or NPM
and then yarn link / pip install from local machine and test that
no changes are introduced when this occurs

If the flag is not passed, then the dependencies are installed
as before where we prefer a local package to be linked / installed
2019-08-20 11:08:09 +03:00
Matt Ellis 900046a6ed
Merge pull request #3089 from pulumi/ellismg/fix-2714
Fix issues in the filestate backend
2019-08-19 10:50:44 -07:00
Mikhail Shilkov 04b169a64b Add a hint about the org name in stack in pulumi new 2019-08-19 09:21:02 +02:00
Mikhail Shilkov 2a88bc8e7e Replace regex with a replacer 2019-08-19 08:28:34 +02:00
Mikhail Shilkov 82276e3418 Skip non-utf8 secrets when regexing logs 2019-08-19 08:28:34 +02:00
Matt Ellis b14e3d9c35 Do not use os package in filestate backend
Since the filestate backend is now written using the go-cloud blob
abstraction, places where we were using functions from the `os`
package were very unlikely to be correct.

This change removes their uses in favor of APIs provided by
go-cloud (which sometimes requires more work than before).
2019-08-16 13:42:27 -07:00
Matt Ellis 828086d638 Do not attempt to load checkpoint before saving a snapshot
For historical reasons, we used to need to require to load an existing
checkpoint to copy some data from it into the snapshot when saving a
new snapshot. The need for this was removed as part of the general
work in #2678, but we continued to load the checkpoint and then just
disregard the data that was returned (unless there was an error and
that error was not FileNotFound, in which case we would fail).

Our logic for checking if something was FileNotFound was correct when
we wrote it, but when we adopted go-cloud in order to have our
filestate backend also write to blob storage backends like S3, we
forgot that we had checks like `os.IsNotExists()` floating around
which were now incorrect. That meant if the file did not exist for
some reason, instead of going along as planned, we'd error out now
with an error saying something wasn't found.

When we write a checkpoint, we first "backed up" the initial version
by renaming it to include a `.bak` suffix, then we write the new file
in place. However, this can run afoul of eventual consistency models
like S3, since there will be a period of time in which a caller may
observe that the object is missing, even after a new version is
written (based on my understanding of [S3's consistency
model](https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel)

Since we no longer need to actually copy any information from the
previous checkpoint, we can simply remove the call entirely to load
it.

As a follow up, we need to audit places inside the filebased backend
that assume `os.*` functions are going to do what we want them to do,
since in general they will not.

Fixes #2714
2019-08-16 13:40:55 -07:00
Matt Ellis 342f8311a1 Fix renaming a freshly created stack using the local backend
Attempting to `pulumi stack rename` a stack which had been created but
never updated, when using the local backend, was broken because
code-paths were not hardened against the snapshot being `nil` (which
is the case for a stack before the initial deployment had been done).

Fixes #2654
2019-08-16 13:39:34 -07:00
Paul Stack f8db8e4209
Allow resource IDs to change on reresh steps (#3087)
* Allow resource IDs to change on reresh steps

This is a requirement for us to be able to move forward with
versions of the Terraform Azurerm provider. In v1.32.1, there was
a state migration that changed the ID format of the azure table
storage resource

We used to have a check in place for old ID being equal to new ID.
This has been changed now and we allow the change of ID to happen
in the RefreshStep

* Update pkg/resource/deploy/step.go

Co-Authored-By: Pat Gavlin <pat@pulumi.com>
2019-08-16 21:04:03 +03:00
Matt Ellis 0bb4e6d70b Respond to PR feedback
Address post commit feedback from Cyrus on
pulumi/pulumi#3071
2019-08-15 12:42:51 -07:00
Chris Smith 952b6d0d89
Remove some unused apitype fields (#3057) 2019-08-14 15:26:58 -07:00
Matt Ellis 9308246114 Do not taint all stack outputs as secrets if just one is
When using StackReference, if the stack you reference contains any
secret outputs, we have to mark the entire `outputs` member as a
secret output. This is because we only track secretness on a per
`Output<T>` basis.

For `getSecret` and friends, however, we know the name of the output
you are looking up and we can be smarter about if the returned
`Output<T>` should be treated as a secret or not.

This change augments the provider for StackReference such that it also
returns a list of top level stack output names who's values contain
secrets. In the language SDKs, we use this information, when present,
to decide if we should return an `Output<T>` that is marked as a
secret or not. Since the SDK and CLI are independent components, care
is taken to ensure that when the CLI does not return this information,
we behave as we did before (i.e. if any output is a secret, we treat
every output as a secret).

Fixes #2744
2019-08-13 16:11:38 -07:00
CyrusNajmabadi 2afe5b88b8
Disable interactive-terminal spinner when on a CI machine. (#3070) 2019-08-13 12:50:09 -07:00
Alex Clemmer 6b4832b416 Emit stderr of npm install
If we don't process and report the stderr of `npm install`, the output
is "orphaned" during error condition, and only something like "exit code
1" is reported.
2019-08-13 12:48:16 -07:00
Pat Gavlin 57b6e84645
Prepare for the v1.0.0-beta.1 release. (#3079)
- Update scripts/get-version and friends to understand the new -alpha
  and -beta tags
- Update the CHANGELOG
2019-08-13 11:41:32 -07:00
Alex Clemmer ef8cc236c4 Implement --policy-pack flag on up and preview
Fixes pulumi/pulumi-policy#43.
2019-08-12 12:45:48 -07:00
Alex Clemmer 0dccc4b447 Normalize policy pack installation path
Fixes pulumi/pulumi-policy#59.
2019-08-12 12:45:48 -07:00
Alex Clemmer 93cd089428 Print help message when too few arguments
Fixes pulumi/pulumi-policy#61.
2019-08-12 12:45:48 -07:00
Praneet Loke a190e54988
Add support for detecting Jenkins CI env vars. (#3039) 2019-08-09 12:53:51 -07:00
Alex Clemmer 154c1caa43 Fix small bug in untar code 2019-08-07 17:06:57 -07:00
Alex Clemmer 12668756d5 Use S3 URL for policy pack download
Companion to the breaking change at pulumi/pulumi-service#3842.
2019-08-07 13:12:49 -07:00
Pat Gavlin 6983e2639b
Fix conversion of empty array properties. (#3047)
Empty `[]interface{}` values were being converted to array property
values with a `nil` element, and empty array property values were being
coverted to `nil` `[]interface{}` values. These changes fix the
converters to return empty but non-nil values in both cases.

This is part of the fix for
https://github.com/pulumi/pulumi-kubernetes/issues/693.
2019-08-07 11:42:40 -07:00
Pat Gavlin 62189e6053
Harden asset and archive deserialization. (#3042)
- Ensure that type assertions are guarded, and that incorrectly-typed
  properties return errors rather than panicking
- Expand the asset/archive tests in the Node SDK to ensure that eventual
  archives and assets serialize and deserialize correctly

Fixes #2836.
Fixes #3016.
2019-08-06 16:32:05 -07:00
Alex Clemmer 7070304f81 Use npm pack for policy publish
Fixes #35.
2019-08-05 16:12:12 -07:00
Alex Clemmer 6360cba588 Improve "project not found" error messages
Fixes pulumi/pulumi-policy#36.
Fixes pulumi/pulumi-policy#37.
2019-08-05 14:14:20 -07:00
Erin Krengel 2a968a2086
Move required PP out of start update (#3023) 2019-08-05 13:57:43 -07:00
Luke Hoban 6ed4bac5af
Support additional cloud secrets providers (#2994)
Adds support for additional cloud secrets providers (AWS KMS, Azure KeyVault, Google Cloud KMS, and HashiCorp Vault) as the encryption backend for Pulumi secrets. This augments the previous choice between using the app.pulumi.com-managed secrets encryption or a fully-client-side local passphrase encryption.

This is implemented using the Go Cloud Development Kit support for pluggable secrets providers.

Like our cloud storage backend support which also uses Go Cloud Development Kit, this PR also bleeds through to users the URI scheme's that the Go CDK defines for specifying each of secrets providers - like `awskms://alias/LukeTesting?region=us-west-2` or `azurekeyvault://mykeyvaultname.vault.azure.net/keys/mykeyname`.

Also like our cloud storage backend support, this PR doesn't solve for how to configure the cloud provider client used to resolve the URIs above - the standard ambient credentials are used in both cases. Eventually, we will likely need to provide ways for both of these features to be configured independently of each other and of the providers used for resource provisioning.
2019-08-02 16:12:16 -07:00
Chris Smith 17ee050abe
Refactor the way secrets managers are provided (#3001) 2019-08-01 10:33:52 -07:00
CyrusNajmabadi c30b8a509e
Always print out the last diagnostics we got. Unless we had an error and we're in the tree-view. (#3011) 2019-08-01 10:21:47 -07:00
Mikhail Shilkov 653501758d Retry renaming a temporary folder during plugin installation (#3008)
A workaround for #2695

During the plugin installation, we create a temporary folder, unzip the binary, and then rename the folder to a permanent name. The rename fails 90% of the time with access denied. An immediate retry of renaming seems to always succeed.
2019-07-31 17:44:26 -07:00
Mikhail Shilkov d7a3987f47 Prevent poorly formatted strings in fatal error messages (#2995)
Fixes #1511
2019-07-31 17:42:59 -07:00
Pat Gavlin 6804d640fc Only parse flags if they are not already parsed.
Just what it says on the tin.
2019-07-31 11:29:00 -07:00
Pat Gavlin 1af7449f1a
Remove references to pulumi/glog. (#3009)
This package's flags conflict with those in google/glog. Replace all
references to this package with references to
pulumi/pulumi/pkg/util/logging, and change that package to explicitly
call `flag.CommandLine.Parse` with an empty slice.

This should make it much easier to consume these packages in downstream
repos that have direct or indirect dependencies on google/glog.
2019-07-31 13:23:33 -05:00
Pat Gavlin 67ec74bdc5
Pass ignoreChanges to providers. (#3005)
These changes add support for passing `ignoreChanges` paths to resource
providers. This is intended to accommodate providers that perform diffs
between resource inputs and resource state (e.g. all Terraform-based
providers, the k8s provider when using API server dry-runs). These paths
are specified using the same syntax as the paths used in detailed diffs.

In addition to passing these paths to providers, the existing support
for `ignoreChanges` in inputs has been extended to accept paths rather
than top-level keys. It is an error to specify a path that is missing
one or more component in the old or new inputs.

Fixes #2936, #2663.
2019-07-31 11:39:07 -05:00
CyrusNajmabadi 27113eb6a9
Rewrite all rows when screen height changes (#2990) 2019-07-26 11:36:20 -07:00
Luke Hoban 7854227fcf Correctly calculate number of changes (#2985)
We were reporting the number of kinds of changes instead of the actual number of changes.

Fixes #2706
2019-07-25 22:40:08 -07:00
Alex Clemmer c4b3745961 Signal policy publish operation completion to the service
Fixes pulumi/pulumi-policy#32
2019-07-25 15:21:07 -07:00
James Nugent 7f6a6501ef Depend on pulumi fork of glog.
This removes the need for a replace directive in every downstream `go.mod`.
2019-07-25 16:10:53 -05:00
James Nugent 23c5a2ffb2
Merge pull request #2981 from pulumi/jen20/replace-gotty
Depend on the `ijc` fork of Gotty
2019-07-25 16:04:15 -05:00
James Nugent 8fed68f2b0 Depend on ijc fork of Gotty
This removes the need for a replace directive in every downstream `go.mod`,
however it does not protect against the case of a `go.mod` being added upstream
with a different import path in future. This seems unlikely given the cadence of
work upstream, however.
2019-07-25 15:37:23 -05:00
Pat Gavlin c6916051f0
Use a bag for misc. resource options in deploytest (#2977)
Most of these options are typically left unset. In order to make it
easier to update the lifecycle test when adding new options, collect
them in a bag s.t. most callsites can go without being updated.
2019-07-25 11:18:40 -07:00
Chris Smith b0055787e6
Fix issue when logging out of local backend (#2951)
* Fix issue when logging out of local backend

* Put actual cloud URL into credentials.json

* Fix test on Windows; disable newly-enabled tests on Appveyor

* Keep track of the URL used to initialize local backend
2019-07-25 07:58:19 -07:00