Commit graph

4040 commits

Author SHA1 Message Date
Alex Clemmer 9c16485152 Don't require a stack to run pulumi query 2019-10-23 15:14:56 -07:00
Matthieu Leclercq 877861eb7e Export GOOGLE_APPLICATION_CREDENTIALS when google credential are given (#3379)
Allows to use that credential when the stack uses Google KMS encryption
provider.
2019-10-23 13:38:00 -07:00
CyrusNajmabadi d88944b268
Fix issue where we tore down our sync-rpc channels in expected recoverable scenarios. (#3387) 2019-10-22 14:13:07 -07:00
David McKay d9f363ea72 fix: ensure plugins are available before Pulumi command (#3228)
This patch ensures, if `package.json` exists, that `npm install` is
always run prior to any pulumi command. This will trigger any scripts
that install plugins; ensuring most CI systems can build Pulumi projects
without hacky workarounds.
2019-10-22 08:11:37 -07:00
Pat Gavlin f14eba46b5
Add a URN validation method. (#3386)
This method can be used to check whether or not a URN is well-formed.
This is used by the provider reference parser to avoid panicking on
malformed URNs.
2019-10-21 19:09:39 -07:00
Pat Gavlin 76a0079641
Fix two colorization issues. (#3385)
- The length of the text content (i.e. the content of a colorized string
  that is not control sequences) was not being correctly tracked. This
  caused the "status" column of the progress display to overflow.
- Colorization was unconditionally disabled on Windows. When we were
  using loreley, we had set the global colorization flag s.t.
  colorization on Windows _should_ have been disabled, but we overrode
  this flag each time we actually colorized anything.

Fixes #3378.
2019-10-21 16:05:45 -07:00
Ryan Campbell 665b4caa89 Update python FileAsset to accept os.PathLike in addition to str. (#3368)
This should fix #2896.
2019-10-18 14:31:59 -07:00
Pat Gavlin fa20d88e12 Update the CHANGELOG for 1.3.4 2019-10-18 13:47:35 -07:00
CyrusNajmabadi da1f27d3ab Remove errant console logging. (#3376) 2019-10-18 13:02:53 -07:00
CyrusNajmabadi 9f00e95e87
Remove unnecessary casts (#3367) 2019-10-17 17:12:45 -07:00
Pat Gavlin 1ae6629d81 Prepare the CHANGELOG for v1.3.3 2019-10-17 15:10:30 -07:00
Pat Gavlin d84e5340c2
Fix a bug in the Brew version check. (#3360)
It is not a fatal error for the brew version check to fail. In this
case, just use the values returned by the service.
2019-10-16 19:18:04 -07:00
CyrusNajmabadi b1f20115cf
Properly handle providers with unknown Ids (#3357) 2019-10-16 15:19:43 -07:00
Paul Stack 795f141311
Update t.Error to t.FailNow (#3354) 2019-10-16 22:39:37 +03:00
CyrusNajmabadi 8272a48447
Update changelist. 2019-10-16 11:58:11 -07:00
Justin Van Patten 95f85a7f28
Load PulumiPolicy.yaml instead of Pulumi.yaml during installation (#3352) 2019-10-16 11:23:50 -07:00
Pat Gavlin 730fe8617e
Buffer events. (#3332)
This avoids unnecessary blocking inside pre/post-step callbacks if the
reader on the other side of the event channel is slow.

We do not use a buffered channel in the event pipe because it is
empirically less likely that the goroutine reading from a buffered
channel will be scheduled when new data is placed in the channel. In the
case of our event system in which events will not be delivered to the
service and display until the copying goroutine is scheduled, this can
lead to unacceptable delay between the send of the original event and
its output.
2019-10-15 15:47:40 -07:00
Pat Gavlin 460ff875ab
Cache account information. (#3340)
- Cache the username and last verified time associated with each logged-in
  backend
- In the HTTP backend, verify the access token explicitly at most once
  per hour

This trades off a little bit of usability for improved inner-loop
latency: if a user's API token becomes invalid less than an hour after
it was last verified, the user will see 4xx errors when attempting stack
operations rather than seeing the login prompt.
2019-10-15 15:37:57 -07:00
CyrusNajmabadi d72872c49a
Fix error message (#3344) 2019-10-15 13:29:32 -07:00
CyrusNajmabadi 91addf2feb
New approach to move us to using deasync as little as possible (and with as little impact to users as possible). (#3325) 2019-10-14 22:08:06 -07:00
Luke Hoban 893e51d0ce
Add Python resource transformations support (#3319)
Adds Python support for resource transformations aligned with the existing NodeJS support in #3174.

This PR also moves processing of transformations to earlier in the resource construction process (for both NodeJS and Python) to ensure that invariants established in the constructor cannot be violated by transformations. This change can technically be a breaking change, but given that (a) the transformations features was just released in 1.3.0 and (b) the cases where this is a breaking change are uncommon and unlikely to have been reliable anyway - it feels like a change we should make now.

Fixes #3283.
2019-10-14 19:35:00 -05:00
Pat Gavlin 14da941b0c
Standardize on Stack for backend methods. (#3336)
The current pattern of using backend.Stack values in the CLI but
accepting backend.StackReference values in backend methods leads to
frequent transitions between these types. In the case of the HTTP
backend, the transition from a StackReference to a Stack requires an API
call. These changes refactor the backend.Backend API such that most of
its methods accept backend.Stack values in place of
backend.StackReference values, which avoids these transitions.

This removes two calls to the getStack API on the startup path of a
`pulumi preview`.
2019-10-14 14:30:42 -07:00
Paul Stack 756534865e
appveyor: Bump go version to 1.12.9 for builds to match travis (#3312) 2019-10-14 21:33:30 +03:00
Pat Gavlin d18b59e9c6
Fix a dependency graph bug during DBR. (#3329)
The dependency graph used to determine the set of resources that
depend on a resource being DBR'd is constructured from the list of
resource states present in the old snapshot. However, the dependencies
of resources that are present in both the old snapshot and the current
plan can be different, which in turn can cause the engine to make
incorrect decisions during DBR with respect to which resources need to
be replaced. For example, consider the following program:

```
var resA = new Resource("a", {dbr: "foo"});
var resB = new Resource("b", {dbr: resA.prop});
```

If this program is then changed to:
```
var resB = new Resource("b", {dbr: "<literal value of resA.prop>"});
var resA = new Resource("a", {dbr: "bar"});
```

The engine will first decide to make no changes to "b", as its input
property values have not changed. "b" has changed, however, such that it
no longer has a dependency on "a".

The engine will then decide to DBR "a". In the process, it will
determine that it first needs to delete "b", because the state for "b"
that is used when calculating "a"'s dependents does not reflect the
changes made during the plan.

To fix this issue, we rely on the observation that dependents can only
have been _removed_ from the base dependency graph: for a dependent to
have been added, it would have had to have been registered prior to the
root--a resource it depends on--which is not a valid operation. This
means that any resources that depend on the root must not yet have
been registered, which in turn implies that resources that have already
been registered must not depend on the root. Thus, we ignore these
resources if they are encountered while walking the old dependency graph
to determine the set of dependents.
2019-10-12 17:22:13 -07:00
Pat Gavlin 2c67288189
Stop using loreley. (#3331)
Loreley's approach to colorization via Go templates has significant
overhead when colorization is used extensively.

Also, stop parsing colorized text with regexes and use the actual
parser.
2019-10-12 13:53:57 -07:00
Paul Stack a30d66557e
Examples: Change the examples to run as individual tests (#3328) 2019-10-12 20:07:08 +03:00
Pat Gavlin 97803a6591 Prepare the CHANGELOG for 1.3.1. 2019-10-10 10:33:50 -07:00
Pat Gavlin 834e583c95
Revert "Propagate inputs to outputs during preview. (#3245)" (#3324)
This reverts commit 80504bf0bc.
2019-10-10 10:33:05 -07:00
Alex Clemmer 5e1c4d31c6 Use PulumiPolicy.yaml instead of Pulumi.yaml for PolicyPacks 2019-10-10 10:15:51 -07:00
CyrusNajmabadi 1574f6f9c2
Enable some tests that weren't actually running (#3320) 2019-10-09 20:16:16 -07:00
Chris Smith bec608d0d3
Improve error message when using PAC on out-of-date SDK (#3307)
* Fix lint warnings

* Improve error message when using PAC on old SDK

* Update pkg/resource/plugin/analyzer_plugin.go

Co-Authored-By: Justin Van Patten <jvp@justinvp.com>
2019-10-09 13:51:10 -07:00
Chris Smith 0e52e965db
Add local policy packs to summary event (#3308) 2019-10-09 13:50:28 -07:00
Pat Gavlin 7778d7ae84 Prepare the CHANGELOG for 1.3.0. 2019-10-09 11:50:21 -07:00
Ken Fukuyama 0700e6fa05 Fix invalid bullet points in stack init (#3303)
This fixes the issue where the bullet points aren't rendered correctly
by hugo.

Signed-off-by: Ken Fukuyama <kenfdev@gmail.com>
2019-10-08 15:07:03 -07:00
Mikhail Shilkov 69743fe2bf Make the path to .pulumi folder configurable with an ENV variable (#3300)
Introduces `PULUMI_HOME` environment variable which points to a path to the path to `.pulumi` folder. Defaults to `<user's home dir> + ".pulumi"` if not specified.

Fixes #2966. In addition to plugins, it "moves" the credentials file, templates, workspaces.

`bin` folder is intact: to move it, we need to adjust all installation scripts to respect `PULUMI_HOME` and put executables in the proper `bin` folder.
2019-10-08 15:01:46 -07:00
Pat Gavlin 22799879e8 Fix the CHANGELOG. 2019-10-08 14:50:00 -07:00
Mikhail Shilkov 0b7cc93711 Run all template tests in parallel (#3296)
* Run all template tests in parallel

* Revert Makefile
2019-10-07 16:47:21 +03:00
Mikhail Shilkov 90e111ba73 Override all options in ProgramTest.With (#3264) 2019-10-07 14:11:36 +03:00
Chris Smith 97d769000d
Support renaming stack projects (#3292)
* Support renaming stack projects

* Update CHANGELOG.md

* Address PR feedback
2019-10-03 09:13:13 -07:00
CyrusNajmabadi e019e12469
Perform our closure tree-shaking when the code contains element accesses, not just property accesses (#3295) 2019-10-02 23:34:09 -07:00
Mikhail Shilkov 96295ce322 Add -v option to go test of templates to avoid stalling in TravisCI 2019-10-03 08:28:58 +02:00
Pat Gavlin 23b7a3f337
Add scripting for updating the Homebrew formula. (#3291)
This script will run during OS X legs, and will open a PR to update the
Homebrew formula for Pulumi on each new release.
2019-10-02 16:04:58 -07:00
Pat Gavlin ecd028104c
Fetch CLI verisons from Homebrew where applicable. (#3290)
If the CLI seems to have been installed using Homebrew, do not consult
the service for the latest version. Instead, consult the Homebrew JSON
API.

Fixes #3230.
2019-10-01 13:34:42 -07:00
Pat Gavlin 5ad307d457
Close test language runtime connections. (#3289)
This should help address the issues we've begun seeing with too many
open files when running tests on OS X.
2019-10-01 12:12:06 -07:00
CyrusNajmabadi 52884096e9
Add support for updating a subset of resources in the stack (i.e. --target) (#3251) 2019-09-30 23:41:56 -07:00
Alex Clemmer f7f4333909 Expose queryable.Resolved<T> publicly
Fixes pulumi/pulumi-policy#92.
2019-09-30 16:49:11 -07:00
Alex Clemmer 429bde332b Print PolicyPacks run as part of update summary
Fixes pulumi/pulumi-policy#69.
2019-09-30 14:49:21 -07:00
Pat Gavlin 80504bf0bc
Propagate inputs to outputs during preview. (#3245)
These changes restore a more-correct version of the behavior that was
disabled with #3014. The original implementation of this behavior was
done in the SDKs, which do not have access to the complete inputs for a
resource (in particular, default values filled in by the provider during
`Check` are not exposed to the SDK). This lack of information meant that
the resolved output values could disagree with the typings present in
a provider SDK. Exacerbating this problem was the fact that unknown
values were dropped entirely, causing `undefined` values to appear in
unexpected places.

By doing this in the engine and allowing unknown values to be
represented in a first-class manner in the SDK, we can attack both of
these issues.

Although this behavior is not _strictly_ consistent with respect to the
resource model--in an update, a resource's output properties will come
from its provider and may differ from its input properties--this
behavior was present in the product for a fairly long time without
significant issues. In the future, we may be able to improve the
accuracy of resource outputs during a preview by allowing the provider
to dry-run CRUD operations and return partially-known values where
possible.

These changes also introduce new APIs in the Node and Python SDKs
that work with unknown values in a first-class fashion:
- A new parameter to the `apply` function that indicates that the
  callback should be run even if the result of the apply contains
  unknown values
- `containsUnknowns` and `isUnknown`, which return true if a value
  either contains nested unknown values or is exactly an unknown value
- The `Unknown` type, which represents unknown values

The primary use case for these APIs is to allow nested, properties with
known values to be accessed via the lifted property accessor even when
the containing property is not fully know. A common example of this
pattern is the `metadata.name` property of a Kubernetes `Namespace`
object: while other properties of the `metadata` bag may be unknown,
`name` is often known. These APIs allow `ns.metadata.name` to return a
known value in this case.

In order to avoid exposing downlevel SDKs to unknown values--a change
which could break user code by exposing it to unexpected values--a
language SDK must indicate whether or not it supports first-class
unknown values as part of each `RegisterResourceRequest`.

These changes also allow us to avoid breaking user code with the new
behavior introduced by the prior commit.

Fixes #3190.
2019-09-30 11:03:58 -07:00
Chris Smith 7254b3db4d
Remove vestigial field from apitype (#3284) 2019-09-30 08:33:47 -07:00
Luke Hoban 9374c374c3
Transformations (#3174)
Adds the ability to provide `transformations` to modify the properties and resource options that will be used for any child resource of a component or stack.

This offers an "escape hatch" to modify the behaviour of a component by peeking behind it's abstraction.  For example, it can be used to add a resource option (`additionalSecretOutputs`, `aliases`, `protect`, etc.) to a specific known child of a component, or to modify some input property to a child resource if the component does not (yet) expose the ability to control that input directly.  It could also be used for more interesting scenarios - such as:
1. Automatically applying tags to all resources that support them in a stack (or component)
2. Injecting real dependencies between stringly-referenced  resources in a Helm Chart 
3. Injecting explicit names using a preferred naming convention across all resources in a stack
4. Injecting `import` onto all resources by doing a lookup into a name=>id mapping

Because this feature makes it possible to peek behind a component abstraction, it must be used with care in cases where the component is versioned independently of the use of transformations.  Also, this can result in "spooky action at a distance", so should be used judiciously.  That said - this can be used as an escape hatch to unblock a wide variety of common use cases without waiting on changes to be made in a component implementation.  

Each transformation is passed the `resource`, `name`, `type`, `props` and `opts` that are passed into the `Resource` constructor for any resource descended from the resource that has the transformation applied.  The transformation callback can optionally return alternate versions of the `props` and `opts` to be used in place of the original values provided to the resource constructor.

Fixes #2068.
2019-09-29 11:27:37 -07:00