Commit graph

431 commits

Author SHA1 Message Date
Pat Gavlin 6e90ab0341
Add support for explicit delete-before-replace (#2415)
These changes add a new flag to the various `ResourceOptions` types that
indicates that a resource should be deleted before it is replaced, even
if the provider does not require this behavior. The usual
delete-before-replace cascade semantics apply.

Fixes #1620.
2019-01-31 14:27:53 -08:00
Matt Ellis a627cf817a Ensure InvokeRequest is defined
InvokeRequest is defined in provider_pb.js not resource_pb.js. Require
that module instead.
2019-01-31 12:00:32 -08:00
Sean Gillespie 1d5526d292
Work around commonjs protoc bug (#2403)
* Work around commonjs protoc bug

When compiling with the commonjs target, the protoc compiler still emits
references to Closure Compiler-isms that whack global state onto the
global object. This is particularly bad for us since we expect to be
able to make backwards-compatible changes to our Protobuf definitions
without breaking things, and this bug makes it impossible to do so.

To remedy the bug, this commit hacks the output of protoc (again) with
sed in order to avoid ever touching the global object. Everything still
works fine because the commonjs target (correctly) exports the protobuf
message types via the module system - it's just not writing to global
anymore.

* Fix status.proto

* Don't hack status.proto
2019-01-29 17:07:47 -08:00
Pat Gavlin 35c60d61eb
Follow up on #2369 (#2397)
- Add support for per-property dependencies to the Go SDK
- Add tests for first-class secret rejection in the checkpoint and RPC
  layers and language SDKs
2019-01-28 17:38:16 -08:00
Pat Gavlin 1ecdc83a33 Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.

We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.

This is perhaps clearer when described by example. Consider the
following dependency graph:

  A
__|__
B   C
|  _|_
D  E F

In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.

In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 09:46:30 -08:00
CyrusNajmabadi edd742a0c4
Allow derived classes to call registerOutputs with no args. (#2393) 2019-01-26 17:59:11 -08:00
CyrusNajmabadi bfcb07a164
Add functions to help make strings from values that may include Outputs (#2359) 2019-01-16 11:03:25 -08:00
Pat Gavlin b85f95acd9
Fix error codes for early exceptions (#2337)
It is possible for the sub-process responsible for running a NodeJS
Pulumi program to exit with a success code before the user's program has
run if the process of loading the runtime generates an unhandled promise
rejection. These changes fix this by registering the unhandled exception
and rejection handlers that are responsible for ensuring a non-zero exit
code in these cases before any other action is taken.

Note that this issue is really only possible because the Node language
host (like the Python language host) is composed of two processes: one
that serves the language host gRPC service and one that loads and runs
the user's program. The former launches the latter in response to a call
to its `Run` gRPC endpoint. The lifetime of the user's program is
considered to be bounded by the lifetime of the `Run` invocation. The
NodeJS process maintains its own connection to the engine over which
resource registrations are communicated. It is tempting to add a message
to the resource monitor RPC interface that signals that no further
registrations are performed, but this is complicated due to the
three-party topology and the possibility that such an RPC may never be
sent (e.g. due to a crash or a downlevel version of the Pulumi Node
runtime).

Fixes #2316.
2019-01-07 09:59:29 -08:00
Pat Gavlin 1a651ab0e0
Always read StackReference resources (#2332)
This is something of a quick hack to work around the limitations that
are at the root of #2310. In short, `StackReference` resources do not
change during a update because their inputs have not changed and we do
not refresh resources as part of the update by default. We want
`StackReference` to remain a resource for myriad reasons (not the least
of which is to avoid a breaking change), but it does seem correct for it
to refresh its state during each update. Because there is no actual CRUD
operation associated with a `StackReferece`, we can obtain this behavior
by changing the implementation of its ctor in the various SDKs s.t. its
options bag always contains an appropriate `id`, thus indicating a
`Read`.

Fixes #2310.
2019-01-03 16:06:21 -08:00
CyrusNajmabadi 87e5a441f5
Convert resource to pojo objects when used as stack outputs. (#2311) 2019-01-03 10:03:11 -08:00
CyrusNajmabadi e86ea70ea0
Force rebuild of htis package to wipe out previously published 'dev' package. (#2300) 2018-12-17 16:00:31 -08:00
CyrusNajmabadi 7e7544e27e
Revert https://github.com/pulumi/pulumi/pull/2281 (#2297) 2018-12-15 21:04:53 -08:00
CyrusNajmabadi 2d92b516b2
Do not pass component resource properties to the engine. (#2296) 2018-12-15 15:32:19 -08:00
CyrusNajmabadi b94cfb22d5
Handle cycles in component resources more simply. (#2283) 2018-12-11 13:42:49 -08:00
CyrusNajmabadi 05f5311055
Prevent cycles when serializing props. (#2281) 2018-12-10 14:31:54 -08:00
Matt Ellis d7575072f2 Do not use relative path when launching dynamic provider
Previously, we assumed that the dynamic provider was located in
`./node_modules/@pulumi/pulumi/../` which is correct in the majority
of cases. However, tools like lerna or yarn workspaces (or custom
workflows) allow the node_modules folder to be located elsewhere on
disk, and node will still find it because of its algorithm for module
resolution.

So, do what we do in the language host itself, first launch node and
ask it to tell us where it resolves a require statement to on disk and
then launch node against that script.

Fixes #2261
2018-12-05 13:45:06 -08:00
Pat Gavlin 663e5342b0
Inherit custom resource providers (#2264)
If a custom resource has explicitly specified a provider, add that
provider to the resource's provider map under the resource's package.
This allows children of the custom resource to inherit the resource's
provider.

Fixes #2262.
2018-12-05 12:35:19 -08:00
Pat Gavlin 95db6439f5
Add stackReference.ts to tsconfig.json (#2253)
Otherwise this file is not picked up by typedoc.
2018-11-29 10:27:29 -08:00
CyrusNajmabadi e53f746c35
Export our 'unwrap' types so that downstream packages can create combinators that TS will accept. (#2252) 2018-11-28 13:33:33 -08:00
CyrusNajmabadi 9a044ff865
make the context-param non-optional for debuggable promises. (#2242) 2018-11-24 18:57:17 -08:00
Sean Gillespie 03e70a188b
Flush stdout and stderr on abnormal lang shutdown (#2226)
The langhost shares its standard out and standard error with the
language executor that it is used (python/nodejs), so we must be sure to
flush our stdout and stderr before reporting a Run failure to the
engine.
2018-11-19 17:59:01 -05:00
Joe Duffy a337fd7379
Permit dependsOn that are Outputs (of Outputs) (#2198)
This changes the input type for dependsOn from simply
`Resource[] | Resource` to `Input<Input<Resource>[]> | Input<Resource>`.
This permits `Output<Resource>`s, etc in addition to
`Promise<Resource>`s. The logic for dynamically unpicking the right
types and recursing through the data structures isn't straightforward,
but I've written a test for all of the interesting permutations.

This fixes pulumi/pulumi#991.
2018-11-19 11:22:55 -05:00
Pat Gavlin bc08574136
Add an API for importing stack outputs (#2180)
These changes add a new resource to the Pulumi SDK,
`pulumi.StackReference`, that represents a reference to another stack.
This resource has an output property, `outputs`, that contains the
complete set of outputs for the referenced stack. The Pulumi account
performing the deployment that creates a `StackReference`  must have
access to the referenced stack or the call will fail.

This resource is implemented by a builtin provider managed by the engine.
This provider will be used for any custom resources and invokes inside
the `pulumi:pulumi` module. Currently this provider supports only the
`pulumi:pulumi:StackReference` resource.

Fixes #109.
2018-11-14 13:33:35 -08:00
CyrusNajmabadi cea165615e
Reenable tests. (#2200) 2018-11-13 17:13:27 -08:00
Matt Ellis d000896a49 Bump @types/node dependency 2018-11-13 11:20:21 -08:00
Matt Ellis 22fef07fcf Remove existing lock files 2018-11-12 15:33:58 -08:00
Joe Duffy 1e9f5b96f2
Merge pull request #2164 from pulumi/ellismg/update-tools
Update to Go 1.11 and golangci-lint
2018-11-10 17:32:14 -08:00
Pat Gavlin 170541546e
Disable the config capture tests (#2183)
These tests fail non-deterministically.
2018-11-08 17:45:51 -08:00
Matt Ellis 6c7092ff65 Pass -count=1 to disable result caching on go 1.10 and above 2018-11-08 14:11:52 -08:00
Matt Ellis 992b048dbf Adopt golangci-lint and address issues
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
2018-11-08 14:11:47 -08:00
CyrusNajmabadi 08fc305b7f
Prevent resource registration from overwriting unknown properties at random points in the future. (#2176) 2018-11-07 20:24:16 -08:00
CyrusNajmabadi 901a238fd5
Get closure serialiation working in Node11 (#2101)
* Make v8 primitives async as there is no way to avoid async in node11.

* Simplify API.

* Move processing of well-known globals into the v8 layer.
We'll need this so that we can map from RemoteObjectIds back to these well known values.

* Remove unnecesssary check.

* Cleanup comments and extract helper.

* Introduce helper bridge method for the simple case of making an entry for a string.

* Make functions async.  They'll need to be async once we move to the Inspector api.

* Make functions async.  They'll need to be async once we move to the Inspector api.

* Make functions async.  They'll need to be async once we move to the Inspector api.

* Move property access behind helpers so they can move to the Inspector API in the future.

* Only call function when we know we have a Function.  Remove redundant null check.

* Properly serialize certain special JavaScript number values that JSON serialization cannot handle.

* Only marshall across the 'source' and 'flags' for a RegExp when serializing.

* Add a simple test to validate a regex without flags.

* Extract functionality into helper method.

* Add test with complex output scenarios.

* Output serialization needs to avoid recursively trying to serialize a serialized value.

* Introduce indirection for introspecting properties of an object.

* Use our own introspection API for examining an Array.

* Hide direct property access through API indirection.

* Produce values like the v8 Inspector does.

* Compute the module map asynchronously.  Will need that when mapping mirrors instead.

* Cleanup a little code in closure creation.

* Get serialization working on Node11 (except function locations).

* Run tests in the same order on <v11 and >=v11

* Make tests run on multiple versions of node.

* Rename file to make PR simpler to review.

* Cleanup.

* Be more careful with global state.

* Remove commented line.

* Only allow getting a session when on Node11 or above.

* Promisify methods.
2018-11-01 15:46:21 -07:00
CyrusNajmabadi c7a46848b9
Ensure method always returns a boolean. (#2120) 2018-10-28 13:02:37 -07:00
CyrusNajmabadi 3e25bc80c8
Add test with complex output scenarios. (#2118) 2018-10-28 13:02:25 -07:00
CyrusNajmabadi 7efd3ddf05
Cleanup a little code in closure creation. (#2122) 2018-10-28 13:02:04 -07:00
CyrusNajmabadi 13d9043771
Produce values like the v8 Inspector does. (#2119) 2018-10-28 01:01:48 -07:00
Joe Duffy 4647aa6cdb
Merge pull request #2114 from pulumi/joeduffy/994_combinators
Add some helpful input/output combinators
2018-10-27 15:41:22 -07:00
joeduffy c5be233f57 Revert formatting changes for Output.all 2018-10-27 13:55:54 -07:00
joeduffy 972d678388 Add an extra layer of Input-ness
CR feedback.
2018-10-27 13:47:15 -07:00
joeduffy 930c86cb50 Fix some lint errors 2018-10-27 13:27:04 -07:00
joeduffy 1daad93ccd Add tests for the new combinators 2018-10-27 12:56:16 -07:00
joeduffy d6f58eb20e Ditch the Map-based combinators in favor of objects 2018-10-27 12:39:22 -07:00
joeduffy 724ae41e1b Add a toObject combinator
This function works like toMap does, except that it returns an
output object with keys and values derived from input values.
2018-10-27 12:31:31 -07:00
joeduffy b261908884 Move combinators to an iterable module
Rather than placing these combinators directly on the Output class,
which feels odd because they are special purpose to iterables, and deal
with not only Outputs but also Inputs, we will place them on a
separate and dedicated iterable module for these utility helpers.
2018-10-27 12:19:42 -07:00
CyrusNajmabadi c83eaf428b
Add a simple test to validate a regex without flags. (#2117) 2018-10-27 00:42:52 -07:00
CyrusNajmabadi 1d7f35352d
Properly serialize certain special JavaScript number values that JSON serialization cannot handle. (#2116) 2018-10-26 23:46:57 -07:00
joeduffy d1d2ceaedd Add create, createMap, and createGroupByMap combinators
This change adds some new constructors for output properties:

1) We alias `Output.create` to `output`, more like Promise's various
   construction methods. This reads better and is more discoverable.

2) A new `Output.createMap` function will accept an array of inputs,
   along with a selector function for key/value pairs, and produces
   an output map with said keys and values inside of it.

3) A new `Output.createGroupByMap` functon will similarly accept an
   array of inputs and a key/value selector, however it creates an
   output map with said keys, but where values are arrays of values,
   and all duplicate keys will lead to appending to said arrays.

Tests to come in a subsequent checkin.
2018-10-26 20:49:16 -07:00
Matt Ellis e84eab0d9f Add Node 11 scouting build 2018-10-25 19:07:04 -07:00
Sean Gillespie ae1a2e8b7e
Fail closure serialization in Node 11 (#2098)
* Fail closure serialization in Node 11

Node 11 changed many of the intrinsics that we depend upon for closure
serialization, so until we fix the underlying issues this commit lazily
fails if a closure is serialized when running on Node 11.

* CR feedback
2018-10-25 10:55:47 -07:00
CyrusNajmabadi 19a313b628
Do not analyze user source code for required packages. We'll analyze their project.json for that. (#1612) 2018-10-18 11:21:47 -07:00