* Revert "Parallelize much more of resource creation in the JS language provider SDK (#1618)"
This reverts commit 4edd244a26.
* Revert "Process our async-work-queue in parallel. (#1619)"
This reverts commit b8c1cb9574.
Closure serialization now keeps track of the `require`d packages it sees in the function bodies that are serialized during a call to `serializeFunction`.
Also, replaces `serializeFunctionAsync` with `serializeFunction` which accepts richer parameters and return type, deprecating the former API (but leaving it available for now to avoid a breaking change).
Before the changes in #1414, all output properties were guaranteed to
have values after deserialization. After #1414, any properties with no
value were no longer resolved, which was treated as an error. These
changes resolve all missing proprties to `undefined`. If a property is
missing during an update, its `undefined` value is marked as known.
These changes add support for distinguishing an output property with
an unknown value from an output property with a known value that is
undefined.
In a broad sense, the Pulumi property type system is just JSON with the
addition of unknown values. Notably absent, however, are undefined
values. As it stands, our marshalers between JavaScript and Pulumi
property values treat all undefined JavaScript values as unknown Pulumi
values. Unfortunately, this conflates two very different concepts:
unknown Pulumi values are intended to represent values of output
properties that are unknown at time of preview, _not_ values that are
known but undefined. This results in difficulty reasoning about when
transforms are run on output properties as well as confusing output in
the `diff` view of Pulumi preview (user-specifed undefined values are
rendered as unknown values).
As it turns out, we already have a way to decide whether or not an
Output value is known or not: Output.performApply. These changes rename
this property to `isKnown`, clarify its meaning, and take advantage of
the result to decide whether or not an Output value should marshal as
an unknown Pulumi value.
This also allowed these changes to improve the serialization of
undefined object keys and array elements s.t. we better match JavaScript
to JSON serialization behavior (undefined object keys are omitted;
undefined array elements are marshaled as `null`).
Fixes https://github.com/pulumi/pulumi-cloud/issues/483.
* Fix a bug in promise leak detection that leaked promises when errors occur
* Add an opt-in to the super-verbose debug error message on promise leaks
* Fix a bad merge
* was/were grammar improvement in error message
* Fail the deployment if a debuggable promise leaks
* Graceful RPC shutdown: CLI side
* Handle unavailable resource monitor in language hosts
* Fix a comment
* Don't commit package-lock.json
* fix mangled pylint pragma
* Rebase against master and fix Gopkg.lock
* Code review feedback
* Fix a race between closing the callerEventsOpt channel and terminating a goroutine that writes to it
* glog -> logging
Rather than filtering out the `id` and `urn` properties when serializing
the inputs to an invoke, pass these properties along. This enables the
use of invoke endpoints that accepts these as inputs (e.g. the endpoint
that backs `aws.ec2.getSubnet`).
This change adopts `x is T` style of RTTI inquiry, which fits much
more nicely with TypeScript's typechecking flow.
Thanks to @lukehoban for teaching me a new trick today! :-)
This change moves us away from using JavaScript RTTI, by way of
`instanceof`, for built-in Pulumi types. If we use `instanceof`,
then the same logical type loaded from separate copies of the
SDK package -- as will happen in SxS scenarios -- are considered
different. This isn't actually what we want. The solution is
simple: implement our own quasi-RTTI solution, using __pulumi*
properties and manual as* and is* functions. Note that we could
have skipped the as* and is* functions, but I found that they led
to slightly easier to read code.
There is one strange thing in here, which I spoke to
@CyrusNajmabadi about: SerializedOutput<T>, because it implements
Output<T> as an _interface_, did not previously masquerade as an
actual Output<T>. In other words, `instanceof` would have returned
false, and indeed a few important properties (like promise) are
missing. This change preserves that behavior, although I'll admit
that this is slightly odd. I suspect we'll want to revisit this as
part of https://github.com/pulumi/pulumi/issues/1074.
Fixes https://github.com/pulumi/pulumi/issues/1203.
* Introduce a simple repetition operator to match expected error messages against actual ones
* Convert required and optional objects to use a Map (node v9 compat), improve the error formatting for failed tests
* Test node v6, v8, and v9 in CI
* Get rid of PULUMI_API env in .travis.yml, it's set from the Travis console now
* Implement closure scope chain analysis in pure TypeScript
This change makes use of four V8 intrinsics to avoid having to use a
native module to inspect the scope chains of live Function objects. This
unfortunately leads to the limitation of not allowing captures of 'this'
in arrow functions, but that is something we are willing to live with
for now.
* Remove native module build and restore from the Makefile
* CR feedback: Be a little more efficient when scanning the scope chain
* Nuke everything related to custom Node versions and the native Node module
* CR feedback: rename native.ts -> v8.ts, document some interfaces in v8.ts
As I began to write code using the ability to perform resource
lookups, especially in the code-generators, I realized the way it
was surfaced as an argument to the Resource base constructor would
lead to overload explosion. Instead of doing that, let's pass it
in the ResourceOptions bag.
Prior to this change, if you ended up with multiple Pulumi SDK
packages loaded side-by-side, we would fail in obscure ways. The
reason for this was that we initialize and store important state
in static variables. In the case that you load the same library
twice, however, you end up with separate copies of said statics,
which means we would be missing engine RPC addresses and so on.
This change adds the ability to recover from this situation by
mirroring the initialized state in process-wide environment
variables. By doing this, we can safely recover simply by reading
them back when we detect that they are missing. I think we can
eventually go even further here, and eliminate the entry point
launcher shim altogether by simply having the engine launch the
Node program with the right environment variables. This would
be a nice simplification to the system (fewer moving pieces).
There is still a risk that the separate copy is incompatible.
Presumably the reason for loading multiple copies is that the
NPM/Yarn version solver couldn't resolve to a shared version.
This may yield obscure failure modes should RPC interfaces change.
Figuring out what to do here is part of pulumi/pulumi#957.
This fixespulumi/pulumi#777 and pulumi/pulumi#1017.
This change skips unknown IDs during read operations. This can happen
when a read is performed using the output property of another resource
during planning. This is intentionally supported via ID being an
Input<ID> and all we need to do for this to work correctly is skip the
actual provider RPC and the runtime will propagate unknown outputs as
usual.
This change wires up the new Read RPC method in such a manner that
Pulumi programs can invoke it. This is technically not required for
refreshing state programmatically (as in pulumi/pulumi#1081), however
it's a feature we had eons ago and have wanted since (see
pulumi/pulumi#83), and will allow us to write code like
let vm = aws.ec2.Instance.get("my-vm", "i-07043cd97bd2c9cfc");
// use any property from here on out ...
The way this works is simply by bridging the Pulumi program via its
existing RPC connection to the engine, much like Invoke and
RegisterResource RPC requests already do, and then invoking the proper
resource provider in order to read the state. Note that some resources
cannot be uniquely identified by their ID alone, and so an extra
resource state bag may be provided with just those properties required.
This came almost for free (okay, not exactly) and will come in handy as
we start gaining experience with reading live state from resources.
The four concerns are:
parsing a v8 function string so we can figure out captured variables.
walkgin the function/object graph producing the graph we will serialize.
rewriting constructors and methods so that 'super' works.
serializing graph to text.
Also, rename/cleanup a bunch of serialization code.
Also, generate better environment names in the serialized closure code. Thsi code should be much easier to make sense of as hte names will better track to the original names in the user code.
Also, dedupe simple non-capturing functions. This helps ensure we don't spit out N copies of __awaiter (one per file it is declared in).
We now unify new Config("package") and new Config("package:config"),
printing a warning when the new Config("package:config") form is
used and pointing consumers towards just new Config("package")
I've updated our examples to use the newer syntax, but I've added a
test ot the langhost to ensure both forms work.
Fixes#923
Make many fixes to closure serialization
Primary things that i've done as part of this change:
Added support for cyclic objects.
Properly serialize objects that are shared across different function. previously you would get multiple copies, now you properly reference the same copy.
Remove the usages of 'hashes' for functions. Because we track identity of objects, we no longer need them.
Serialize properties of functions (if they have any).
Handle Objects/Functions with different __proto__s than normal. i.e. classes/constructors. but also anything the user may have done themselves to the object.
Handle generator functions.
Handle functions with 'computed' names.
Handle functions with 'symbol' names.
Handle serializing Promises as Promises.
Removed the dual Closure/AsyncClosure tree. One existed solely so we could have a tree without promises (for use in testing maybe?). Because this all exists in a part of our codebase that is entirely async, it's fine to have promises in the tree, and to await them when serializing the Closure to a string.
Handle serializing class-constructors and methods. Including properly handling 'super' calls.
1. Various idiomatic Go and TypeScript fixes
2. Add an integration test that end-to-end roundtrips dependency
information for a simple Pulumi program
3. Add an additional test assert that tests that dependency information
comes from the language host as expected
This commit does two things:
1. All dependencies of a resource, both implicit and explicit, are
communicated directly to the engine when registering a resource. The
engine keeps track of these dependencies and ultimately serializes
them out to the checkpoint file upon successful deployment.
2. Once a successful deployment is done, the new `pulumi stack
graph` command reads the checkpoint file and outputs the dependency
information within in the DOT format.
Keeping track of dependency information within the checkpoint file is
desirable for a number of reasons, most notably delete-before-create,
where we want to delete resources before we have created their
replacement when performing an update.
Fixes#356. Instead of downloading a node binary with our closure
serialization code linked-in, this PR instead publishes the
`nativeruntime.node` produced by the NodeJS SDK build as part of the SDK.
This has a number of advantages. First, it is vastly more easy to
develop closure.cc in this configuration. Second, we have the ability
to ship different `nativeruntime.node`s side-by-side, paving the way
for enabling future versions of Node. Third, we don't have to stay
in the business of shipping custom builds of Node, although we do still
need to ship a version of Node with minor modifications in order for
Windows to still work.
This improves the failure messages in two circumstances:
1) If the resource monitor RPC connection is missing. This can happen
two ways: either you run a Pulumi program using vanilla Node.js, instead
of the CLI, or you've accidentally loaded the Pulumi SDK more than once.
2) Failure to load the custom Pulumi SDK Node.js extension. This is a new
addition and would happen if you tried running a Pulumi program using a
vanilla Node.js, rather than using the Pulumi CLI.
* experimental: separate language host from node
* Remove langhost details from the NodeJS SDK runtime
* Cleanup
* Work around an issue where Node sometimes loads the same module twice in two different contexts, resulting in two distinct module objects. Some additional cleanup.
* Add some tests
* Fix up the Windows script
* Fix up the install scripts and Windows build
* Code review feedback
* Code review feedback: error capitalization