This change implements recursive closure captures. This permits
cases like the following
{
function f() { g(); }
function g() { f(); }
}
and the slightly more useful
class C {
this.x = 42;
this.f = () => x;
}
To do this requires caching the environment objects and permitting
cycles in the resulting environment graph. The closure emitter code
already knows how to handle this.
In addition, we must mark captures of `this` as free variables.
This resolvespulumi/pulumi-fabric#333.
This ensures RPC channels stay alive until logs finish. It also
makes provisions for logs that come in *after* shutdown has begun,
but before it has finished, by observing that the keepalive promise
has changed between the time of initiating the callback and running it.
* Initialize the diganostics logger with opts.Debug when doing
a Deploy, like we do Plan.
* Don't spew leaked promises if there were Log.errors.
* Serialize logging RPC calls so that they can't appear out of order.
* Print stack traces in more places and, in particular, remember
the original context for any errors that may occur asynchronously,
like resource registration and calls to mapValue.
* Include origin stack traces generally in more error messages.
* Add some more mapValue test cases.
* Only undefined-propagate mapValue values during dry-runs.
This change serializes all resource operations. Please see
pulumi/pulumi#335 for more details. In a nutshell, there are
resources that have implicit hidden dependencies and now that
the runtime is fully asynchronous, we are tripping over problems
left and right (even worse, they are non-deterministic). All
of the problems have been in the AWS API Gateway resources;
until we come up with a holistic solution here, serializing all
calls should make things more stable in the interim.
This change upgrades gRPC to 1.6.0 to pick up a few bug fixes.
We also use the full address for gRPC endpoints, including the
interface name, as otherwise we pick the wrong interface on Linux.
There's a fair bit of clean up in here, but the meat is:
* Allocate the language runtime gRPC client connection on the
goroutine that will use it; this eliminates race conditions.
* The biggie: there *appears* to be a bug in gRPC's implementation
on Linux, where it doesn't implement WaitForReady properly. The
behavior I'm observing is that RPC calls will not retry as they
are supposed to, but will instead spuriously fail during the RPC
startup. To work around this, I've added manual retry logic in
the shared plugin creation function so that we won't even try
to use the client connection until it is in a well-known state.
pulumi/pulumi-fabric#337 tracks getting to the bottom of this and,
ideally, removing the work around.
The other minor things are:
* Separate run.js into its own module, so it doesn't include
index.js and do a bunch of random stuff it shouldn't be doing.
* Allow run.js to be invoked without a --monitor. This makes
testing just the run part of invocation easier (including
config, which turned out to be super useful as I was debugging).
* Tidy up some messages.
The change to tear down RPC connections after the program exits --
to fix problems on Linux presumably due to the way libuv is implemented --
unfortunately introduces nondeterminism and overzealous termination that
can happen at inopportune times. Instead, we need to wait for the current
RPC queue to drain. To fix this, we'll maintain a list of currently active
RPC calls and, only once they have completed, will we close the clients.
This change closes the gRPC client connections, as they keep the
Node.js message loop alive on Linux (but, strangely, not Mac;
regardless, a good thing to do anyway...)
We have an issue in the runtime right now where we serialize closures
asynchronously, meaning we make it possible to form cycles between
resource graphs (something that ought to be impossible in our model,
where resources are "immutable" after creation and cannot form cycles).
Let me tell you a tale of debugging this ...
Well, no, let's not do that. But thankfully I've left behind some
little utilities that might make debugging such a thing easier down
the road. Namely:
* By default, most of our core runtime promises leverage a leak handler
that will log an error message should the process exit with certain
critical unresolved promises. This error message will include some
handy context (like whether it was an input promise) as well as a
stack trace for its point of creation.
* Optionally, with a flag in runtime/debuggable.ts, you may wire up
a hang detector, for situations where we may want to detect this
situation sooner than process exit, using the regular message loop.
This uses a defined timeout, prints the same diagnostics as the
leak detector when a hang is detected, and is disabled by default.
This fixes a few problems with dependent resolutions and hardens
even more promises-related error paths, so we swallow precisely zero
errors (or at least we hope so). This also digs through multi-level
chains of promises and computed properties as needed for nested mapValues.
This change adds support for awaiting any Computed<T> and Promise<T>s
that were captured inside of a function's closure. This preserves our
ability to capture, for example, resource state that ends up getting
serialized as the final resource state, rather than a snapshot of the
(mostly unresolved) resource state at the time of serialization.
This change moves the environment entry serialization logic into
JavaScript, where it's a bit easier to author and maintain. We
also switch to using Object.keys, so that we only walk the enumerable
properties of objects (to avoid internal member functions and to
generally leverage our current style of writing code). This is
just a temporary stopgap until we figure out more rigorous semantics
for what it means to serialize entire objects ...
* Use `global.hasOwnProperty(ident)`, rather than `global[ident] !== undefined`,
to avoid classifying references to globals as free variables. Surprise(!!),
the prior logic wouldn't work for `undefined` itself... 😒
* Expand this check to include the built-in Node.js module variables, namely
`__dirname`, `__filename`, `exports`, `module`, and `require`, so that
references to them don't get classified as serializable free variables either.
* Place catch variables in scope, so that `catch (err) { ... }` won't yield
free variables for references to `err` within `...`.
* Place recursive function definitions into the top-level `var`-like scope of
variables so that we don't consider references to them free.
* Harden all error pathways in the native C++ add-on so that we terminate
anytime an exception is in-flight, rather than limping along and making
things worse...
As I started rolling this out, I realized that end user code actually
has to use this type sometimes. And that the current names are inconsistent,
after eschewing Property<T> in favor of Computed<T>. The new names read better.
This change makes a few simplifications to how properties are exposed in
the system, mostly in the name of usability, but also to feel a bit more
like "idiomatic JavaScript". Namely:
* Rename `then` to `mapValue`. This hopefully helps to suggest that this
is meant for a dataflow style of programming.
* Move Property<T> into the runtime module, and remove PropertyState<T>,
collapsing back down to a single type. This also eliminates some of the
messy internal runtime casting, accessing of internal members, etc.
* Export a Computed<T> interface from the root of the module. This is
the entirety of the public-facing surface area for properties, and
exposes that single `mapValue` member function. The internal runtime
logic understands how to handle Property<T>s specifically in addition
to Computed<T>s more generally (in case someone writes their own).
If a resource's planning operation is to do nothing, we can safely
assume that all of its properties are stable. This can be used during
planning to avoid cascading updates that we know will never happen.
This changes a few aspects of resource properties:
* Move all runtime-related goo into the runtime module, in an
internal PromiseState class. This encapsulates the internal
state transitions and protects against misuse. It also allows
us to clean up the public API for the Property<T> type so that
it's entirely suitable for external usage.
* Track input and output property values distinctly. It turns
out we want to key off events differently. For example, to marshal
property values to a resource provider, we only care about the
inputs. For final property values that are used in, say, thens
or as inputs to other properties, we want the output property value.
* Be more precise about when an output is truly final, and known, or
unknown due to planning/dry-runs. Note that this does mean that
we'll encounter unknown values more frequently because, aside from
IDs and URNs, we can't say for sure that arbitrary properties will never
change post-creation. We have ideas on how to denote this; see
pulumi/pulumi-fabric#330 for more details.
This change renames String, File, and Remote to StringAsset, FileAsset,
and RemoteAsset, largely to avoid conflicting with the built-in JavaScript
String type, but also because it mirrors our Archive naming strategy.
This fixes a few things in the SDK preventing deployments (versus plans):
* Don't fully resolve when a link resolves. This will be handled during
the final completion of the resource state.
* Skip "id" and "urn" for property resolution, since they are handled
explicitly based on the RPC messages. The "id" is often in the response
payload because Terraform stores it as a property. We don't need it.
* Lazily allocate Property<T> objects if necessary when the response
from the resulting resource operation comes back.
* Improve a few error messages.
This change adds the ability to perform runtime logging, including
debug logging, that wires up to the Pulumi Fabric engine in the usual
ways. Most stdout/stderr will automatically go to the right place,
but this lets us add some debug tracing in the implementation of the
runtime itself (and should come in handy in other places, like perhaps
the Pulumi Framework and even low-level end-user code).
We don't actually need to copy the headers, becasue the include
path order for the GYP-generated project files will include them
in the correct order. This simplifies the script and ordering.
This change adds a `make configure` target, which handles preparing
the environment for building the project. This includes existing
steps, like dep ensure and yarn installing the Node.js SDK NPM
dependencies, and also includes downloading the right Node.js/V8
includes, putting them in the right place, and then generating the
appropriate node-gyp project files that reference those includes.
