This fixes two closure bugs.
First, we had special cased `__awaiter` from days of yore, when we had
special cased its capture. I also think we were confused at some point
and instead of fixing the fact that we captured `this` for non-arrow
functions, which `__awaiter` would trigger, we doubled down on this
incorrect hack. This means we missed a real bonafide `this` capture.
Second, we had a global cache of captured variable objects. So, if a
free variable resolved to the same JavaScript object, it always resolved
to the first serialization of that object. This is clearly wrong if
the object had been mutated in the meantime. The cache is required to
reach a fixed point during mutually recursive captures, but we should
only be using it for the duration of a single closure serialization
call. That's precisely what this commit does.
Also add a fix for this case.
This fixespulumi/pulumi#663.
There were two problems:
- node-gyp configure was failing because of different shell syntax
between windows and *nix.
- MSVC 2015 is not smart enough to understand our use of strlen actually
results in a constant value and prevents us from using it to create an
array, move to a macro based solution.
This change moves the environment entry serialization logic into
JavaScript, where it's a bit easier to author and maintain. We
also switch to using Object.keys, so that we only walk the enumerable
properties of objects (to avoid internal member functions and to
generally leverage our current style of writing code). This is
just a temporary stopgap until we figure out more rigorous semantics
for what it means to serialize entire objects ...
* Use `global.hasOwnProperty(ident)`, rather than `global[ident] !== undefined`,
to avoid classifying references to globals as free variables. Surprise(!!),
the prior logic wouldn't work for `undefined` itself... 😒
* Expand this check to include the built-in Node.js module variables, namely
`__dirname`, `__filename`, `exports`, `module`, and `require`, so that
references to them don't get classified as serializable free variables either.
* Place catch variables in scope, so that `catch (err) { ... }` won't yield
free variables for references to `err` within `...`.
* Place recursive function definitions into the top-level `var`-like scope of
variables so that we don't consider references to them free.
* Harden all error pathways in the native C++ add-on so that we terminate
anytime an exception is in-flight, rather than limping along and making
things worse...
We don't actually need to copy the headers, becasue the include
path order for the GYP-generated project files will include them
in the correct order. This simplifies the script and ordering.
This change adds a `make configure` target, which handles preparing
the environment for building the project. This includes existing
steps, like dep ensure and yarn installing the Node.js SDK NPM
dependencies, and also includes downloading the right Node.js/V8
includes, putting them in the right place, and then generating the
appropriate node-gyp project files that reference those includes.
This change implements free variable calculations and wires it up
to closure serialization. This is recursive, in the sense that
the serializer may need to call back to fetch free variables for
nested functions encountered during serialization.
The free variable calculation works by parsing the serialized
function text and walking the AST, applying the usual scoping rules
to determine what is free. In particular, it respects nested
function boundaries, and rules around var, let, and const scoping.
We are using Acorn to perform the parsing. I'd originally gone
down the path of using V8, so that we have one consistent parser
in the game, however unfortunately neither V8's parser nor its AST
is a stable API meant for 3rd parties. Unlike the exising internal
V8 dependencies, this one got very deep very quickly, and I became
nervous about maintaining all those dependencies. Furthermore,
by doing it this way, we can write the free variable logic in
JavaScript, which means one fewer C++ component to maintain.
This also includes a fairly significant amount of testing, all
of which passes! 🎉
This change contains an initial implementation of closure serialization
built atop V8, rather than our own custom runtime. This requires that
we use a Node.js dynamic C++ module, so that we can access the V8
APIs directly. No build magic is required beyond node-gyp.
For the most part, this was straight forward, except for one part: we
have to use internal V8 APIs. This is required for two reasons:
1) We need access to the function's lexical closure environment, so
that we may look up closure variables. Although there is a
tantalizingly-close v8::Object::CreationContext, its implementation
intentionally pokes through closure contexts in order to recover
the Function constructor context instead. That's not what we
want. We want the raw, unadulterated Function::context.
2) We need to control the lexical lookups of free variables so that
they can look past chained contexts, lexical contexts, withs, and
eval-style context extensions. Simply runing a v8::Script, or
simulating an eval, doesn't do the trick. Hence, we need to access
the unexported v8::internal::Context::Lookup function.
There is a third reason which is not yet implemented: free variable
calculation. I could use Esprima, or do my own scanner for free
variables, but I'd prefer to simply use the V8 parser so that we're
using the same JavaScript parser across all components. That too
is not part of the v8.h API, so we'll need to crack it open more.
To be clear, these are still exported public APIs, in proper headers
that are distributed with both Node and V8. They simply aren't part
of the "stable" v8.h surface area. As a result, I do expect that
maintaining this will be tricky, and I'd like to keep exploring how
to do this without needing the internal dependency. For instance,
although this works with node-gyp just fine, we will probably be
brittle across versions of Node/V8, when the internal APIs might be
changing. This will introduce unfortunate versioning headaches (all,
hopefully and thankfully, caught at compile-time).
