Turns out we hadn't been transforming interface heritage clauses --
extends and implements -- like we were classes. This change fixes
that. As a result, we're down to 14 verification errors for AWS.
This change also accepts MuPackages during MuJS dependency resolution.
The old logic wasn't quite right, because it would skip over MuPackage
files, and keep probing upwards, possibly binding to the wrong, pre-
compilation, Mufile. Note that we must still probe for both Mupackages
and Mufiles, because of intra-package dependencies.
After this change, the AWS MuPackage is down to 28 verification errors.
To prepare a MuPackage for consumption by another MuPackage, we need
to save the definition files output by the TypeScript compiler. In
theory, we could regenerate these definitions from the MuPackage itself;
in fact, when we tackle cross-language, this will be necessary. For
now, however, it's much simpler to simply save the definition outputs
alongside the Mupack.json package metadata.
Note that we do not save "code" outputs. The MuIL format encodes all
computations and so the only thing we need for dependent packages is
the header files to compile against; all "link" and runtime dependencies
are satisfied by the MuIL format and Mu toolchain.
There are a variety of ways to specify a MuJS output location, including
the command line --out (-o for short) flag. If that isn't present, we simply
dump the Mupack.json file in the current working directory. However, in order
to prepare Mupackage files for packaging, distribution, and reuse, we actually
want to store the Mupack.json file alongside all the other package assets (like
TypeScript header files). So it makes sense to recognize the outDir, if present,
in the tsconfig.json file, and use it as the default if not otherwise specified.
This changes transformIdentifierExpression to call through to the
general purpose function that understands how to transform any Symbol
reference into a fully qualified token suitable for serialization.
This change recognizes identifier expressions that mention the null and
undefined constants, transforming them into null literals. Note that
marapongo/mu#70 tracks mimicking the subtle differences between
undefined and null in ECMAScript.
In the event that we're loading a property straight from a module,
we needn't actually load the module at all; instead, we refer to it
using a fully qualified token.
In many cases, property access expressions interact with types. For
example, the expression `o.j.bar()` likely accesses variable `o`, then
its property `j`, then `j`'s method `bar`. In other cases, however,
property access elements are actually modules. For example, in:
import * as o from "o";
o.j.bar();
This change recognizes this situation and emits code correctly.
This change fixes a whole host of issues with our current token binding
logic. There are two primary aspects of this change:
First, the prior token syntax was ambiguous, due to our choice of
delimiter characters. For instance, "/" could be used both as a module
member delimiter, in addition to being a valid character for sub-modules.
The result is that we could not look at a token and know for certain
which kind it is. There was also some annoyance with "." being the
delimiter for class members in addition to being the leading character
for special names like ".this", ".super", and ".ctor". Now, we just use
":" as the delimiter character for everything. The result is unambiguous.
Second, the simplistic token table lookup really doesn't work. This is
for three reasons: 1) decorated types like arrays, maps, pointers, and
functions shouldn't need token lookup in the classical sense; 2) largely
because of decorated naming, the mapping of token pieces to symbolic
information isn't straightforward and requires parsing; 3) default modules
need to be expanded and the old method only worked for simple cases and,
in particular, would not work when combined with decorated names.
This change recognizes the special case of a `super` variable load
in the position of a function call. Left on its own, this yields
unverifiable MuIL, because we attempt to invoke a non-function.
This change fixes the type token emission logic in MuJS to emit fully
resolved tokens for intra-module type references. The prior logic emitted
simple identifiers when type references were contained within a single file.
This hearkens back to the days where we used simple names, coupled with a
lexical symbol table, to resolve such things.
This brings the AWS MuPack verification error count to 57 from 84.
This change restructures how MuJS selects the format and output a
bit, so that simply running `mujs -f yaml` will output a default
`Mupack.yaml` in the current directory.
This change stops printing the AST to output after compilation. It's
a little confusing and, especially as Mu test programs are getting larger,
staring at 1,000s of lines of JSON whiz by is losing it's fun.
By default, the compiler saves output into Mupack.json. This can be
overridden with the --out switch (-o for short). Both JSON and YAML files
are supported (as are unrecognized extensions, which get JSON by default).
Another switch --format (-f for short) is supported to explicitly specify
the output format. A special -out of "-" means "print to stdoutput", so
for instance, running `mujs -o=- -f=yaml` will print YAML to the console.
This change doesn't attempt to transform the resulting AST if there
were any compiler errors. The old code erroneously proceeded in cases
where errors occurred but an AST was actually produced, leading to
possible assertions and other unexpected conditions.
I've had a few times where I missed errors/warnings whizzing by alongside
other spew. Although this was probably technically a "waste" of 10 minutes,
it actually would have saved me at least 10 in the past. So, there you go.
This adds map tests for ES6-style maps encoded in MuPack/MuIL.
It also fixes one simple bug, where we should not populate the
compiler options with TypeScript defaults *unless* they are (a)
missing from the arguments and (b) not discovered from a tsconfig
file. Otherwise, we will rewrite overrides with the defaults.
This change just adds a clean target to the MuJS compiler's package
file. If you rename tests, the old files stick around inside of bin/,
possibly causing superfluous test failures. A clean does the trick.
This change adds support for generating array type tokens from MuJS.
Previously, we'd try to load up the Array type's associate MuPackage,
which is TypeScript's standard library; and then we'd error out
because, of course, it doesn't have one. Now we recognize references
to it (and ES6-style Map<K, V>) to generate the correct tokens.
This includes a bunch of new array tests.
To prevent shell expansion of the glob "**", we must quote the path.
This fixes a bug that @lukehoban ran into last night on Windows, where
globbing wasn't be expanded the same way; ironically, it seems to have
been expanded the way we actually wanted it to...but obviously we want
globbing to work well on *NIX filesystems even moreso than Windows.
This also fies a couple missed lint errors in our tests that are
revealed as a result.
This change stops digging into the "new" expression and instead uses
the TypeScript-assigned type (based on its bound signature). The result
is that expressions of the form
import * as aws from "aws";
let vm = new aws.ec2.VM(...);
now typecheck and emit fully resolved type tokens for the constructed types.
This overhauls how MuJS creates and emits tokens, so that they are
properly fully qualified. Before this change, we frequently emitted
"naked" identifier names that, though the compiler had already resolved
and bound them, looked to the MuIL toolchain as though they hadn't been.
This required rearranging the "symbols" (now "tokens") layer to look more
like the types in the MuIL toolchain.
More impactfully, however, it means we need to load MuIL packages from
dependencies as part of creating the fully resolved tokens. This injects
I/O into places it didn't exist before, so a large part of this change is
simply playing the async/await whack-a-mole game.
This is one big step along the path of fixing marapongo/mu#46. It
still isn't 100% because we need to emit array/property symbols correctly,
and also I suspect as we do more multi-module testing, a few other things
will crop up. But we're almost there.
This change looks up the main module from a package, and that module's
entrypoint, when performing evaluation. In any case, the arguments are
validated and bound to the resulting function's parameters.
This change checks in an enormously rudimentary interpreter. There is
a lot left to do, as evidenced by the countless TODOs scattered throughout
pkg/compiler/eval/eval.go. Nevertheless, the scaffolding and some important
pieces are included in this change.
In particular, to evaluate a package, we must locate its entrypoint and then,
using the results of binding, systematically walk the full AST. As we do
so, we will assert that aspects of the AST match the expected shape,
including symbols and their types, and produce value objects for expressions.
An *unwind structure is used for returns, throws, breaks, and continues (both
labeled and unlabeled). Each statement or expression visitation may optionally
return one and its presence indicates that control flow transfer is occurring.
The visitation logic then needs to propagate this; usually just by bailing out
of the current context immediately, but sometimes -- as is the case with
TryCatchBlock statements, for instance -- the unwind is manipulated in more
"interesting" ways.
An *Object structure is used for expressions yielding values. This is a
"runtime object" in our system and is comprised of three things: 1) a Type
(essentially its v-table), 2) an optional data pointer (for tagged primitives),
and 3) an optional bag of properties (for complex object property values).
I am still on the fence about whether to unify the data representations.
The hokiest aspect of this change is the scoping and value management. I am
trying to avoid needing to implement any sort of "garbage collection", which
means our bag-of-values approach will not work. Instead, we will need to
arrange for scopes to be erected and discarded in the correct order during
evaluation. I will probably tackle that next, along with fleshing out the
many missing statement and expression cases (...and tests, of course).
