Commit graph

179 commits

Author SHA1 Message Date
joeduffy
af45bfd53f Fix up a few things
* Improve some logging and assertions.

* Initialize the prototype map (it was always empty before).

* Actually memoize the prototype objects in this map.

* During class initialization, ensure the base class is also initialized.

* Trigger class initialization anytime a property or function is loaded
  from that class.  Similarly, trigger module initialization on loads too.

* Treat For/ForIn/ForOfStatements as legal parents for local variables.
  This ensures for loops at the top-level in a module have their variables
  treated as locals, rather than module properties.  Add a test also.

* Track source locations for more nodes.

* Mark auto-generated constructors as public so that they may be called.

* Fix the naming of class init methods; it should be ".init", not ".ctor".

* Clone the loop test cases into both function and module top-level variants.
2017-02-16 06:48:39 -08:00
joeduffy
c6f77ae3d6 Implement for loops
This change introduces a for loop node to the AST, for much the same
reasons we elevated switch nodes to the AST, and lowers MuJS for loops.
2017-02-16 05:38:01 -08:00
joeduffy
563fad29ec Add 1st class switch support
One guiding principle for what makes it into the MuIL AST is that
the gap between source language and AST should not be too great; the
projection of switch statements from MuJS into MuIL clearly violated
that principle, particularly considering that the logic wasn't even
right due to the incorrect emulation of conditional breaks.

Instead of digging deeper into the hole, I've encoded switch logic
in the AST, and implemented support in the evaluator.
2017-02-16 04:58:04 -08:00
joeduffy
a6aa17e2b7 Make variable types required
This change requires variable types in the MuIL.  It is up to the MetaMu
compiler to pick a default if it so chooses (object, dynamic, etc.)
2017-02-15 18:42:24 -08:00
joeduffy
64554621cf Implement switch statements 2017-02-15 18:19:24 -08:00
joeduffy
2af011d6ab Implement TypeOf expressions 2017-02-15 15:58:46 -08:00
joeduffy
bd8faf313f Implement parenthesized expressions 2017-02-15 15:52:56 -08:00
joeduffy
af983183d9 Fix a couple asserts
These asserts need to use Object.hasOwnProperty, rather than simply
looking up the property; otherwise, if we have a function whose name
happens to match a built-in ECMAScript property name -- like
toString -- then the assert will fire!
2017-02-15 15:47:20 -08:00
joeduffy
6719a6070a Generalize default modules
This change generalizes the support for default modules (added as
part of marapongo/mu#57), to use an "alias" table.  This is just a
table of alternative module names to the real defining module name.
The default module is now just represented as a mapping from the
special default module token, ".default", to the actual defining
module.  This generalization was necessary to support Node.js-style
alternative names for modules, like "lib" mapping to "lib/index".
2017-02-15 12:53:36 -08:00
joeduffy
f22514fc74 Strengthen the super test
This change adds runtime verification of the super test's expected
behavior.  This has proven useful in testing out some of the recent
runtime/interpreter logic around super, etc.  Most likely we will
start testing all of the MuJS tests as real MuIL/etc. tests soon.
2017-02-15 12:47:20 -08:00
joeduffy
5b29215195 Implement a MuJS standard library
This change introduces a MuJS standard library.  This will allow us
to support an increasing amount of ECMAScript functionality without
needing to do it in the runtime or the compiler.

The basic idea is as follows.

In the TypeScript compiler, all uses of ECMAScript standard types and
functions get bound to the TypeScript library headers.  These headers
are in the node_modules/typescript/lib/ directory, and there is one
per ECMAScript "standard" (lib.d.ts for ECMAScript 5, lib.es2015.d.ts
for ECMAScript 6, lib.es5016.d.ts for ECMAScript 7, and so on).  Note
that these libraries are "header only", since the functionality for
these are of course supplied by whatever runtime (V8, Chakra, etc)
actually executes the resulting JavaScript code.

In our case, we have no such luxury.  So instead, we intercept these
bindings and transform them into bindings to a MuJS standard library.
This library is just a MuIL library written in MuJS code.  Only the
set of operations expressable in basic MuIL can be used to implement
these.  In select few cases, we will need runtime intrinsics, but those
will not be specific to MuJS and will go into the basic Mu library and
shared among all MetaMu compilers.

At this point, the MuJS standard library only contains the ECMAScript
Error type, so that we can represent throwing errors in MuJS/MuIL.
2017-02-15 11:57:25 -08:00
joeduffy
971fdfbf8d Fall back to dynamic on object too
We already fall back to dynamic loads when the type is `dynamic`, however
we also need to do that when it is the weakly typed `object` also.
2017-02-13 11:59:06 -08:00
joeduffy
a2653093db Make export binding more robust
This changes the way we discover the kind of export in the case of
an export declaration.  This might be exporting a module member, as in

    let x = 42;
    export { x as y };

or it could be exporting an entire module previously imported, as in

    import * as other from "other";
    export { other };

In each case, we must emit a proper fully qualified token.

Our logic for detecting these cases was a bit busted previously.  We
now rely on real symbolic information rather than doing hokey name lookups.
2017-02-13 11:37:23 -08:00
joeduffy
32960be0fb Use export tables
This change redoes the way module exports are represented.  The old
mechanism -- although laudible for its attempt at consistency -- was
wrong.  For example, consider this case:

    let v = 42;
    export { v };

The old code would silently add *two* members, both with the name "v",
one of which would be dropped since the entries in the map collided.

It would be easy enough just to detect collisions, and update the
above to mark "v" as public, when the export was encountered.  That
doesn't work either, as the following two examples demonstrate:

    let v = 42;
    export { v as w };
    let x = w; // error!

This demonstrates:

    * Exporting "v" with a different name, "w" to consumers of the
      module.  In particular, it should not be possible for module
      consumers to access the member through the name "v".

    * An inability to access the exported name "w" from within the
      module itself.  This is solely for external consumption.

Because of this, we will use an export table approach.  The exports
live alongside the members, and we are smart about when to consult
the export table, versus the member table, during name binding.
2017-02-13 09:56:25 -08:00
joeduffy
a09a49b94a Regenerate test baselines w/ line numbering change 2017-02-13 07:55:40 -08:00
joeduffy
1af9cd720b Use 1-based column numbers 2017-02-13 06:44:48 -08:00
joeduffy
f0181d8933 Emit more array types; permit more dynamic conversions
This change plucks array element types out of more AST nodes, tightening
up the MuIL that MuJS produces.  It also adds a few cases where we should
be emitting "object", and permits more dynamic conversions (which are
technically unsafe, but necessary for the sort of duck typing we anticipate).
2017-02-13 06:39:50 -08:00
joeduffy
d82adefd38 Implement casts 2017-02-13 05:56:39 -08:00
joeduffy
aec1dccd5d Tidy up property initializers
This change fixes a few things about property initializers:

* Instance properties on classes need an object (`this`) for initialization.
  Previously, this wasn't being passed, which leads to a verification error.

* Static properties on classes should go into the `.init` function, not the
  `.ctor` function.

* Synthesized constructors need to invoke `super()` when there is a base
  class.  It's legal for this call to be missing from the source program
  when the base's constructor has no args.

* All initializers happen immediately after such a call to `super()`, to
  achieve the intended initialization order.  Therefore, for an existing
  constructor, we must search for the insertion point manually.

* Also add a superclass case to the existing property tests.
2017-02-13 04:56:20 -08:00
joeduffy
4b05735374 Add some missing source node info 2017-02-13 04:13:22 -08:00
joeduffy
9d157a25c6 Issue a warning if there is no default module 2017-02-13 03:58:31 -08:00
joeduffy
2fd4355cf6 Add some array accesses to basic/arrays tests 2017-02-13 03:45:23 -08:00
joeduffy
9ba0d461b2 Don't use static transforms for element loads
These are generally speaking dynamic lookups, based on expressions
that are evaluated, rather than being statically recognized property names.
2017-02-12 12:09:20 -08:00
joeduffy
344f87a3e6 Detect more load cases
This change fixes up some cases that weren't quite right, when it
comes to load location expressions.  This includes functions and
local variables.  Furthermore, the loadIdentifierExpression codepath
should be routed through the general purpose load location logic,
so that it can detect the various cases.

After this change, the ec2instance example fully verifies.
2017-02-12 11:44:20 -08:00
joeduffy
16be3e181b Add missing source location information
A couple dynamic load nodes didn't have source information propagated,
resulting in hard-to-diagnose failures.  This adds it to them.
2017-02-12 09:46:54 -08:00
joeduffy
37d63555dc Implement basic default modules and entrypoints
Per marapongo/mu#57, this change marks the "index" module as the default.
We don't yet discover and crack open `package.json` files which may override
this by specifying a "main" function.

This change also generates entrypoint functions for all MuJS modules.  To
emulate the way Node.js works -- in that every module is still a valid script
target -- we are generating entrypoints for everything; this is in contrast to
only generating them for "blueprint" modules, or somehow marking `main`
somehow.  See the notes in marapongo/mu#57 for additional thoughts on this.
2017-02-12 09:43:59 -08:00
joeduffy
93d9df1c4c Issue a MuJS warning on unused dependencies 2017-02-12 06:23:22 -08:00
joeduffy
11faf22c60 Produce a string literal for dynamic loads 2017-02-11 15:38:12 -08:00
joeduffy
3759f71e67 Transform element access expressions properly
This change transforms element access expressions much like we do
property access expressions, in that we recognize several static load
situations.  (The code is rearranged to share more logic.)  The code
also now recognizes situations where we must devolve into a dynamic
load (e.g., due to a string literal, dynamic LHS, etc).
2017-02-11 15:32:14 -08:00
joeduffy
10f5d43245 Also erase TypeLiterals
This change erases TypeLiterals to ObjectLiterals, for much the same reason.
Also for much the same reason, it'd be nice to eventually preserve these.
2017-02-11 14:51:05 -08:00
joeduffy
a7f23fc65a Fix a few verification errors
This change fixes a few issues:

* Emit ObjectLiteral types as `dynamic`.  We can do better here, including
  spilling the "anonymous" types, for cases where cross-module exports/imports
  are leveraged.  For the time being, however, we will erase them to dynamic.
  This is a heck of a lot better than emitting garbage `__object` type tokens.

* Implement TypeScript TypeAssertionExpression lowering into casts.

* Recognize static class property references.  Previously, we tried to load
  the class as a location, which resulted in nonsense, unverifiable MuIL.  Now
  we properly detect that the target is a property symbol with the static
  modifier, and emit the token without an object reference (as expected).

* Accompany the above with a bunch of tests that stress these paths.
2017-02-11 14:45:31 -08:00
joeduffy
a60dc045cb Tokenize properties and block scoped variables
We weren't properly emitting fully qualified tokens for properties
and "block scoped variables" (of which, module properties qualify).
This change fixes that, bringing the ec2instance verification error
count down from 22 to 7.
2017-02-11 10:10:02 -08:00
joeduffy
dc9f91c3cf Support dynamic object literals
This change introduces support for dynamic object literals.  If the
type of a literal is dynamic, then the property initializers for that
literal are naked strings; if the type of a literal is anything else,
however, we expect that the property names are full blown member tokens.
2017-02-11 08:20:06 -08:00
joeduffy
c6b9abb7a0 Issue an error if a dependency is missing 2017-02-11 08:02:28 -08:00
joeduffy
8f24cd20ba Simplify certain types
This change simplifies certain types while lowering to MuIL, fixing
a number of assertion failures that cropped up after my tightening of
the type checking (namely, asserting rather than silently logging).

This comes in the following forms:

1) StringLiteral, NumberLiteral, and BooleanLiteral types can simply
   become String, Number, and Boolean, respectively.  This loses some
   static type-checking (e.g., ensuring that values are within range),
   however at least the physical runtime type will match.

2) EnumLiteral types can simply adopt their base types, with similar
   downsides and caveats to the other literal types (range widening).

3) Any union types that contain Undefined or Null types can be simplified
   to omit those types.  If, after filtering them out, we are left with
   a single type (admitting the recursive case of multiple XLiterals that
   simplified down to the exact same primitive type), we will eliminate
   the union altogether and just use the underlying simple type.  This
   handles the case of optional interface properties, e.g. `T?`, which are
   internally represented in TypeScript as `T|undefined` union types.

There are two follow up work items for later on:

* marapongo/mu#64 tracks support for non-nullability.  At the moment we
  are tossing away any `|undefined` or `|null` unions, including `T?`,
  because MuIL currently permits nullability everywhere, but we want to
  consider honoring these annotations (eventually).

* marapongo/mu#82 tracks support for union types.  This is how we intended
  to support true enum types, so tossing away this information is a bit
  unfortunate.  But this is a feature for another day.
2017-02-11 07:13:49 -08:00
joeduffy
c5807c4b57 Issue proper errors for unsupported types 2017-02-10 17:04:01 -08:00
joeduffy
02aa1c4ee0 Add some MuJS debug logging 2017-02-10 16:53:27 -08:00
joeduffy
0802399a8c Distinguish between object and dynamic (fka any)
This change prepares to mimick the TypeScript behavior of `any` which,
it turns out, is closer to Python and Ruby duck typing than what we have
here.  It also introduces `object` to mean "the base of all types".

MuJS has been updated to map `any` to `dynamic`.

At this point, the conversion logic now distinguishes between NoConversion,
ImplicitConversion, and AutoCastConversion cases, the latter being a case
in which `dynamic` is the source and so a duck-type-style cast must occur.
Right now we do not honor the AutoCastConversion case anywhere, so we still
have not achieved the duck-typing semantics of the source language.

This is part of fixing marapongo/mu#79.
2017-02-10 16:44:18 -08:00
joeduffy
7b81580fb9 Remove some unused variables 2017-02-10 15:30:48 -08:00
joeduffy
8485929192 Transform interface heritage clauses
Turns out we hadn't been transforming interface heritage clauses --
extends and implements -- like we were classes.  This change fixes
that.  As a result, we're down to 14 verification errors for AWS.
2017-02-09 13:27:08 -08:00
joeduffy
42bdcb7bca Fix an inner break bug
We need to break out of two levels of loops, now that we've added
the outer loop around the possible filename candidates.
2017-02-09 12:53:09 -08:00
joeduffy
3a27a44d2c Strip .js, .d.ts, and .ts module name suffixes 2017-02-09 12:43:32 -08:00
joeduffy
6ad42d9063 Probe for MuPackages in MuJS dependency resolution
This change also accepts MuPackages during MuJS dependency resolution.
The old logic wasn't quite right, because it would skip over MuPackage
files, and keep probing upwards, possibly binding to the wrong, pre-
compilation, Mufile.  Note that we must still probe for both Mupackages
and Mufiles, because of intra-package dependencies.

After this change, the AWS MuPackage is down to 28 verification errors.
2017-02-09 12:35:53 -08:00
joeduffy
9bcfbe859f Dealias symbols when producing tokens 2017-02-09 11:32:33 -08:00
joeduffy
1069372cca Save TypeScript definition files
To prepare a MuPackage for consumption by another MuPackage, we need
to save the definition files output by the TypeScript compiler.  In
theory, we could regenerate these definitions from the MuPackage itself;
in fact, when we tackle cross-language, this will be necessary.  For
now, however, it's much simpler to simply save the definition outputs
alongside the Mupack.json package metadata.

Note that we do not save "code" outputs.  The MuIL format encodes all
computations and so the only thing we need for dependent packages is
the header files to compile against; all "link" and runtime dependencies
are satisfied by the MuIL format and Mu toolchain.
2017-02-09 09:07:10 -08:00
joeduffy
26c16474c2 Use tsconfig.json's outDir by default
There are a variety of ways to specify a MuJS output location, including
the command line --out (-o for short) flag.  If that isn't present, we simply
dump the Mupack.json file in the current working directory.  However, in order
to prepare Mupackage files for packaging, distribution, and reuse, we actually
want to store the Mupack.json file alongside all the other package assets (like
TypeScript header files).  So it makes sense to recognize the outDir, if present,
in the tsconfig.json file, and use it as the default if not otherwise specified.
2017-02-09 08:42:49 -08:00
joeduffy
9de4b7ce5d Fully qualify more tokens
This changes transformIdentifierExpression to call through to the
general purpose function that understands how to transform any Symbol
reference into a fully qualified token suitable for serialization.
2017-02-08 16:49:32 -08:00
joeduffy
75eb10203b Transform null and undefined literals
This change recognizes identifier expressions that mention the null and
undefined constants, transforming them into null literals.  Note that
marapongo/mu#70 tracks mimicking the subtle differences between
undefined and null in ECMAScript.
2017-02-08 16:33:53 -08:00
joeduffy
914e88672c Omit object expression for module loads
In the event that we're loading a property straight from a module,
we needn't actually load the module at all; instead, we refer to it
using a fully qualified token.
2017-02-08 16:12:18 -08:00
joeduffy
840f0bd05d Transform cross-module references into correct tokens
In many cases, property access expressions interact with types.  For
example, the expression `o.j.bar()` likely accesses variable `o`, then
its property `j`, then `j`'s method `bar`.  In other cases, however,
property access elements are actually modules.  For example, in:

    import * as o from "o";
    o.j.bar();

This change recognizes this situation and emits code correctly.
2017-02-08 16:07:19 -08:00