Per Eric's suggestion, this moves the colors we use into a central
location so that it'll be easier someday down the road to reconfigure
and/or disable them, etc. This does not include a --no-colors option
although we should really include this soon before it gets too hairy.
This change adds support for serializing snapshots in MuGL, per the
design document in docs/design/mugl.md. At the moment, it is only
exposed from the `mu plan` command, which allows you to specify an
output location using `mu plan --output=file.json` (or `-o=file.json`
for short). This serializes the snapshot with monikers, resources,
and so on. Deserialization is not yet supported; that comes next.
* Specify MinCount/MaxCount when creating an EC2 instance. These
are required properties on the RunInstances API.
* Only attempt to unmarshal egressArray/ingressArray when non-nil.
* Remember the context object on the instanceProvider.
* Move the moniker and object maps into the shared context object.
* Marshal object monikers as the resource IDs to which they refer,
since monikers are useless on "the other side" of the RPC boundary.
This ensures that, for example, the AWS provider gets IDs it can use.
* Add some paranoia assertions.
This change does two things:
First, and foremost, it adds interface members to prototypes. This
ensures that interface members are available statically on all objects
of types that implement those interfaces.
Second, we permit dynamic loads through LoadLocationExpression when
the `this` object is `dynamic`. This is a convenient shortcut compared
to creating different LoadDynamicExpressions when the `this` happens to
be of a dynamic type.
This commit includes a basic AWS resource provider. Mostly it is just
scaffolding, however, it also includes prototype implementations for EC2
instance and security group resource creation operations.
This change implements `mu apply`, by driving compilation, evaluation,
planning, and then walking the plan and evaluating it. This is the bulk
of marapongo/mu#21, except that there's a ton of testing/hardening to
perform, in addition to things like progress reporting.
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
This moves us one step closer from planning to application (marapongo/mu#21).
Namely, we now drive the right resource provider operations in response to
the plan's steps. Those providers, however, are still empty shells.
This change adds a flag to `plan` so that we can create deletion plans:
$ mu plan --delete
This will have an equivalent in the `apply` command, achieving the ability
to delete entire sets of resources altogether (see marapongo/mu#58).
This change introduces object monikers. These are unique, serializable
names that refer to resources created during the execution of a MuIL
program. They are pretty darned ugly at the moment, but at least they
serve their desired purpose. I suspect we will eventually want to use
more information (like edge "labels" (variable names and what not)),
but this should suffice for the time being. The names right now are
particularly sensitive to simple refactorings.
This is enough for marapongo/mu#69 during the current sprint, although
I will keep the work item (in a later sprint) to think more about how
to make these more stable. I'd prefer to do that with a bit of
experience under our belts first.
This change introduces a new package, pkg/resource, that will form
the foundation for actually performing deployment plans and applications.
It contains the following key abstractions:
* resource.Provider is a wrapper around the CRUD operations exposed by
underlying resource plugins. It will eventually defer to resource.Plugin,
which itself defers -- over an RPC interface -- to the actual plugin, one
per package exposing resources. The provider will also understand how to
load, cache, and overall manage the lifetime of each plugin.
* resource.Resource is the actual resource object. This is created from
the overall evaluation object graph, but is simplified. It contains only
serializable properties, for example. Inter-resource references are
translated into serializable monikers as part of creating the resource.
* resource.Moniker is a serializable string that uniquely identifies
a resource in the Mu system. This is in contrast to resource IDs, which
are generated by resource providers and generally opaque to the Mu
system. See marapongo/mu#69 for more information about monikers and some
of their challenges (namely, designing a stable algorithm).
* resource.Snapshot is a "snapshot" taken from a graph of resources. This
is a transitive closure of state representing one possible configuration
of a given environment. This is what plans are created from. Eventually,
two snapshots will be diffable, in order to perform incremental updates.
One way of thinking about this is that a snapshot of the old world's state
is advanced, one step at a time, until it reaches a desired snapshot of
the new world's state.
* resource.Plan is a plan for carrying out desired CRUD operations on a target
environment. Each plan consists of zero-to-many Steps, each of which has
a CRUD operation type, a resource target, and a next step. This is an
enumerator because it is possible the plan will evolve -- and introduce new
steps -- as it is carried out (hence, the Next() method). At the moment, this
is linearized; eventually, we want to make this more "graph-like" so that we
can exploit available parallelism within the dependencies.
There are tons of TODOs remaining. However, the `mu plan` command is functioning
with these new changes -- including colorization FTW -- so I'm landing it now.
This is part of marapongo/mu#38 and marapongo/mu#41.
This change fixes two aspects of binding names to objects. First, we need
to adjust pointers from prototype functions, since they will have the wrong
"this" (it points to the prototype instance and not the actual object).
Second, we need to bind `super` accesses using the correct prototype object.
* Improve some logging and assertions.
* Initialize the prototype map (it was always empty before).
* Actually memoize the prototype objects in this map.
* During class initialization, ensure the base class is also initialized.
* Trigger class initialization anytime a property or function is loaded
from that class. Similarly, trigger module initialization on loads too.
* Treat For/ForIn/ForOfStatements as legal parents for local variables.
This ensures for loops at the top-level in a module have their variables
treated as locals, rather than module properties. Add a test also.
* Track source locations for more nodes.
* Mark auto-generated constructors as public so that they may be called.
* Fix the naming of class init methods; it should be ".init", not ".ctor".
* Clone the loop test cases into both function and module top-level variants.
One guiding principle for what makes it into the MuIL AST is that
the gap between source language and AST should not be too great; the
projection of switch statements from MuJS into MuIL clearly violated
that principle, particularly considering that the logic wasn't even
right due to the incorrect emulation of conditional breaks.
Instead of digging deeper into the hole, I've encoded switch logic
in the AST, and implemented support in the evaluator.
* Add a TODO as a reminder to implement number toString formatting.
* Change the Loreley delimiters to something obscure ("<{%" and "%}>")
to avoid conflicting with actual characters we might use in messages.
Also, make the assertions more descriptive should Loreley fail.
* Rip out a debug.PrintStack() in verbose logging.
* Check the underlying pointer's object type for +=, not the pointer type.
This change implements intrinsic function support in the runtime.
The basic idea is to swap out references to MuIL functions with
runtime-implemented functionality, for operations that cannot be
expressed in the MuIL-subset. For example, this change includes
two such cases: 1) a mu.runtime.isFunction(obj) function to check if
the target object is a function; and 2) a
mu.runtime.dynamicInvoke(obj,obj,obj[]) function to dynamically
invoke a target object.
These will be used most immediately to implement ECMAScript toString
functionality in the MuJS runtime library, however it will also come
in handy shortly to implement things like printf (marapongo/mu#86),
string and array functions (marapongo/mu#80), and, eventually, the
ECMAScript-specific operators and behavior (marapongo/mu#61).
This change generalizes the support for default modules (added as
part of marapongo/mu#57), to use an "alias" table. This is just a
table of alternative module names to the real defining module name.
The default module is now just represented as a mapping from the
special default module token, ".default", to the actual defining
module. This generalization was necessary to support Node.js-style
alternative names for modules, like "lib" mapping to "lib/index".
Previously, I had thought we would ask MetaMu compilers to map their
own exception/error types to ours. That causes some complexity, however,
in that the exception types in each language (if they even exist) are
"different". So we would need to wrap things, etc., which seems cumbersome.
Partly I had done this after having been burned on the CLR by permitting
throws of any types; e.g., C++/CLI could throw integers, which would rip
through C# code unknowingly, because all C# throws had to derive from the
Exception base class. This actually caused some security issues!
But, we're in a different place, and a different time. The first three
languages I envision supporting -- ECMAScript, Python, and Ruby -- each
permit you to throw anything. And the only other languages I am seriously
contemplating right now -- Go -- doesn't even have exceptions.
So this change backs off and is less opinionated about what is thrown.
Instead, we track stack traces, etc., in the unwind information, and happily
propagate whatever object is thrown.
This change more accurately implements ECMAScript prototype chains.
This includes using the prototype chain to lookup properties when
necessary, and copying them down upon writes.
This still isn't 100% faithful -- for example, classes and
constructor functions should be represented as real objects with
the prototype link -- so that examples like those found here will
work: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/constructor.
I've updated marapongo/mu#70 with additional details about this.
I'm sure we'll be forced to fix this as we encounter more "dynamic"
JavaScript. (In fact, it would be interesting to start running the
pre-ES6 output of TypeScript through the compiler as test cases.)
See http://www.ecma-international.org/ecma-262/6.0/#sec-objects
for additional details on the prototype chaining semantics.
The complexity of the last round of property initialization code was
starting to bug me. I've switched away from lazily initializing them
and now eagerly initialize at the proper times: module initialization,
class initialization (for statics), and object allocation (for instances).
This includes the logic around readonly and freezing. The code is a
lot simpler now and actually enforces some invariants that we'd like to
enforce, like not silently adding new properties when a supposedly static
member load is happening (all such accesses should be dynamic ones).
This change accurately enforces readonly properties. Namely, they
should not be written to anywhere outside of the respective initializer,
but writes must be allowed within them. This means the module initializer
for module properties, the class initializer for statics, and the object
constructor for instance properties.
Now that some name lookups are context-sensitive (namely, whether
or not we should use the export or member table for inter vs. intra
module token references), we need to faithfully track the context.
This change fixes up some issues in the big export refactoring;
namely, we need to chase down references one link at a time during
binding, before they settle. This is because an export might depend
on another export, which might depend on ...
This change redoes the way module exports are represented. The old
mechanism -- although laudible for its attempt at consistency -- was
wrong. For example, consider this case:
let v = 42;
export { v };
The old code would silently add *two* members, both with the name "v",
one of which would be dropped since the entries in the map collided.
It would be easy enough just to detect collisions, and update the
above to mark "v" as public, when the export was encountered. That
doesn't work either, as the following two examples demonstrate:
let v = 42;
export { v as w };
let x = w; // error!
This demonstrates:
* Exporting "v" with a different name, "w" to consumers of the
module. In particular, it should not be possible for module
consumers to access the member through the name "v".
* An inability to access the exported name "w" from within the
module itself. This is solely for external consumption.
Because of this, we will use an export table approach. The exports
live alongside the members, and we are smart about when to consult
the export table, versus the member table, during name binding.
This commit implements array l-values (for dynamic loads only, since we do
not yet ever produce static ones). Note that there are some ECMAScript
compliance noteworthies about this change, which are captured in comments.
Namely, we are emulating ECMAScript's ability to index into an array
anywhere (except for negative numbers, which is a problem). This is in
contrast to the usual approach of throwing an error on out-of-bounds access,
which will crop up when we move on to other languages like Python. And yet we
are usuing a real array as the backing store, which can cause problems with
some real ECMAScript programs that use sparse arrays and expect the "bag of
properties" approach to keep memory usage reasonable.
The work item marapongo/mu#70 tracks this among other things.
This change plucks array element types out of more AST nodes, tightening
up the MuIL that MuJS produces. It also adds a few cases where we should
be emitting "object", and permits more dynamic conversions (which are
technically unsafe, but necessary for the sort of duck typing we anticipate).
