This change adds support for serializing snapshots in MuGL, per the
design document in docs/design/mugl.md. At the moment, it is only
exposed from the `mu plan` command, which allows you to specify an
output location using `mu plan --output=file.json` (or `-o=file.json`
for short). This serializes the snapshot with monikers, resources,
and so on. Deserialization is not yet supported; that comes next.
* Specify MinCount/MaxCount when creating an EC2 instance. These
are required properties on the RunInstances API.
* Only attempt to unmarshal egressArray/ingressArray when non-nil.
* Remember the context object on the instanceProvider.
* Move the moniker and object maps into the shared context object.
* Marshal object monikers as the resource IDs to which they refer,
since monikers are useless on "the other side" of the RPC boundary.
This ensures that, for example, the AWS provider gets IDs it can use.
* Add some paranoia assertions.
This change does two things:
First, and foremost, it adds interface members to prototypes. This
ensures that interface members are available statically on all objects
of types that implement those interfaces.
Second, we permit dynamic loads through LoadLocationExpression when
the `this` object is `dynamic`. This is a convenient shortcut compared
to creating different LoadDynamicExpressions when the `this` happens to
be of a dynamic type.
This commit includes a basic AWS resource provider. Mostly it is just
scaffolding, however, it also includes prototype implementations for EC2
instance and security group resource creation operations.
This change implements `mu apply`, by driving compilation, evaluation,
planning, and then walking the plan and evaluating it. This is the bulk
of marapongo/mu#21, except that there's a ton of testing/hardening to
perform, in addition to things like progress reporting.
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
This moves us one step closer from planning to application (marapongo/mu#21).
Namely, we now drive the right resource provider operations in response to
the plan's steps. Those providers, however, are still empty shells.
This change adds a flag to `plan` so that we can create deletion plans:
$ mu plan --delete
This will have an equivalent in the `apply` command, achieving the ability
to delete entire sets of resources altogether (see marapongo/mu#58).
This change introduces object monikers. These are unique, serializable
names that refer to resources created during the execution of a MuIL
program. They are pretty darned ugly at the moment, but at least they
serve their desired purpose. I suspect we will eventually want to use
more information (like edge "labels" (variable names and what not)),
but this should suffice for the time being. The names right now are
particularly sensitive to simple refactorings.
This is enough for marapongo/mu#69 during the current sprint, although
I will keep the work item (in a later sprint) to think more about how
to make these more stable. I'd prefer to do that with a bit of
experience under our belts first.
This change introduces a new package, pkg/resource, that will form
the foundation for actually performing deployment plans and applications.
It contains the following key abstractions:
* resource.Provider is a wrapper around the CRUD operations exposed by
underlying resource plugins. It will eventually defer to resource.Plugin,
which itself defers -- over an RPC interface -- to the actual plugin, one
per package exposing resources. The provider will also understand how to
load, cache, and overall manage the lifetime of each plugin.
* resource.Resource is the actual resource object. This is created from
the overall evaluation object graph, but is simplified. It contains only
serializable properties, for example. Inter-resource references are
translated into serializable monikers as part of creating the resource.
* resource.Moniker is a serializable string that uniquely identifies
a resource in the Mu system. This is in contrast to resource IDs, which
are generated by resource providers and generally opaque to the Mu
system. See marapongo/mu#69 for more information about monikers and some
of their challenges (namely, designing a stable algorithm).
* resource.Snapshot is a "snapshot" taken from a graph of resources. This
is a transitive closure of state representing one possible configuration
of a given environment. This is what plans are created from. Eventually,
two snapshots will be diffable, in order to perform incremental updates.
One way of thinking about this is that a snapshot of the old world's state
is advanced, one step at a time, until it reaches a desired snapshot of
the new world's state.
* resource.Plan is a plan for carrying out desired CRUD operations on a target
environment. Each plan consists of zero-to-many Steps, each of which has
a CRUD operation type, a resource target, and a next step. This is an
enumerator because it is possible the plan will evolve -- and introduce new
steps -- as it is carried out (hence, the Next() method). At the moment, this
is linearized; eventually, we want to make this more "graph-like" so that we
can exploit available parallelism within the dependencies.
There are tons of TODOs remaining. However, the `mu plan` command is functioning
with these new changes -- including colorization FTW -- so I'm landing it now.
This is part of marapongo/mu#38 and marapongo/mu#41.
This change fixes two aspects of binding names to objects. First, we need
to adjust pointers from prototype functions, since they will have the wrong
"this" (it points to the prototype instance and not the actual object).
Second, we need to bind `super` accesses using the correct prototype object.
* Improve some logging and assertions.
* Initialize the prototype map (it was always empty before).
* Actually memoize the prototype objects in this map.
* During class initialization, ensure the base class is also initialized.
* Trigger class initialization anytime a property or function is loaded
from that class. Similarly, trigger module initialization on loads too.
* Treat For/ForIn/ForOfStatements as legal parents for local variables.
This ensures for loops at the top-level in a module have their variables
treated as locals, rather than module properties. Add a test also.
* Track source locations for more nodes.
* Mark auto-generated constructors as public so that they may be called.
* Fix the naming of class init methods; it should be ".init", not ".ctor".
* Clone the loop test cases into both function and module top-level variants.
One guiding principle for what makes it into the MuIL AST is that
the gap between source language and AST should not be too great; the
projection of switch statements from MuJS into MuIL clearly violated
that principle, particularly considering that the logic wasn't even
right due to the incorrect emulation of conditional breaks.
Instead of digging deeper into the hole, I've encoded switch logic
in the AST, and implemented support in the evaluator.
* Add a TODO as a reminder to implement number toString formatting.
* Change the Loreley delimiters to something obscure ("<{%" and "%}>")
to avoid conflicting with actual characters we might use in messages.
Also, make the assertions more descriptive should Loreley fail.
* Rip out a debug.PrintStack() in verbose logging.
* Check the underlying pointer's object type for +=, not the pointer type.
This change implements some key ECMAScript runtime helpers, straight from
the specification: https://tc39.github.io/ecma262/. The goal here is to get
a runtime intrinsic for ECMAScript-style toString to work.
These asserts need to use Object.hasOwnProperty, rather than simply
looking up the property; otherwise, if we have a function whose name
happens to match a built-in ECMAScript property name -- like
toString -- then the assert will fire!
This change implements intrinsic function support in the runtime.
The basic idea is to swap out references to MuIL functions with
runtime-implemented functionality, for operations that cannot be
expressed in the MuIL-subset. For example, this change includes
two such cases: 1) a mu.runtime.isFunction(obj) function to check if
the target object is a function; and 2) a
mu.runtime.dynamicInvoke(obj,obj,obj[]) function to dynamically
invoke a target object.
These will be used most immediately to implement ECMAScript toString
functionality in the MuJS runtime library, however it will also come
in handy shortly to implement things like printf (marapongo/mu#86),
string and array functions (marapongo/mu#80), and, eventually, the
ECMAScript-specific operators and behavior (marapongo/mu#61).
This change generalizes the support for default modules (added as
part of marapongo/mu#57), to use an "alias" table. This is just a
table of alternative module names to the real defining module name.
The default module is now just represented as a mapping from the
special default module token, ".default", to the actual defining
module. This generalization was necessary to support Node.js-style
alternative names for modules, like "lib" mapping to "lib/index".
This change adds runtime verification of the super test's expected
behavior. This has proven useful in testing out some of the recent
runtime/interpreter logic around super, etc. Most likely we will
start testing all of the MuJS tests as real MuIL/etc. tests soon.
