This change fixes a couple issues that prevented restarting a
deployment after partial failure; this was due to the fact that
unchanged resources didn't propagate IDs from old to new. This
is remedied by making unchanged a map from new to old, and making
ID propagation the first thing plan application does.
This change does a few things:
* First and foremost, it tracks configuration variables that are
initialized, and optionally prints them out as part of the
prelude/header (based on --show-config), both in a dry-run (plan)
and in an actual deployment (apply).
* It tidies up some of the colorization and messages, and includes
nice banners like "Deploying changes:", etc.
* Fix an assertion.
* Issue a new error
"One or more errors occurred while applying X's configuration"
just to make it easier to distinguish configuration-specific
failures from ordinary ones.
* Change config keys to tokens.Token, not tokens.ModuleMember,
since it is legal for keys to represent class members (statics).
This change adds support for configuration maps.
This is a new feature that permits initialization code to come from markup,
after compilation, but before evaluation. There is nothing special with this
code as it could have been authored by a user. But it offers a convenient
way to specialize configuration settings per target husk, without needing
to write code to specialize each of those husks (which is needlessly complex).
For example, let's say we want to have two husks, one in AWS's us-west-1
region, and the other in us-east-2. From the same source package, we can
just create two husks, let's say "prod-west" and "prod-east":
prod-west.json:
{
"husk": "prod-west",
"config": {
"aws:config:region": "us-west-1"
}
}
prod-east.json:
{
"husk": "prod-east",
"config": {
"aws:config:region": "us-east-2"
}
}
Now when we evaluate these packages, they will automatically poke the
right configuration variables in the AWS package *before* actually
evaluating the CocoJS package contents. As a result, the static variable
"region" in the "aws:config" package will have the desired value.
This is obviously fairly general purpose, but will allow us to experiment
with different schemes and patterns. Also, I need to whip up support
for secrets, but that is a task for another day (perhaps tomorrow).
* Delete husks if err == nil, not err != nil.
* Swizzle the formatting padding on array elements so that the
diff modifier + or - binds more tightly to the [N] part.
* Print the un-doubly-indented padding for array element headers.
* Add some additional logging to step application (it helped).
* Remember unchanged resources even when glogging is off.
This change adds a --show-sames flag to `coco husk deploy`. This is
useful as I'm working on updates, to show what resources haven't changed
during a deployment.
This change checkpoints deployments properly. That is, even in the
face of partial failure, we should keep the huskfile up to date. This
accomplishes that by tracking the state during plan application.
There are still ways in which this can go wrong, however. Please see
pulumi/coconut#101 for additional thoughts on what we might do here
in the future to make checkpointing more robust in the face of failure.
This command is handy for development, so I whipped up a quick implementation.
All it does is print all known husks with their associated deployment time
and resource count (if any, or "n/a" for initialized husks with no deployments).
This change recognizes TextUnmarshaler during object mapping, and
will defer to it when we have a string but are assigning to a
non-string target that implements the interface.
As part of pulumi/coconut#94 -- adding targeting capabilities -- I've
decided to (yet again) reorganize the deployment commands a bit. This
makes targets ("husks") more of a first class thing.
Namely, you must first initialize a husk before using it:
$ coco husk init staging
Coconut husk 'staging' initialized; ready for deployments
Eventually, this is when you will be given a choice to configure it.
Afterwards, you can perform deployments. The first one is like a create,
but subsequent ones just figure out the right thing to do and do it:
$ ... make some changes ...
$ coco husk deploy staging
... standard deployment progress spew ...
Finally, should you want to teardown an entire environment:
$ coco husk destroy staging
... standard deletion progress spew for all resources ...
Coconut husk 'staging' has been destroyed!
This change partially implements pulumi/coconut#94, by adding the
ability to name targets during creation and reuse those names during
deletion and update. This simplifies the management of deployment
records, checkpoints, and snapshots.
I've opted to call these things "husks" (perhaps going overboard with
joy after our recent renaming). The basic idea is that for any
executable Nut that will be deployed, you have a nutpack/ directory
whose layout looks roughly as follows:
nutpack/
bin/
Nutpack.json
... any other compiled artifacts ...
husks/
... one snapshot per husk ...
For example, if we had a stage and prod husk, we would have:
nutpack/
bin/...
husks/
prod.json
stage.json
In the prod.json and stage.json files, we'd have the most recent
deployment record for that environment. These would presumably get
checked in and versioned along with the overall Nut, so that we
can use Git history for rollbacks, etc.
The create, update, and delete commands look in the right place for
these files automatically, so you don't need to manually supply them.
* Eliminate some superfluous "\n"s.
* Remove the redundant properties stored on AWS resources.
* Compute array diff lengths properly (+1).
* Display object property changes from null to non-null as
adds; and from non-null to null as deletes.
* Fix a boolean expression from ||s to &&s. (Bone-headed).
* Print a pretty message if the plan has nothing to do:
"info: nothing to do -- resources are up to date"
* Add an extra validation step after reading in a snapshot,
so that we detect more errors sooner. For example, I've
fed in the wrong file several times, and it just chugs
along as though it were actually a snapshot.
* Skip printing nulls in most plan outputs. These just
clutter up the output.
This change detects duplicate object names (monikers) and issues a nice
error message with source context include. For example:
index.ts(260,22): error MU2006: Duplicate objects with the same name:
prod::ec2instance:index::aws:ec2/securityGroup:SecurityGroup::group
The prior code asserted and failed abruptly, whereas this actually points
us to the offending line of code:
let group1 = new aws.ec2.SecurityGroup("group", { ... });
let group2 = new aws.ec2.SecurityGroup("group", { ... });
^^^^^^^^^^^^^^^^^^^^^^^^^
This change overhauls the way we do object monikers. The old mechanism,
generating monikers using graph paths, was far too brittle and prone to
collisions. The new approach mixes some amount of "automatic scoping"
plus some "explicit naming." Although there is some explicitness, this
is arguably a good thing, as the monikers will be relatable back to the
source more readily by developers inspecting the graph and resource state.
Each moniker has four parts:
<Namespace>::<AllocModule>::<Type>::<Name>
wherein each element is the following:
<Namespace> The namespace being deployed into
<AllocModule> The module in which the object was allocated
<Type> The type of the resource
<Name> The assigned name of the resource
The <Namespace> is essentially the deployment target -- so "prod",
"stage", etc -- although it is more general purpose to allow for future
namespacing within a target (e.g., "prod/customer1", etc); for now
this is rudimentary, however, see marapongo/mu#94.
The <AllocModule> is the token for the code that contained the 'new'
that led to this object being created. In the future, we may wish to
extend this to also track the module under evaluation. (This is a nice
aspect of monikers; they can become arbitrarily complex, so long as
they are precise, and not prone to false positives/negatives.)
The <Name> warrants more discussion. The resource provider is consulted
via a new gRPC method, Name, that fetches the name. How the provider
does this is entirely up to it. For some resource types, the resource
may have properties that developers must set (e.g., `new Bucket("foo")`);
for other providers, perhaps the resource intrinsically has a property
that explicitly and uniquely qualifies the object (e.g., AWS SecurityGroups,
via `new SecurityGroup({groupName: "my-sg"}`); and finally, it's conceivable
that a provider might auto-generate the name (e.g., such as an AWS Lambda
whose name could simply be a hash of the source code contents).
This should overall produce better results with respect to moniker
collisions, ability to match resources, and the usability of the system.
This change is a first whack at implementing updates.
Creation and deletion plans are pretty straightforward; we just take
a single graph, topologically sort it, and perform the operations in
the right order. For creation, this is in dependency order (things
that are depended upon must be created before dependents); for deletion,
this is in reverse-dependency order (things that depend on others must
be deleted before dependencies). These are just special cases of the more
general idea of performing DAG operations in dependency order.
Updates must work in terms of this more general notion. For example:
* It is an error to delete a resource while another refers to it; thus,
resources are deleted after deleting dependents, or after updating
dependent properties that reference the resource to new values.
* It is an error to depend on a create a resource before it is created;
thus, resources must be created before dependents are created, and/or
before updates to existing resource properties that would cause them
to refer to the new resource.
Of course, all of this is tangled up in a graph of dependencies. As a
result, we must create a DAG of the dependencies between creates, updates,
and deletes, and then topologically sort this DAG, in order to determine
the proper order of update operations.
To do this, we slightly generalize the existing graph infrastructure,
while also specializing two kinds of graphs; the existing one becomes a
heapstate.ObjectGraph, while this new one is resource.planGraph (internal).
This change introduces a simple retry package which will be used in
resource providers in order to perform waits, backoffs, etc. It's
pretty basic right now but leverages the fancy new Golang context
support for deadlines and timeouts. Soon we will layer AWS-specific
acceptors, etc. atop this more general purpose thing.
This change introduces a new informational message category to the
overall diagnostics infrastructure, and then wires up the resource
provider plugins stdout/stderr streams to it. In particular, a
write to stdout implies an informational message, whereas a write to
stderr implies an error. This is just a very simple and convenient
way for plugins to provide progress reporting; eventually we may
need something more complex, due to parallel evaluation of resource
graphs, however I hope we don't have to deviate too much from this.
This change adds custom serialization and deserialization to preserve
the ordering of resources in snapshot files. Unfortunately, because
Go maps are unordered, the results are scrambled (actually, it turns out,
Go will sort by the key). An alternative would be to resort the results
after reading in the file, or storing as an array, but this would be a
change to the MuGL file format and is less clear than simply keying each
resource by its moniker as we are doing today.
This change repivots the plan/apply commands slightly. This is largely
in preparation for performing deletes and updates of existing environments.
The old way was slightly confusing and made things appear more "magical"
than they actually are. Namely, different things are needed for different
kinds of deployment operations, and trying to present them each underneath
a single pair of CLI commands just leads to weird modality and options.
The new way is to offer three commands: create, update, and delete. Each
does what it says on the tin: create provisions a new environment, update
makes resource updates to an existing one, and delete tears down an existing
one entirely. The arguments are what make this interesting: create demands
a MuPackage to evaluate (producing the new desired state snapshot), update
takes *both* an existing snapshot file plus a MuPackage to evaluate (producing
the new desired state snapshot to diff against the existing one), and delete
merely takes an existing snapshot file and no MuPackage, since all it must
do is tear down an existing known environment.
Replacing the plan functionality is the --dry-run (-n) flag that may be
passed to any of the above commands. This will print out the plan without
actually performing any opterations.
All commands produce serializable resource files in the MuGL file format,
and attempt to do smart things with respect to backups, etc., to support the
intended "Git-oriented" workflow of the pure CLI dev experience.
Per Eric's suggestion, this moves the colors we use into a central
location so that it'll be easier someday down the road to reconfigure
and/or disable them, etc. This does not include a --no-colors option
although we should really include this soon before it gets too hairy.
This change adds support for serializing snapshots in MuGL, per the
design document in docs/design/mugl.md. At the moment, it is only
exposed from the `mu plan` command, which allows you to specify an
output location using `mu plan --output=file.json` (or `-o=file.json`
for short). This serializes the snapshot with monikers, resources,
and so on. Deserialization is not yet supported; that comes next.
* Specify MinCount/MaxCount when creating an EC2 instance. These
are required properties on the RunInstances API.
* Only attempt to unmarshal egressArray/ingressArray when non-nil.
* Remember the context object on the instanceProvider.
* Move the moniker and object maps into the shared context object.
* Marshal object monikers as the resource IDs to which they refer,
since monikers are useless on "the other side" of the RPC boundary.
This ensures that, for example, the AWS provider gets IDs it can use.
* Add some paranoia assertions.
This change does two things:
First, and foremost, it adds interface members to prototypes. This
ensures that interface members are available statically on all objects
of types that implement those interfaces.
Second, we permit dynamic loads through LoadLocationExpression when
the `this` object is `dynamic`. This is a convenient shortcut compared
to creating different LoadDynamicExpressions when the `this` happens to
be of a dynamic type.
This commit includes a basic AWS resource provider. Mostly it is just
scaffolding, however, it also includes prototype implementations for EC2
instance and security group resource creation operations.
This change implements `mu apply`, by driving compilation, evaluation,
planning, and then walking the plan and evaluating it. This is the bulk
of marapongo/mu#21, except that there's a ton of testing/hardening to
perform, in addition to things like progress reporting.
This change adds basic support for discovering, loading, binding to,
and invoking RPC methods on, resource provider plugins.
In a nutshell, we add a new context object that will share cached
state such as loaded plugins and connections to them. It will be
a policy decision in server scenarios how much state to share and
between whom. This context also controls per-resource context
allocation, which in the future will allow us to perform structured
cancellation and teardown amongst entire groups of requests.
Plugins are loaded based on their name, and can be found in one of
two ways: either simply by having them on your path (with a name of
"mu-ressrv-<pkg>", where "<pkg>" is the resource package name with
any "/"s replaced with "_"s); or by placing them in the standard
library installation location, which need not be on the path for this
to work (since we know precisely where to look).
If we find a protocol, we will load it as a child process.
The protocol for plugins is that they will choose a port on their
own -- to eliminate races that'd be involved should Mu attempt to
pre-pick one for them -- and then write that out as the first line
to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT
that Mu cares about; from there, the plugin is free to write all it
pleases (e.g., for logging, debugging purposes, etc).
Afterwards, we then bind our gRPC connection to that port, and create
a typed resource provider client. The CRUD operations that get driven
by plan application are then simple wrappers atop the underlying gRPC
calls. For now, we interpret all errors as catastrophic; in the near
future, we will probably want to introduce a "structured error"
mechanism in the gRPC interface for "transactional errors"; that is,
errors for which the server was able to recover to a safe checkpoint,
which can be interpreted as ResourceOK rather than ResourceUnknown.
This moves us one step closer from planning to application (marapongo/mu#21).
Namely, we now drive the right resource provider operations in response to
the plan's steps. Those providers, however, are still empty shells.
This change adds a flag to `plan` so that we can create deletion plans:
$ mu plan --delete
This will have an equivalent in the `apply` command, achieving the ability
to delete entire sets of resources altogether (see marapongo/mu#58).
This change introduces object monikers. These are unique, serializable
names that refer to resources created during the execution of a MuIL
program. They are pretty darned ugly at the moment, but at least they
serve their desired purpose. I suspect we will eventually want to use
more information (like edge "labels" (variable names and what not)),
but this should suffice for the time being. The names right now are
particularly sensitive to simple refactorings.
This is enough for marapongo/mu#69 during the current sprint, although
I will keep the work item (in a later sprint) to think more about how
to make these more stable. I'd prefer to do that with a bit of
experience under our belts first.
This change introduces a new package, pkg/resource, that will form
the foundation for actually performing deployment plans and applications.
It contains the following key abstractions:
* resource.Provider is a wrapper around the CRUD operations exposed by
underlying resource plugins. It will eventually defer to resource.Plugin,
which itself defers -- over an RPC interface -- to the actual plugin, one
per package exposing resources. The provider will also understand how to
load, cache, and overall manage the lifetime of each plugin.
* resource.Resource is the actual resource object. This is created from
the overall evaluation object graph, but is simplified. It contains only
serializable properties, for example. Inter-resource references are
translated into serializable monikers as part of creating the resource.
* resource.Moniker is a serializable string that uniquely identifies
a resource in the Mu system. This is in contrast to resource IDs, which
are generated by resource providers and generally opaque to the Mu
system. See marapongo/mu#69 for more information about monikers and some
of their challenges (namely, designing a stable algorithm).
* resource.Snapshot is a "snapshot" taken from a graph of resources. This
is a transitive closure of state representing one possible configuration
of a given environment. This is what plans are created from. Eventually,
two snapshots will be diffable, in order to perform incremental updates.
One way of thinking about this is that a snapshot of the old world's state
is advanced, one step at a time, until it reaches a desired snapshot of
the new world's state.
* resource.Plan is a plan for carrying out desired CRUD operations on a target
environment. Each plan consists of zero-to-many Steps, each of which has
a CRUD operation type, a resource target, and a next step. This is an
enumerator because it is possible the plan will evolve -- and introduce new
steps -- as it is carried out (hence, the Next() method). At the moment, this
is linearized; eventually, we want to make this more "graph-like" so that we
can exploit available parallelism within the dependencies.
There are tons of TODOs remaining. However, the `mu plan` command is functioning
with these new changes -- including colorization FTW -- so I'm landing it now.
This is part of marapongo/mu#38 and marapongo/mu#41.
This change fixes two aspects of binding names to objects. First, we need
to adjust pointers from prototype functions, since they will have the wrong
"this" (it points to the prototype instance and not the actual object).
Second, we need to bind `super` accesses using the correct prototype object.
* Improve some logging and assertions.
* Initialize the prototype map (it was always empty before).
* Actually memoize the prototype objects in this map.
* During class initialization, ensure the base class is also initialized.
* Trigger class initialization anytime a property or function is loaded
from that class. Similarly, trigger module initialization on loads too.
* Treat For/ForIn/ForOfStatements as legal parents for local variables.
This ensures for loops at the top-level in a module have their variables
treated as locals, rather than module properties. Add a test also.
* Track source locations for more nodes.
* Mark auto-generated constructors as public so that they may be called.
* Fix the naming of class init methods; it should be ".init", not ".ctor".
* Clone the loop test cases into both function and module top-level variants.
One guiding principle for what makes it into the MuIL AST is that
the gap between source language and AST should not be too great; the
projection of switch statements from MuJS into MuIL clearly violated
that principle, particularly considering that the logic wasn't even
right due to the incorrect emulation of conditional breaks.
Instead of digging deeper into the hole, I've encoded switch logic
in the AST, and implemented support in the evaluator.
* Add a TODO as a reminder to implement number toString formatting.
* Change the Loreley delimiters to something obscure ("<{%" and "%}>")
to avoid conflicting with actual characters we might use in messages.
Also, make the assertions more descriptive should Loreley fail.
* Rip out a debug.PrintStack() in verbose logging.
* Check the underlying pointer's object type for +=, not the pointer type.
This change implements intrinsic function support in the runtime.
The basic idea is to swap out references to MuIL functions with
runtime-implemented functionality, for operations that cannot be
expressed in the MuIL-subset. For example, this change includes
two such cases: 1) a mu.runtime.isFunction(obj) function to check if
the target object is a function; and 2) a
mu.runtime.dynamicInvoke(obj,obj,obj[]) function to dynamically
invoke a target object.
These will be used most immediately to implement ECMAScript toString
functionality in the MuJS runtime library, however it will also come
in handy shortly to implement things like printf (marapongo/mu#86),
string and array functions (marapongo/mu#80), and, eventually, the
ECMAScript-specific operators and behavior (marapongo/mu#61).
This change generalizes the support for default modules (added as
part of marapongo/mu#57), to use an "alias" table. This is just a
table of alternative module names to the real defining module name.
The default module is now just represented as a mapping from the
special default module token, ".default", to the actual defining
module. This generalization was necessary to support Node.js-style
alternative names for modules, like "lib" mapping to "lib/index".
Previously, I had thought we would ask MetaMu compilers to map their
own exception/error types to ours. That causes some complexity, however,
in that the exception types in each language (if they even exist) are
"different". So we would need to wrap things, etc., which seems cumbersome.
Partly I had done this after having been burned on the CLR by permitting
throws of any types; e.g., C++/CLI could throw integers, which would rip
through C# code unknowingly, because all C# throws had to derive from the
Exception base class. This actually caused some security issues!
But, we're in a different place, and a different time. The first three
languages I envision supporting -- ECMAScript, Python, and Ruby -- each
permit you to throw anything. And the only other languages I am seriously
contemplating right now -- Go -- doesn't even have exceptions.
So this change backs off and is less opinionated about what is thrown.
Instead, we track stack traces, etc., in the unwind information, and happily
propagate whatever object is thrown.
This change more accurately implements ECMAScript prototype chains.
This includes using the prototype chain to lookup properties when
necessary, and copying them down upon writes.
This still isn't 100% faithful -- for example, classes and
constructor functions should be represented as real objects with
the prototype link -- so that examples like those found here will
work: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/constructor.
I've updated marapongo/mu#70 with additional details about this.
I'm sure we'll be forced to fix this as we encounter more "dynamic"
JavaScript. (In fact, it would be interesting to start running the
pre-ES6 output of TypeScript through the compiler as test cases.)
See http://www.ecma-international.org/ecma-262/6.0/#sec-objects
for additional details on the prototype chaining semantics.
The complexity of the last round of property initialization code was
starting to bug me. I've switched away from lazily initializing them
and now eagerly initialize at the proper times: module initialization,
class initialization (for statics), and object allocation (for instances).
This includes the logic around readonly and freezing. The code is a
lot simpler now and actually enforces some invariants that we'd like to
enforce, like not silently adding new properties when a supposedly static
member load is happening (all such accesses should be dynamic ones).
This change accurately enforces readonly properties. Namely, they
should not be written to anywhere outside of the respective initializer,
but writes must be allowed within them. This means the module initializer
for module properties, the class initializer for statics, and the object
constructor for instance properties.
Now that some name lookups are context-sensitive (namely, whether
or not we should use the export or member table for inter vs. intra
module token references), we need to faithfully track the context.
This change fixes up some issues in the big export refactoring;
namely, we need to chase down references one link at a time during
binding, before they settle. This is because an export might depend
on another export, which might depend on ...
This change redoes the way module exports are represented. The old
mechanism -- although laudible for its attempt at consistency -- was
wrong. For example, consider this case:
let v = 42;
export { v };
The old code would silently add *two* members, both with the name "v",
one of which would be dropped since the entries in the map collided.
It would be easy enough just to detect collisions, and update the
above to mark "v" as public, when the export was encountered. That
doesn't work either, as the following two examples demonstrate:
let v = 42;
export { v as w };
let x = w; // error!
This demonstrates:
* Exporting "v" with a different name, "w" to consumers of the
module. In particular, it should not be possible for module
consumers to access the member through the name "v".
* An inability to access the exported name "w" from within the
module itself. This is solely for external consumption.
Because of this, we will use an export table approach. The exports
live alongside the members, and we are smart about when to consult
the export table, versus the member table, during name binding.
This commit implements array l-values (for dynamic loads only, since we do
not yet ever produce static ones). Note that there are some ECMAScript
compliance noteworthies about this change, which are captured in comments.
Namely, we are emulating ECMAScript's ability to index into an array
anywhere (except for negative numbers, which is a problem). This is in
contrast to the usual approach of throwing an error on out-of-bounds access,
which will crop up when we move on to other languages like Python. And yet we
are usuing a real array as the backing store, which can cause problems with
some real ECMAScript programs that use sparse arrays and expect the "bag of
properties" approach to keep memory usage reasonable.
The work item marapongo/mu#70 tracks this among other things.
This change plucks array element types out of more AST nodes, tightening
up the MuIL that MuJS produces. It also adds a few cases where we should
be emitting "object", and permits more dynamic conversions (which are
technically unsafe, but necessary for the sort of duck typing we anticipate).
In select few cases (right now, just nulls and boolean true/false),
we use predefined object constants to avoid allocating wastefully.
In the future, it's possible we will do this for other types, like
interned strings. So, the graph generator should tolerate this.
This change starts tracking all objects in our MuGL graph. The reason is
that resources can be buried deep within a nest of objects, and unless we
do "on the fly" reachability analysis, we can't know a priori whether any
given object will be of interest or not. So, we track 'em all. For large
programs, this would obviously create space leak problems, so we'll
eventually, I assume, want to prune the graph at some point.
With this change, the EC2instance example produces a (gigantic) graph!
This change implements stack traces. This is primarily so that we
can print out a full stack trace in the face of an unhandled exception,
and is done simply by recording the full trace during evaluation
alongside the existing local variable scopes.
In dynamic scenarios, property keys could be arbitrary strings, including
invalid identifiers. This change reflects that in the runtime representation
of objects and also in the object literal and overall indexing logic.
This change "kicks the can" of dealing with `dynamic` down the road.
Namely, the binder will be permissive about letting dynamic objects
flow through to the evaluation phase. At that point, the evaluator
will be responsible for performing duck typing, etc.
This change introduces support for dynamic object literals. If the
type of a literal is dynamic, then the property initializers for that
literal are naked strings; if the type of a literal is anything else,
however, we expect that the property names are full blown member tokens.
This change prepares to mimick the TypeScript behavior of `any` which,
it turns out, is closer to Python and Ruby duck typing than what we have
here. It also introduces `object` to mean "the base of all types".
MuJS has been updated to map `any` to `dynamic`.
At this point, the conversion logic now distinguishes between NoConversion,
ImplicitConversion, and AutoCastConversion cases, the latter being a case
in which `dynamic` is the source and so a duck-type-style cast must occur.
Right now we do not honor the AutoCastConversion case anywhere, so we still
have not achieved the duck-typing semantics of the source language.
This is part of fixing marapongo/mu#79.
After a bit more thinking, we will create new SDK packages for each
of the languages we wish to support writing resource providers in.
This is where the RPC goo will live, so I have created a new sdk/
directory, moved the Protobuf/gRPC definitions underneath sdk/proto/,
and put the generated code into sdk/go/ and sdk/js/.
This change moves the RPC definitions to the pkg/murpc package,
underneath the proto folder. This is to ensure that the resulting
Protobufs get a good package name "murpc" that is unique within the
overall toolchain. It also includes a `generate.sh` script that
can be used to manually regenerate client/server code. Finally,
we are actually checking in the generated files underneath pkg/murpc.
This change adds a --dot option to the eval command, which will simply
output the MuGL graph using the DOT language. This allows you to use
tools like Graphviz to inspect the resulting graph, including using the
`dot` command to generate images (like PNGs and whatnot).
For example, the simple MuGL program:
class C extends mu.Resource {...}
class B extends mu.Resource {...}
class A extends mu.Resource {
private b: B;
private c: C;
constructor() {
this.b = new B();
this.c = new C();
}
}
let a = new A();
Results in the following DOT file, from `mu eval --dot`:
strict digraph {
Resource0 [label="A"];
Resource0 -> {Resource1 Resource2}
Resource1 [label="B"];
Resource2 [label="C"];
}
Eventually the auto-generated ResourceN identifiers will go away in
favor of using true object monikers (marapongo/mu#76).
Glog doesn't actually print out the stack traces for all goroutines,
when --logtostderr is enabled, the same way it normally does. This
makes debugging more complex in some cases. So, we'll manually do it.
This change renames the old Error type to Exception -- more consistent
with our AST, etc. nodes anyway -- and introduces a new Error type ("<error>")
to use when something during typechecking or binding fails.
The old way led to errors like:
error: MU504: tags.ts:32:18: Symbol 'Tag:push' not found
error: MU522: tags.ts:32:8: Cannot invoke a non-function; 'any' is not a function
This is because of cascading errors during type-checking; the symbol not found
error means we cannot produce the right type for the function invoke that
consumes it. But the 'any' part is weird. Instead, this new change produces:
error: MU504: tags.ts:32:18: Symbol 'Tag:push' not found
error: MU522: tags.ts:32:8: Cannot invoke a non-function; '<error>' is not a function
It's slightly better. And furthermore, it gives us a leg to stand on someday
should we decide to get smarter about detecting cascades and avoiding issuing
the secondary error messages (we can just check for the Error type).
This change fixes a few more phasing issues in the compiler. Namely,
it now splits all passes into three distinct phases:
1. Declarations: simply populating names.
2. Definitions: chasing down any references to other names from those
declared entities. For instance, base classes, other modules, etc.
3. Bodies: fully type-checking everything else, which will depend
upon both declarations and definitions being fully present.
This changes a few things with dependency probing:
1) Probe for Mupack files, not Mufiles.
2) Substitute defaults in the PackageURL before probing.
3) Trace the full search paths when an import fails to resolve.
This will help diagnose dependency resolution issues.
This adds support for sugared "*" semvers, which is a shortcut for
">=0.0.0" (in other words, any version matches). This is a minor part
of marapongo/mu#18. I'm just doing it now as a convenience for dev
scenarios, as we start to do real packages and dependencies between them.
This change permits object literals with the type of `any`; although
they will typically get coerced to record/interface types prior to use,
there is no reason to ban `any`. In fact, it is the idiomatic way of
encoding "objects as bags of properties" that is common to dynamic
languages like JavaScript, Python, etc.
This change adds support for loading from export locations. All we
need to do is keep chasing the referent pointer until we bottom out
on an actual method, property, etc.
This change fixes a whole host of issues with our current token binding
logic. There are two primary aspects of this change:
First, the prior token syntax was ambiguous, due to our choice of
delimiter characters. For instance, "/" could be used both as a module
member delimiter, in addition to being a valid character for sub-modules.
The result is that we could not look at a token and know for certain
which kind it is. There was also some annoyance with "." being the
delimiter for class members in addition to being the leading character
for special names like ".this", ".super", and ".ctor". Now, we just use
":" as the delimiter character for everything. The result is unambiguous.
Second, the simplistic token table lookup really doesn't work. This is
for three reasons: 1) decorated types like arrays, maps, pointers, and
functions shouldn't need token lookup in the classical sense; 2) largely
because of decorated naming, the mapping of token pieces to symbolic
information isn't straightforward and requires parsing; 3) default modules
need to be expanded and the old method only worked for simple cases and,
in particular, would not work when combined with decorated names.
Previously we asserted that typemap entries are never nil. It turns
out we represent "void" as the absence of a type in MuIL, and so we need
to permit these for constructors, etc.
This change further rearranges the phasing of binding to account for
the fact that class definitions may freely reference exports, which won't
be known until after binding class names. Therefore, we bind class names
first and then the definitions (function signatures and variables).
This brings the AWS MuPackage's number of verification errors down to 84
from 136.
The old method of specifying a default module for a package was using
a bit on the module definition itself. The new method is to specify the
module name as part of the package definition itself.
The new approach makes more sense for a couple reasons. 1) it doesn't
make sense to have more than one "default" for a given package, and yet
the old model didn't prevent this; the new one prevents it by construction.
2) The defaultness of a module is really an aspect of the package, not the
module, who is generally unaware of the package containing it.
The other reason is that I'm auditing the code for nondeterministic map
enumerations, and this came up, which simply pushed me over the edge.
The prior module binding logic was naive. It turns out that, because of inter-module
references within the same package, the order of binding certain modules and its members
within a package matters. This leads to a more Java/C#/JavaScript/etc. multi-pass model,
versus a classical single-pass C/C++ model (which requires forward declarations).
To do this, we bind things in this order within a package:
* First, we add an entry for each module. It is largely empty, but at least names resolve.
* Second, we bind all imports. Since all modules are known, all inter-module tokens resolve.
* Next, we can bind class symbols. Note that we must do this prior to module properties,
methods, and exports, since they might be referenced and need to exist first. As before,
we do not yet bind function bodies, since those can reference anything.
* Now we bind all module "members", meaning any module scoped properties and methods. Again,
we do not bind function bodies just yet. This is just a symbolic binding step.
* Next, we can finally bind exports. Because exports may reference any members, all of the
above must have been done first. But all possibly exported symbols are now resolved.
* Finally, we can bind function bodies.
This adds a `mu verify` command that simply runs the verification
pass against a MuPackage and its MuIL. This is handy for compiler
authors to verify that the right stuff is getting emitted.
This is pretty worthless, but will help me debug some issues locally.
Eventually we want MuGL to be fully serializable, including the option
to emit DOT files.
This change actually invokes the OnVariableAssign interpreter hook
at the right places. (Renamed from OnAssignProperty, as it will now
handle all variable assignments, and not just properties.) This
requires tracking a bit more information about l-values so that we
can accurately convey the target object and symbol associated with
the assignment (resulting in the new "location" struct type).
This change lowers the information collected about resource allocations
and dependencies into the MuGL graph representation.
As part of this, we've moved pkg/compiler/eval out into its own top-level
package, pkg/eval, and split up its innards into a smaller sub-package,
pkg/eval/rt, that contains the Object and Pointer abstractions. This
permits the graph generation logic to use it without introducing cycles.
This change refactors the interpreter hooks into a first class interface
with many relevant event handlers (including enter/leave functions for
packages, modules, and functions -- something necessary to generate object
monikers). It also includes a rudimentary start for tracking actual object
allocations and their dependencies, a step towards creating a MuGL graph.
This change adds a new Allocator type that will handle all object allocations
that occur during evaluation. This allows us to hook into interesting lifecycle
events that are required in order to create a MuGL resource graph.