The prior workaround to avoid truncated pending stdout writes, it
turns out, doesn't actually work. (Piping output leads to more buffering
and asynchrony, and turned up additional problems.) Digging through some
GitHub issues led me to these "best practices":
https://nodejs.org/api/process.html#process_process_exit_code
The reason this is problematic is because writes to process.stdout in
Node.js are sometimes non-blocking and may occur over multiple ticks of
the Node.js event loop. Calling process.exit(), however, forces the
process to exit before those additional writes to stdout can be performed.
Rather than calling process.exit() directly, the code should set the
process.exitCode and allow the process to exit naturally by avoiding
scheduling any additional work for the event loop.
This change adopts this part of the best practices, by simply setting
exitCode upon normal termination and letting the event loop quiesce.
Note that I am still not obeying the other part of the guidance:
If it is necessary to terminate the Node.js process due to an error
condition, throwing an uncaught error and allowing the process to
terminate accordingly is safer than calling process.exit().
Somewhat confusingly, writes to process.stderr do not suffer from these
same problems, because writes to stderr are synchronous. We prefer to
tear down the process gracefully, without an unhandled exception, and
we are okay losing some stdout writes as a result, given that all error-
related ones will have gone to stderr.
This adds scaffolding but no real functionality yet, as part of
marapongo/mu#41. I am landing this now because I need to take a
not-so-brief detour to gut and overhaul the core of the existing
compiler (parsing, semantic analysis, binding, code-gen, etc),
including merging the new pkg/pack contents back into the primary
top-level namespaces (like pkg/ast and pkg/encoding).
After that, I can begin driving the compiler to achieve the
desired effects of mu compile, first and foremost, and then plan
and apply later on.
This change now recognizes Mu.yaml files, in addition to Mu.json,
from the MuJS compiler. Not the most important thing in the world,
however all of our project files are in YAML and it's less work to
implement this than to convert them all to JSON ...
This change tracks the set of imported modules in the ast.Module
structure. Although we can in principle gather up all imports simply
by looking through the fully qualified names, that's slightly hokey;
and furthermore, to properly initialize all modules, we need to know
in which order to do it (in case there are dependencies). I briefly
considered leaving it up to MetaMu compilers to inject the module
initialization calls explicitly -- for infinite flexibility and perhaps
greater compatibility with the source languages -- however, I'd much
prefer that all Mu code use a consistent module initialization story.
Therefore, MetaMus declare the module imports, in order, and we will
evaluate the initializers accordingly.
This change emits more types. In particular:
* Previously, only primitive types got emitted, yielding "any" for any
custom types. Now we emit custom types, including fully qualified
module names for type references resolving to imported modules.
* Prior to this change, we erroneously used the type node on the function
declaration itself as an approximation for return type. To get the
true return type, we need to dig through a few nodes, including the
Declaration and Signature. This change now properly emits return types.
This doesn't close out marapongo/mu#46, however we are getting close.
The prior code wasn't extracting the member part of the token name
when reexporting symbols, meaning the member was fully qualified. For
example, when reexporting a class "C" from another module "other",
the resulting name was "other:C". This is correct for the export node's
token -- so it can be resolved correctly to its source -- however,
clearly the identifier for the export should not contain the module
part. This change strips it in the appropriate place.
The compiler was emitting incorrect variable initialization code,
resulting in a "=" BinaryOperatorExpression in a Statement position.
This fix wraps it in an ExpressionStatement so it results in valid IL.
This change makes considerable progress on the `mu describe` command;
the only thing remaining to be implemented now is full IL printing. It
now prints the full package/module structure.
For example, to print the set of exports from our scenarios/point test:
$ mujs tools/mujs/tests/output/scenarios/point/ | mu describe - -e
package "scenarios/point" {
dependencies []
module "index" {
class "Point" [public] {
method "add": (other: any): any
property "x" [public, readonly]: number
property "y" [public, readonly]: number
method ".ctor": (x: number, y: number): any
}
}
}
This is just a pretty-printed, but is coming in handy with debugging.
The NewExpression AST node type was missing a JSON annotation on
its Type field, leading to decoding errors.
Now, with this, the full suite of MuJS test cases can be unmarshaled
into fully populated MuPack and MuIL structures.
MuIL doesn't support identifiers as expressions; instead, an explicit
LoadLocationExpression is required. There was a problem in the current
MuJS translation process -- masked by TypeScript silently duck typing
from Identifier to Expression -- but our Mu decoding library discovered
it. This fixes the problem by wrapping Identifiers used in Expression
positions in a proper LoadLocationExpression node.
This change overhauls the approach to custom decoding. Instead of decoding
the parts of the struct that are "trivial" in one pass, and then patching up
the structure afterwards with custom decoding, the decoder itself understands
the notion of custom decoder functions.
First, the general purpose logic has moved out of pkg/pack/encoding and into
a new package, pkg/util/mapper. Most functions are now members of a new top-
level type, Mapper, which may be initialized with custom decoders. This
is a map from target type to a function that can decode objects into it.
Second, the AST-specific decoding logic is rewritten to use it. All AST nodes
are now supported, including definitions, statements, and expressions. The
overall approach here is to simply define a custom decoder for any interface
type that will occur in a node field position. The mapper, upon encountering
such a type, will consult the custom decoder map; if a decoder is found, it
will be used, otherwise an error results. This decoder then needs to switch
on the type discriminated kind field that is present in the metadata, creating
a concrete struct of the right type, and then converting it to the desired
interface type. Note that, subtly, interface types used only for "marker"
purposes don't require any custom decoding, because they do not appear in
field positions and therefore won't be encountered during the decoding process.
This change splits up the decoding logic into multiple files, to
mirror the AST package structure that the functions correspond to.
Additionally, there is now less "loose" reflection and dynamic lookup
code scattered throughout; it is now consolidated into the decoder,
with a set of "generic" functions like `fieldObject`, `asString`, etc.
This change implements custom class member decoding. As with module methods,
the function body AST nodes remain nil, as custom AST decoding isn't yet done.
This change begins to implement some of the AST custom decoding, beneath
the Package's Module map. In particular, we now unmarshal "one level"
beyond this, populating each Module's ModuleMember map. This includes
Classes, Exports, ModuleProperties, and ModuleMethods. The Class AST's
Members have been marked "custom", in addition to Block's Statements,
because they required kind-directed decoding. But Exports and
ModuleProperties can be decoded entirely using the tag-directed decoding
scheme. Up next, custom decoding of ClassMembers. At that point, all
definition-level decoding will be done, leaving MuIL's ASTs.
This change just moves the assertion/failure functions from the pkg/util
package to pkg/util/contract, so things read a bit nicer (i.e.,
`contract.Assert(x)` versus `util.Assert(x)`).
This fixes a few things so that MuPackages now unmarshal:
* Mark Module.Members as requiring "custom" decoding. This is required
because that's the first point in the tree that leverages polymorphism.
Everything "above" this unmarshals just fine (e.g., package+modules).
* As such, stop marking Package.Modules as "custom".
* Add the Kind field to the Node. Although we won't use this for type
discrimination in the same way, since Go gives us RTTI on the structs,
it is required for unmarshaling (to avoid "unrecognized fields" errors)
and it's probably handy to have around for logging, messages, etc.
* Mark Position.Line and Column with "json" annotations so that they
unmarshal correctly.
This change adjusts pointers correctly when unmarshaling into target
pointer types. This handles arrays and maps of pointer elements, in
addition to consolidating existing logic for marshaling into a
destination top-level pointer as well.
This change eliminates boilerplate decoding logic in all the different
data structures, and instead uses a new tag-directed decoding scheme.
This works a lot like the JSON deserializers, in that it recognizes the
`json:"name"` tags, except that we permit annotation of fields that
require custom deserialization, as `json:"name,custom"`. The existing
`json:"name,omitempty"` tag is recognized for optional fields.
This adds basic custom decoding for the MuPack metadata section of
the incoming JSON/YAML. Because of the type discriminated union nature
of the incoming payload, we cannot rely on the simple built-in JSON/YAML
unmarshaling behavior. Note that for the metadata section -- what is
in this checkin -- we could have, but the IL AST nodes are problematic.
(To know what kind of structure to creat requires inspecting the "kind"
field of the IL.) We will use a reflection-driven walk of the target
structure plus a weakly typed deserialized map[string]interface{}, as
is fairly customary in Go for scenarios like this (though good libaries
seem to be lacking in this area...).
This command will simply pretty-print the contents of a MuPackage.
My plan is to use it for my own development and debugging purposes,
however, I suspect it will be generally useful (since MuIL can be
quite verbose). Again, just scaffolding, but I'll flesh it out
incrementally as more of the goo in here starts working...
This change carries over all of the metadata shapes in the MuPack
and MuIL file formats to our Go toolset. This includes creating a
proper discriminated AST type tree along with correct annotations
so that the metadata will serialize and deserialize correctly.
