This change rejiggers a few things so that we can more clearly introduce
a boundary between front- and back-end compiler phases, including sharing more,
like a diagnostics sink. Future extensions will include backend code-generation
options.
This change includes a few steps towards AWS backend code-generation:
* Add a BoundDependencies property to ast.Stack to remember the *ast.Stack
objects bound during Stack binding.
* Make a few CloudFormation properties optional (cfOutput Export/Condition).
* Rename clouds.ArchMap, clouds.ArchNames, schedulers.ArchMap, and
schedulers.ArchNames to clouds.Values, clouds.Names, schedulers.Values,
and schedulers.Names, respectively. This reads much nicer to my eyes.
* Create a new anonymous ast.Target for deployments if no specific target
was specified; this is to support quick-and-easy "one off" deployments,
as will be common when doing local development.
* Sketch out more of the AWS Cloud implementation. We actually map the
Mu Services into CloudFormation Resources; well, kinda sorta, since we
don't actually have Service-specific logic in here yet, however all of
the structure and scaffolding is now here.
This change adds a Backend Phase to the compiler, implemented by each of the
cloud/scheduler implementations. It also reorganizes some of the modules to
ensure we can do everything we need without cycles, including introducing the
mu/pkg/compiler/backends package, under which the clouds/ and schedulers/
sub-packages now reside. The backends.New(Arch) factory function acts as the
entrypoint into the entire thing so callers can easily create new Backend instances.
This change is more consistent with our approach to file extensions
elsewhere in the compiler, and prevents an explosion of APIs should we
ever want to support more.
This adds some tests around cloud targeting, in addition to enabling builds
to use in-memory Mufiles (mostly to make testing simpler, but this is a
generally useful capability to have when hosting the compiler API).
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
This change creates a new mu/pkg/compiler/core package for any fundamental
compiler types that need to be shared among the various compiler packages
(.../compiler, .../compiler/clouds/..., and .../compiler/schedulers/...).
This avoids package cycles.
This adds two packages:
mu/pkg/compiler/clouds
mu/pkg/compiler/schedulers
And introduces enums for the cloud targets we expect to support.
It also adds the ability at the command line to specify a provider;
for example:
$ mu build --target=aws # AWS native
$ mu build --target=aws:ecs # AWS ECS
$ mu build -t=gcp:kubernetes # Kube on GCP
This change introduces a check during parse-tree analysis that dependencies
are valid, along with some tests. Note that this could technically happen later
during semantic analysis and I will likely move it so that we can get better
diagnostics (more errors before failing). I've also cleaned up and unified some
of the logic by introducing the general notion of a Visitor interface, which the
parse tree analyzer, binder, and analyzers to come will all implement.
This change begins to lay the groundwork for doing semantic analysis and
lowering to the cloud target's representation. In particular:
* Split the mu/schema package. There is now mu/ast which contains the
core types and mu/encoding which concerns itself with JSON and YAML
serialization.
* Notably I am *not* yet introducing a second AST form. Instead, we will
keep the parse tree and AST unified for the time being. I envision very
little difference between them -- at least for now -- and so this keeps
things simpler, at the expense of two downsides: 1) the trees will be
mutable (which turns out to be a good thing for performance), and 2) some
fields will need to be ignored during de/serialization. We can always
revisit this later when and if the need to split them arises.
* Add a binder phase. It is currently a no-op.
This change adds a few more compiler tests and rearranges some bits and pieces
that came up while doing so. For example, we now issue warnings for incorrect
casing and/or extensions of the Mufile (and test these conditions). As part of
doing that, it became clear the layering between the mu/compiler and mu/workspace
packages wasn't quite right, so some logic got moved around; additionally, the
separation of concerns between mu/workspace and mu/schema wasn't quite right, so
this has been fixed also (workspace just understands Mufile related things while
schema understands how to unmarshal the specific supported extensions).
This change includes some progress on actual compilation (albeit with several
TODOs remaining before we can actually spit out a useful artifact). There are
also some general cleanups sprinkled throughout. In a nutshell:
* Add a compiler.Context object that will be available during template expansion.
* Introduce a diag.Document abstraction. This is better than passing raw filenames
around, and lets us embellish diagnostics as we go. In particular, we will be
in a better position to provide line/column error information.
* Move IO out of the Parser and into the Compiler, where it can cache and reuse
Documents. This will become important as we start to load up dependencies.
* Rename PosRange to Location. This reads nicer with the new Document terminology.
* Rename the mu/api package to mu/schema. It's likely we will need to introduce a
true AST that is decoupled from the serialization format and contains bound nodes.
As a result, treating the existing types as "schema" is more honest.
* Add in a big section of TODOs at the end of the compiler.Compiler.Build function.
* Rename Meta to Metadata.
* Rename Target's CloudOS and CloudScheduler properties to Cloud
and Scheduler, respectively. Also rename Target's JSON properties
to match (they had drifted); they are now "cloud" and "scheduler".
* Rename Diags() to Diag() on the Compiler and Parser interfaces.
* Rename defaultDiags to defaultSink, to match the interface name.
* Add a few useful logging outputs.
This adds a bunch of general scaffolding and the beginning of a `build` command.
The general engineering scaffolding includes:
* Glide for dependency management.
* A Makefile that runs govet and golint during builds.
* Google's Glog library for logging.
* Cobra for command line functionality.
The Mu-specific scaffolding includes some packages:
* mu/pkg/diag: A package for compiler-like diagnostics. It's fairly barebones
at the moment, however we can embellish this over time.
* mu/pkg/errors: A package containing Mu's predefined set of errors.
* mu/pkg/workspace: A package containing workspace-related convenience helpers.
in addition to a main entrypoint that simply wires up and invokes the CLI. From
there, the mu/cmd package takes over, with the Cobra-defined CLI commands.
Finally, the mu/pkg/compiler package actually implements the compiler behavior.
Or, it will. For now, it simply parses a JSON or YAML Mufile into the core
mu/pkg/api types, and prints out the result.
