As part of marapongo/mu#9, we want to enable extensible schema types
for stronger typechecking at compile-time. JSON Schema seems like a
decent starting place (http://json-schema.org/), although it's not yet
clear whether we want to use its module/naming schema or our own. I
suspect we want to use our own so that stack schemas can be managed
using the same discipline as stack management generally.
This change eliminates the special type mu/extension in favor of extensible
intrinsic types. This subsumes the previous functionality while also fixing
a number of warts with the old model.
In particular, the old mu/extension approach deferred property binding until
very late in the compiler. In fact, too late. The backend provider for an
extension simply received an untyped bag of stuff, which it then had to
deal with. Unfortunately, some operations in the binder are inaccessible
at this point because doing so would cause a cycle. Furthermore, some
pertinent information is gone at this point, like the scopes and symtables.
The canonical example where we need this is binding services names to the
services themselves; e.g., the AWS CloudFormation "DependsOn" property should
resolve to the actual service names, not the string values. In the limit,
this requires full binding information.
There were a few solutions I considered, including ones that've required
less code motion, however this one feels the most elegant.
Now we permit types to be marked as "intrinsic." Binding to these names
is done exactly as ordinary name binding, unlike the special mu/extension
provider name. In fact, just about everything except code-generation for
these types is the same as ordinary types. This is perfect for the use case
at hand, which is binding properties.
After this change, for example, "DependsOn" is expanded to real service
names precisely as we need.
As part of this change, I added support for three new basic schema types:
* ast.StringList ("string[]"): a list of strings.
* ast.StringMap ("map[string]any"): a map of strings to anys.
* ast.ServiceList ("service[]"): a list of service references.
Obviously we need to revisit this and add a more complete set. This work
is already tracked by marapongo/mu#9.
At the end of the day, it's likely I will replace all hard-coded predefined
types with intrinsic types, for similar reasons to the above.
This change introduces the notion of "perturbing" properties. Changing
one of these impacts the live service, possibly leading to downtime. As
such, we will likely encourage blue/green deployments of them just to be
safe. Note that this is really just a placeholder so I can keep track of
metadata as we go, since AWS CF has a similar notion to this.
I'm not in love with the name. I considered `interrupts`, however,
I must admit I liked that `readonly` and `perturbs` are symmetric in
the number of characters (meaning stuff lines up nicely...)
This change adds the notion of readonly properties to stacks. Although these
*can* be "changed", doing so implies recreation of the resources all over again.
As a result, all dependents must be recreated, in a cascading manner.
I've gone backwards and forwards on the design for dependency version
management. However, I think what's written in this commit represents
a pretty sane "sweet spot" between all available options.
In a nutshell, anytime reference to a stack type in your Mufile is a
full-blown StackRef; in other words, it has a protocol (e.g., "https://"),
a base URL (e.g., "hub.mu.com/"), a name (e.g., "aws/s3/bucket"), and a
version ("@^1.0.6"). Each reference carries all of these components.
For convenience, you may omit the components. In that case, Mu chooses
reasonable defaults:
* "https://" as the default protocol (or "git://"; this is TBD).
* "hub.mu.com/" as the default base URL.
* "@latest" as the default version number.
Note that a version can be "latest" to mean "tip", a specific SHA hash
to pin to a precise version, or a semantic version number/range.
I had originally shied away from specifying versions inline as you reference
stacks in your Mufile, and was leaning towards an explicit dependencies
section, however I was swayed for two reasons:
1. It's very common to only consume a given stack once in a file. Needing
to specify it in two places each time is verbose and violates DRY.
2. We have decided that each Mufile stands on its own and forms the unit
of distribution. I had previously thought we might move dependencies
out of Mufiles into a "package manager" specification. Lacking that,
there is even less reason to call them out in a dedicated section.
Now, managing all these embedded version numbers across multiple stacks in
a single workspace would have been annoying. (Many edits for a single
version bump.) Instead, I've added provisions for storing this in your
workspace.yaml file. The way it works is if any StackRef lacks a version
number, before defaulting to "@latest" we check the workspace.yaml file and,
if a default is found in there, we will use it. For example:
dependencies:
aws/s3/bucket: ^1.0.6
The provision for pinning an entire namespace is also preserved. E.g.:
dependencies:
aws/...: ^1.0.6
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
This change introduces the notion of "Stack subclassing" in two ways:
1. A Stack may declare that it subclasses another one using the base property:
name: mystack
base: other/stack
.. as before ..
2. A Stack may declare that it is abstract; in other words, that it is meant
solely for subclassing, and cannot be compiled and deployed independently:
name: mystack
abstract: true
.. as before ..
Note that non-abstract Stacks are required to declare at least one Service,
whether that is public, private, or both.
This change separates the metadata specification from the targets doc,
since they are both meant to be consumable independent of one another
and will evolve at their own paces.
This sketches out some more details in the Metadata format, in addition to
starting to articulate what the AWS IaaS provider looks like. Sadly, more
questions than answers, but it's good to have the placeholders...
Simple developer scenarios can be done solely in terms of Stacks. However, to
support more sophisticated cases -- where multiplexing many Stacks onto a shared
set of resources is preferred (for cost savings, provisioning time savings,
control, etc) -- we need to introduce the concept of a Cluster. Each Cluster has
a fixed combination of IaaS/CaaS provider, decided at provisioning time.
This just contains primarily scaffolding, however lays out a general direction
and table of contents for the metadata specification document (marapongo/mu#1).
