2016-11-16 02:42:22 +01:00
|
|
|
// Copyright 2016 Marapongo, Inc. All rights reserved.
|
|
|
|
|
|
2016-11-16 18:29:44 +01:00
|
|
|
// This package contains the core Mu abstract syntax tree types.
|
|
|
|
|
//
|
|
|
|
|
// N.B. for the time being, we are leveraging the same set of types for parse trees and abstract syntax trees. The
|
|
|
|
|
// reason is that minimal "extra" information is necessary between front- and back-end parts of the compiler, and so
|
|
|
|
|
// reusing the trees leads to less duplication in types and faster runtime performance. As the compiler matures in
|
|
|
|
|
// functionality, we may want to revisit this. The "back-end-only" parts of the data structures are easily identified
|
|
|
|
|
// because their fields do not map to any serializable fields (i.e., `json:"-"`).
|
|
|
|
|
//
|
|
|
|
|
// Another controversial decision is to mutate nodes in place, rather than taking the performance hit of immutability.
|
|
|
|
|
// This can certainly be tricky to deal with, however, it is simpler and we can revisit it down the road if needed.
|
|
|
|
|
// Of course, during lowering, sometimes nodes will be transformed to new types entirely, allocating entirely anew.
|
|
|
|
|
package ast
|
2016-11-16 02:42:22 +01:00
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
import (
|
|
|
|
|
"github.com/marapongo/mu/pkg/diag"
|
|
|
|
|
)
|
|
|
|
|
|
2016-11-16 02:42:22 +01:00
|
|
|
// Name is an identifier. Names may be optionally fully qualified, using the delimiter `/`, or simple. Each element
|
|
|
|
|
// conforms to the regex [A-Za-z_][A-Za-z0-9_]*. For example, `marapongo/mu/stack`.
|
|
|
|
|
type Name string
|
|
|
|
|
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
// Ref is a dependency reference. It is "name-like", in that it contains a Name embedded inside of it, but also carries
|
|
|
|
|
// a URL-like structure. A Ref starts with an optional "protocol" (like https://, git://, etc), followed by an optional
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// "base" part (like hub.mu.com/, github.com/, etc), followed by the "name" part (which is just a Name), followed by
|
|
|
|
|
// an optional "@" and version number (where version may be "latest", a semantic version range, or a Git SHA hash).
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
type Ref string
|
|
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// Version represents a precise version number. It may be either a Git SHA hash or a semantic version (not a range).
|
|
|
|
|
type Version string
|
|
|
|
|
|
|
|
|
|
// VersionSpec represents a specification of a version that is bound to a precise number through a separate process.
|
|
|
|
|
// It may take the form of a Version (see above), a semantic version range, or the string "latest", to indicate that the
|
|
|
|
|
// latest available sources are to be used at compile-time.
|
|
|
|
|
type VersionSpec string
|
2016-11-16 02:42:22 +01:00
|
|
|
|
2016-11-16 18:29:44 +01:00
|
|
|
// Node is the base of all abstract syntax tree types.
|
|
|
|
|
type Node struct {
|
|
|
|
|
}
|
|
|
|
|
|
Support Workspaces
This change adds support for Workspaces, a convenient way of sharing settings
among many Stacks, like default cluster targets, configuration settings, and the
like, which are not meant to be distributed as part of the Stack itself.
The following things are included in this checkin:
* At workspace initialization time, detect and parse the .mu/workspace.yaml
file. This is pretty rudimentary right now and contains just the default
cluster targets. The results are stored in a new ast.Workspace type.
* Rename "target" to "cluster". This impacts many things, including ast.Target
being changed to ast.Cluster, and all related fields, the command line --target
being changed to --cluster, various internal helper functions, and so on. This
helps to reinforce the desired mental model.
* Eliminate the ast.Metadata type. Instead, the metadata moves directly onto
the Stack. This reflects the decision to make Stacks "the thing" that is
distributed, versioned, and is the granularity of dependency.
* During cluster targeting, add the workspace settings into the probing logic.
We still search in the same order: CLI > Stack > Workspace.
2016-11-22 19:41:07 +01:00
|
|
|
// Workspace defines settings shared amongst many related Stacks.
|
|
|
|
|
type Workspace struct {
|
2016-11-16 18:29:44 +01:00
|
|
|
Node
|
2016-11-16 20:51:50 +01:00
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
Clusters Clusters `json:"clusters,omitempty"` // an optional set of predefined target clusters.
|
2016-11-16 02:42:22 +01:00
|
|
|
Dependencies Dependencies `json:"dependencies,omitempty"`
|
2016-11-23 16:26:45 +01:00
|
|
|
|
|
|
|
|
Doc *diag.Document `json:"-"` // the document from which this came.
|
2016-11-16 02:42:22 +01:00
|
|
|
}
|
|
|
|
|
|
Support Workspaces
This change adds support for Workspaces, a convenient way of sharing settings
among many Stacks, like default cluster targets, configuration settings, and the
like, which are not meant to be distributed as part of the Stack itself.
The following things are included in this checkin:
* At workspace initialization time, detect and parse the .mu/workspace.yaml
file. This is pretty rudimentary right now and contains just the default
cluster targets. The results are stored in a new ast.Workspace type.
* Rename "target" to "cluster". This impacts many things, including ast.Target
being changed to ast.Cluster, and all related fields, the command line --target
being changed to --cluster, various internal helper functions, and so on. This
helps to reinforce the desired mental model.
* Eliminate the ast.Metadata type. Instead, the metadata moves directly onto
the Stack. This reflects the decision to make Stacks "the thing" that is
distributed, versioned, and is the granularity of dependency.
* During cluster targeting, add the workspace settings into the probing logic.
We still search in the same order: CLI > Stack > Workspace.
2016-11-22 19:41:07 +01:00
|
|
|
// Clusters is a map of target names to metadata about those targets.
|
|
|
|
|
type Clusters map[string]Cluster
|
|
|
|
|
|
|
|
|
|
// Cluster describes a predefined cloud runtime target, including its OS and Scheduler combination.
|
|
|
|
|
type Cluster struct {
|
|
|
|
|
Node
|
|
|
|
|
|
|
|
|
|
Default bool `json:"default,omitempty"` // a single target can carry default settings.
|
|
|
|
|
Description string `json:"description,omitempty"` // a human-friendly description of this target.
|
|
|
|
|
Cloud string `json:"cloud,omitempty"` // the cloud target.
|
|
|
|
|
Scheduler string `json:"scheduler,omitempty"` // the cloud scheduler target.
|
|
|
|
|
Options map[string]interface{} `json:"options,omitempty"` // any options passed to the cloud provider.
|
|
|
|
|
|
|
|
|
|
Name string `json:"-"` // name is decorated post-parsing, since it is contextual.
|
|
|
|
|
}
|
|
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// Dependencies maps dependency refs to the semantic version the consumer depends on.
|
|
|
|
|
type Dependencies map[Ref]Dependency
|
|
|
|
|
|
|
|
|
|
// Dependency is metadata describing a dependency target (for now, just its target version).
|
|
|
|
|
type Dependency VersionSpec
|
|
|
|
|
|
|
|
|
|
// Stack represents a collection of private and public cloud resources, a method for constructing them, and optional
|
|
|
|
|
// dependencies on other Stacks (by name).
|
|
|
|
|
type Stack struct {
|
|
|
|
|
Node
|
|
|
|
|
|
|
|
|
|
Name Name `json:"name,omitempty"` // a friendly name for this node.
|
|
|
|
|
Version Version `json:"version,omitempty"` // a specific version number.
|
|
|
|
|
Description string `json:"description,omitempty"` // an optional friendly description.
|
|
|
|
|
Author string `json:"author,omitempty"` // an optional author.
|
|
|
|
|
Website string `json:"website,omitempty"` // an optional website for additional info.
|
|
|
|
|
License string `json:"license,omitempty"` // an optional license governing legal uses of this package.
|
|
|
|
|
Clusters Clusters `json:"clusters,omitempty"` // an optional set of predefined target clusters.
|
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
Base Ref `json:"base,omitempty"` // an optional base Stack type.
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
Abstract bool `json:"abstract,omitempty"` // true if this stack is "abstract" (uninstantiable).
|
|
|
|
|
Properties Properties `json:"properties,omitempty"`
|
|
|
|
|
Services Services `json:"services,omitempty"`
|
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
Predef bool `json:"-"` // true if this is a predefined type (treated specially).
|
|
|
|
|
Doc *diag.Document `json:"-"` // the document from which this came.
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
BoundBase *Stack `json:"-"` // base, if available, is bound during semantic analysis.
|
|
|
|
|
BoundDependencies BoundDependencies `json:"-"` // dependencies are bound during semantic analysis.
|
|
|
|
|
}
|
|
|
|
|
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
// Propertys maps property names to metadata about those propertys.
|
|
|
|
|
type Properties map[string]Property
|
2016-11-16 02:42:22 +01:00
|
|
|
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
// Property describes the requirements of arguments used when constructing Stacks, etc.
|
|
|
|
|
type Property struct {
|
2016-11-16 18:29:44 +01:00
|
|
|
Node
|
2016-11-16 20:51:50 +01:00
|
|
|
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
Type PropertyType `json:"type,omitempty"` // the type of the property; required.
|
|
|
|
|
Description string `json:"description,omitempty"` // an optional friendly description of the property.
|
|
|
|
|
Default interface{} `json:"default,omitempty"` // an optional default value if the caller elides one.
|
|
|
|
|
Optional bool `json:"optional,omitempty"` // true if may be omitted (inferred if a default value).
|
Retain unrecognized service properties
During unmarshaling, the default behavior of the stock Golang JSON marshaler,
and consequently the YAML one we used which mimics its behavior, is to toss away
unrecognized properties. This isn't what we want for two reasons:
First, we want to issue errors/warnings on unrecognized fields to aid in diagnostics;
we will set aside some extensible section for 3rd parties to use. This is not
addressed in this change, however.
Second, and more pertinent, is that we need to retain unrecognized fields for certain
types like services, which are extensible by default.
Until golang/go#6213 is addressed -- imminent, it seems -- we will have to do a
somewhat hacky workaround to this problem. This change contains what I consider to
be the "least bad" in that we won't introduce a lot of performance overhead, and
just have to deal with the slight annoyance of the ast.Services node type containing
both Public/Private *and* PublicUntyped/PrivateUntyped fields alongside one another.
The marshaler dumps property bags into the *Untyped fields, and the parsetree analyzer
expands them out into a structured ast.Service type. Subsequent passes can then
ignore the *Untyped fields altogether.
Note that this would cause some marshaling funkiness if we ever wanted to remarshal
the mutated ASTs back into JSON/YAML. Since we don't do that right now, however, I've
not made any attempt to keep the two pairs in synch. Post-parsetree analyzer, we
literally just forget about the *Untyped guys.
2016-11-19 18:01:23 +01:00
|
|
|
|
|
|
|
|
Name string `json:"-"` // name is decorated post-parsing, since it is contextual.
|
2016-11-16 02:42:22 +01:00
|
|
|
}
|
|
|
|
|
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
// PropertyType stores the name of a property's type.
|
|
|
|
|
type PropertyType Name
|
2016-11-19 19:22:16 +01:00
|
|
|
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
// A set of known property types. Note that this is extensible, so names outside of this list are legal.
|
2016-11-19 19:22:16 +01:00
|
|
|
// TODO: support complex types (like arrays, custom JSON shapes, and so on).
|
|
|
|
|
const (
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
PropertyTypeAny PropertyType = "any" // any structure.
|
|
|
|
|
PropertyTypeString = "string" // a JSON-like string.
|
|
|
|
|
PropertyTypeNumber = "number" // a JSON-like number (integer or floating point).
|
|
|
|
|
PropertyTypeBoolean = "boolean" // a JSON-like boolean (`true` or `false`).
|
|
|
|
|
PropertyTypeService = "service" // an untyped service reference; the runtime manifestation is a URL.
|
2016-11-19 19:22:16 +01:00
|
|
|
)
|
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
// BoundDependencies contains a map of dependencies, populated during semantic analysis.
|
|
|
|
|
type BoundDependencies map[Ref]BoundDependency
|
2016-11-20 16:28:58 +01:00
|
|
|
|
|
|
|
|
// BoundDependency contains information about a binding.
|
|
|
|
|
type BoundDependency struct {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
Ref RefParts // the reference used to bind to this dependency.
|
|
|
|
|
Stack *Stack // the bound stack for this dependency.
|
2016-11-20 16:28:58 +01:00
|
|
|
}
|
Sketch out more AWS backend code-generator bits and pieces
This change includes a few steps towards AWS backend code-generation:
* Add a BoundDependencies property to ast.Stack to remember the *ast.Stack
objects bound during Stack binding.
* Make a few CloudFormation properties optional (cfOutput Export/Condition).
* Rename clouds.ArchMap, clouds.ArchNames, schedulers.ArchMap, and
schedulers.ArchNames to clouds.Values, clouds.Names, schedulers.Values,
and schedulers.Names, respectively. This reads much nicer to my eyes.
* Create a new anonymous ast.Target for deployments if no specific target
was specified; this is to support quick-and-easy "one off" deployments,
as will be common when doing local development.
* Sketch out more of the AWS Cloud implementation. We actually map the
Mu Services into CloudFormation Resources; well, kinda sorta, since we
don't actually have Service-specific logic in here yet, however all of
the structure and scaffolding is now here.
2016-11-19 01:46:36 +01:00
|
|
|
|
2016-11-17 02:30:03 +01:00
|
|
|
// Services is a list of public and private service references, keyed by name.
|
|
|
|
|
type Services struct {
|
Retain unrecognized service properties
During unmarshaling, the default behavior of the stock Golang JSON marshaler,
and consequently the YAML one we used which mimics its behavior, is to toss away
unrecognized properties. This isn't what we want for two reasons:
First, we want to issue errors/warnings on unrecognized fields to aid in diagnostics;
we will set aside some extensible section for 3rd parties to use. This is not
addressed in this change, however.
Second, and more pertinent, is that we need to retain unrecognized fields for certain
types like services, which are extensible by default.
Until golang/go#6213 is addressed -- imminent, it seems -- we will have to do a
somewhat hacky workaround to this problem. This change contains what I consider to
be the "least bad" in that we won't introduce a lot of performance overhead, and
just have to deal with the slight annoyance of the ast.Services node type containing
both Public/Private *and* PublicUntyped/PrivateUntyped fields alongside one another.
The marshaler dumps property bags into the *Untyped fields, and the parsetree analyzer
expands them out into a structured ast.Service type. Subsequent passes can then
ignore the *Untyped fields altogether.
Note that this would cause some marshaling funkiness if we ever wanted to remarshal
the mutated ASTs back into JSON/YAML. Since we don't do that right now, however, I've
not made any attempt to keep the two pairs in synch. Post-parsetree analyzer, we
literally just forget about the *Untyped guys.
2016-11-19 18:01:23 +01:00
|
|
|
// These fields are expanded after parsing:
|
|
|
|
|
Public ServiceMap `json:"-"`
|
|
|
|
|
Private ServiceMap `json:"-"`
|
|
|
|
|
|
|
|
|
|
// These fields are "untyped" due to limitations in the JSON parser. Namely, Go's parser will ignore
|
|
|
|
|
// properties in the payload that it doesn't recognize as mapping to a field. That's not what we want, especially
|
|
|
|
|
// for services since they are highly extensible and the contents will differ per-type. Therefore, we will first
|
|
|
|
|
// map the services into a weakly typed map, and later on during compilation, expand them to the below fields.
|
2016-11-19 21:31:00 +01:00
|
|
|
// TODO[marapongo/mu#4]: support for `json:",inline"` or the equivalent so we can eliminate these fields.
|
Retain unrecognized service properties
During unmarshaling, the default behavior of the stock Golang JSON marshaler,
and consequently the YAML one we used which mimics its behavior, is to toss away
unrecognized properties. This isn't what we want for two reasons:
First, we want to issue errors/warnings on unrecognized fields to aid in diagnostics;
we will set aside some extensible section for 3rd parties to use. This is not
addressed in this change, however.
Second, and more pertinent, is that we need to retain unrecognized fields for certain
types like services, which are extensible by default.
Until golang/go#6213 is addressed -- imminent, it seems -- we will have to do a
somewhat hacky workaround to this problem. This change contains what I consider to
be the "least bad" in that we won't introduce a lot of performance overhead, and
just have to deal with the slight annoyance of the ast.Services node type containing
both Public/Private *and* PublicUntyped/PrivateUntyped fields alongside one another.
The marshaler dumps property bags into the *Untyped fields, and the parsetree analyzer
expands them out into a structured ast.Service type. Subsequent passes can then
ignore the *Untyped fields altogether.
Note that this would cause some marshaling funkiness if we ever wanted to remarshal
the mutated ASTs back into JSON/YAML. Since we don't do that right now, however, I've
not made any attempt to keep the two pairs in synch. Post-parsetree analyzer, we
literally just forget about the *Untyped guys.
2016-11-19 18:01:23 +01:00
|
|
|
PublicUntyped UntypedServiceMap `json:"public,omitempty"`
|
|
|
|
|
PrivateUntyped UntypedServiceMap `json:"private,omitempty"`
|
2016-11-17 02:30:03 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ServiceMap is a map of service names to metadata about those services.
|
|
|
|
|
type ServiceMap map[Name]Service
|
2016-11-16 02:42:22 +01:00
|
|
|
|
|
|
|
|
// Service is a directive for instantiating another Stack, including its name, arguments, etc.
|
|
|
|
|
type Service struct {
|
2016-11-16 18:29:44 +01:00
|
|
|
Node
|
2016-11-16 20:51:50 +01:00
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
Type Ref `json:"type,omitempty"` // an explicit type; if missing, the name is used.
|
Retain unrecognized service properties
During unmarshaling, the default behavior of the stock Golang JSON marshaler,
and consequently the YAML one we used which mimics its behavior, is to toss away
unrecognized properties. This isn't what we want for two reasons:
First, we want to issue errors/warnings on unrecognized fields to aid in diagnostics;
we will set aside some extensible section for 3rd parties to use. This is not
addressed in this change, however.
Second, and more pertinent, is that we need to retain unrecognized fields for certain
types like services, which are extensible by default.
Until golang/go#6213 is addressed -- imminent, it seems -- we will have to do a
somewhat hacky workaround to this problem. This change contains what I consider to
be the "least bad" in that we won't introduce a lot of performance overhead, and
just have to deal with the slight annoyance of the ast.Services node type containing
both Public/Private *and* PublicUntyped/PrivateUntyped fields alongside one another.
The marshaler dumps property bags into the *Untyped fields, and the parsetree analyzer
expands them out into a structured ast.Service type. Subsequent passes can then
ignore the *Untyped fields altogether.
Note that this would cause some marshaling funkiness if we ever wanted to remarshal
the mutated ASTs back into JSON/YAML. Since we don't do that right now, however, I've
not made any attempt to keep the two pairs in synch. Post-parsetree analyzer, we
literally just forget about the *Untyped guys.
2016-11-19 18:01:23 +01:00
|
|
|
Extra map[string]interface{} `json:"-"` // all of the "extra" properties, other than what is above.
|
|
|
|
|
|
2016-11-16 22:11:58 +01:00
|
|
|
Name Name `json:"-"` // a friendly name; decorated post-parsing, since it is contextual.
|
|
|
|
|
Public bool `json:"-"` // true if this service is publicly exposed; also decorated post-parsing.
|
2016-11-16 20:51:50 +01:00
|
|
|
|
2016-11-19 03:20:19 +01:00
|
|
|
BoundType *Stack `json:"-"` // services are bound to stacks during semantic analysis.
|
2016-11-16 02:42:22 +01:00
|
|
|
}
|
|
|
|
|
|
Retain unrecognized service properties
During unmarshaling, the default behavior of the stock Golang JSON marshaler,
and consequently the YAML one we used which mimics its behavior, is to toss away
unrecognized properties. This isn't what we want for two reasons:
First, we want to issue errors/warnings on unrecognized fields to aid in diagnostics;
we will set aside some extensible section for 3rd parties to use. This is not
addressed in this change, however.
Second, and more pertinent, is that we need to retain unrecognized fields for certain
types like services, which are extensible by default.
Until golang/go#6213 is addressed -- imminent, it seems -- we will have to do a
somewhat hacky workaround to this problem. This change contains what I consider to
be the "least bad" in that we won't introduce a lot of performance overhead, and
just have to deal with the slight annoyance of the ast.Services node type containing
both Public/Private *and* PublicUntyped/PrivateUntyped fields alongside one another.
The marshaler dumps property bags into the *Untyped fields, and the parsetree analyzer
expands them out into a structured ast.Service type. Subsequent passes can then
ignore the *Untyped fields altogether.
Note that this would cause some marshaling funkiness if we ever wanted to remarshal
the mutated ASTs back into JSON/YAML. Since we don't do that right now, however, I've
not made any attempt to keep the two pairs in synch. Post-parsetree analyzer, we
literally just forget about the *Untyped guys.
2016-11-19 18:01:23 +01:00
|
|
|
// UntypedServiceMap is a map of service names to untyped, bags of parsed properties for those services.
|
|
|
|
|
type UntypedServiceMap map[Name]PropertyBag
|
|
|
|
|
|
|
|
|
|
// PropertyBag is simply a map of string property names to untyped data values.
|
|
|
|
|
type PropertyBag map[string]interface{}
|
|
|
|
|
|
2016-11-16 18:29:44 +01:00
|
|
|
// TODO: several more core types still need to be mapped:
|
|
|
|
|
// - Schema
|
|
|
|
|
// - Identity: User, Role, Group
|
|
|
|
|
// - Configuration
|
|
|
|
|
// - Secret
|