2016-11-16 02:42:22 +01:00
|
|
|
// Copyright 2016 Marapongo, Inc. All rights reserved.
|
|
|
|
|
|
|
|
package compiler
|
|
|
|
|
|
|
|
import (
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
"strings"
|
|
|
|
|
2016-11-16 20:09:45 +01:00
|
|
|
"github.com/golang/glog"
|
2016-11-16 19:00:52 +01:00
|
|
|
|
2016-11-16 18:29:44 +01:00
|
|
|
"github.com/marapongo/mu/pkg/ast"
|
2016-11-17 17:52:54 +01:00
|
|
|
"github.com/marapongo/mu/pkg/compiler/core"
|
2016-11-16 02:42:22 +01:00
|
|
|
"github.com/marapongo/mu/pkg/diag"
|
|
|
|
"github.com/marapongo/mu/pkg/errors"
|
|
|
|
)
|
|
|
|
|
|
|
|
// PTAnalyzer knows how to walk and validate parse trees.
|
|
|
|
type PTAnalyzer interface {
|
2016-11-17 17:52:54 +01:00
|
|
|
core.Visitor
|
2016-11-16 02:42:22 +01:00
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// AnalyzeStack checks the validity of an entire Stack parse tree.
|
2016-11-23 16:26:45 +01:00
|
|
|
AnalyzeStack(stack *ast.Stack)
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// AnalyzeWorkspace checks the validity of an entire Workspace parse tree.
|
2016-11-23 16:26:45 +01:00
|
|
|
AnalyzeWorkspace(w *ast.Workspace)
|
2016-11-16 02:42:22 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
// NewPTAnalayzer allocates a new PTAnalyzer associated with the given Compiler.
|
|
|
|
func NewPTAnalyzer(c Compiler) PTAnalyzer {
|
|
|
|
return &ptAnalyzer{c: c}
|
|
|
|
}
|
|
|
|
|
|
|
|
type ptAnalyzer struct {
|
|
|
|
c Compiler
|
|
|
|
}
|
|
|
|
|
2016-12-09 22:12:57 +01:00
|
|
|
var _ PTAnalyzer = (*ptAnalyzer)(nil) // compile-time assert that binder implements PTAnalyzer.
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
|
2016-11-16 02:42:22 +01:00
|
|
|
func (a *ptAnalyzer) Diag() diag.Sink {
|
|
|
|
return a.c.Diag()
|
|
|
|
}
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
func (a *ptAnalyzer) AnalyzeStack(stack *ast.Stack) {
|
2016-11-16 20:09:45 +01:00
|
|
|
glog.Infof("Parsetree analyzing Mu Stack: %v", stack.Name)
|
|
|
|
if glog.V(2) {
|
|
|
|
defer func() {
|
|
|
|
glog.V(2).Infof("Parsetree analysis for Mu Stack %v completed w/ %v warnings and %v errors",
|
|
|
|
stack.Name, a.Diag().Warnings(), a.Diag().Errors())
|
|
|
|
}()
|
2016-11-16 02:42:22 +01:00
|
|
|
}
|
2016-11-16 20:45:41 +01:00
|
|
|
|
|
|
|
// Use an InOrderVisitor to walk the tree in-order; this handles determinism for us.
|
2016-11-17 17:52:54 +01:00
|
|
|
v := core.NewInOrderVisitor(a, nil)
|
2016-11-23 16:26:45 +01:00
|
|
|
v.VisitStack(stack)
|
2016-11-16 20:09:45 +01:00
|
|
|
}
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
func (a *ptAnalyzer) AnalyzeWorkspace(w *ast.Workspace) {
|
|
|
|
glog.Infof("Parsetree analyzing workspace file: %v", w.Doc.File)
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
if glog.V(2) {
|
|
|
|
defer func() {
|
|
|
|
glog.V(2).Infof("Parsetree analysis for workspace %v completed w/ %v warnings and %v errors",
|
2016-11-23 16:26:45 +01:00
|
|
|
w.Doc.File, a.Diag().Warnings(), a.Diag().Errors())
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
}()
|
2016-11-16 20:09:45 +01:00
|
|
|
}
|
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// Use an InOrderVisitor to walk the tree in-order; this handles determinism for us.
|
|
|
|
v := core.NewInOrderVisitor(a, nil)
|
2016-11-23 16:26:45 +01:00
|
|
|
v.VisitWorkspace(w)
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
}
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
func (a *ptAnalyzer) VisitWorkspace(w *ast.Workspace) {
|
2016-11-16 20:09:45 +01:00
|
|
|
}
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
func (a *ptAnalyzer) VisitCluster(name string, cluster *ast.Cluster) {
|
Support Workspaces
This change adds support for Workspaces, a convenient way of sharing settings
among many Stacks, like default cluster targets, configuration settings, and the
like, which are not meant to be distributed as part of the Stack itself.
The following things are included in this checkin:
* At workspace initialization time, detect and parse the .mu/workspace.yaml
file. This is pretty rudimentary right now and contains just the default
cluster targets. The results are stored in a new ast.Workspace type.
* Rename "target" to "cluster". This impacts many things, including ast.Target
being changed to ast.Cluster, and all related fields, the command line --target
being changed to --cluster, various internal helper functions, and so on. This
helps to reinforce the desired mental model.
* Eliminate the ast.Metadata type. Instead, the metadata moves directly onto
the Stack. This reflects the decision to make Stacks "the thing" that is
distributed, versioned, and is the granularity of dependency.
* During cluster targeting, add the workspace settings into the probing logic.
We still search in the same order: CLI > Stack > Workspace.
2016-11-22 19:41:07 +01:00
|
|
|
// Decorate the AST with contextual information so subsequent passes can operate context-free.
|
|
|
|
cluster.Name = name
|
2016-11-16 20:09:45 +01:00
|
|
|
}
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
func (a *ptAnalyzer) VisitDependency(parent *ast.Workspace, ref ast.Ref, dep *ast.Dependency) {
|
2016-11-16 20:09:45 +01:00
|
|
|
}
|
|
|
|
|
2016-11-23 16:26:45 +01:00
|
|
|
func (a *ptAnalyzer) VisitStack(stack *ast.Stack) {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
}
|
|
|
|
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
func (a *ptAnalyzer) VisitSchemas(parent *ast.Stack, schemas *ast.Schemas) {
|
|
|
|
}
|
|
|
|
|
|
|
|
func (a *ptAnalyzer) VisitSchema(pstack *ast.Stack, parent *ast.Schemas, name ast.Name, public bool,
|
|
|
|
schema *ast.Schema) {
|
|
|
|
// Decorate the AST with contextual information.
|
|
|
|
schema.Name = name
|
|
|
|
schema.Public = public
|
Custom types, round 2
This checkin continues progress on marapongo/mu#9. It's still not
complete, however we're getting there. In particular, this includes:
* Rename of ComplexLiteral to SchemaLiteral, as it is used exclusively
for schematized types. Also includes a set of changes associated
with this, like deep value conversion to `map[string]interface{}`.
* Binding of schema types included within a Stack. This allows names in
type references to be bound to those schema types during typechecking.
This also includes binding schema properties, reusing all the existing
property binding logic for stacks. In this way, properties between
stacks and custom schema types are one and the same, which is nice.
* Enforcement for custom schema constraints; this includes Pattern,
MaxLength, MinLength, Maximum, and Minimum, as per the JSON Schema
specification.
2016-12-07 05:51:05 +01:00
|
|
|
|
|
|
|
// If the schema has a base type listed, parse it to the best of our ability.
|
|
|
|
if schema.Base != "" {
|
|
|
|
schema.BoundBase = a.parseType(schema.Base)
|
|
|
|
}
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
}
|
|
|
|
|
Custom types, round 2
This checkin continues progress on marapongo/mu#9. It's still not
complete, however we're getting there. In particular, this includes:
* Rename of ComplexLiteral to SchemaLiteral, as it is used exclusively
for schematized types. Also includes a set of changes associated
with this, like deep value conversion to `map[string]interface{}`.
* Binding of schema types included within a Stack. This allows names in
type references to be bound to those schema types during typechecking.
This also includes binding schema properties, reusing all the existing
property binding logic for stacks. In this way, properties between
stacks and custom schema types are one and the same, which is nice.
* Enforcement for custom schema constraints; this includes Pattern,
MaxLength, MinLength, Maximum, and Minimum, as per the JSON Schema
specification.
2016-12-07 05:51:05 +01:00
|
|
|
func (a *ptAnalyzer) VisitProperty(parent *ast.Stack, schema *ast.Schema, name string, prop *ast.Property) {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// Decorate the AST with contextual information so subsequent passes can operate context-free.
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
prop.Name = name
|
|
|
|
|
|
|
|
// Parse the property type to the best of our ability at this phase in the compiler.
|
Custom types, round 2
This checkin continues progress on marapongo/mu#9. It's still not
complete, however we're getting there. In particular, this includes:
* Rename of ComplexLiteral to SchemaLiteral, as it is used exclusively
for schematized types. Also includes a set of changes associated
with this, like deep value conversion to `map[string]interface{}`.
* Binding of schema types included within a Stack. This allows names in
type references to be bound to those schema types during typechecking.
This also includes binding schema properties, reusing all the existing
property binding logic for stacks. In this way, properties between
stacks and custom schema types are one and the same, which is nice.
* Enforcement for custom schema constraints; this includes Pattern,
MaxLength, MinLength, Maximum, and Minimum, as per the JSON Schema
specification.
2016-12-07 05:51:05 +01:00
|
|
|
prop.BoundType = a.parseType(prop.Type)
|
|
|
|
}
|
|
|
|
|
|
|
|
func (a *ptAnalyzer) VisitServices(parent *ast.Stack, svcs *ast.Services) {
|
|
|
|
// We need to expand the UntypedServiceMaps into strongly typed ServiceMaps. As part of this, we also decorate the
|
|
|
|
// AST with extra contextual information so subsequent passes can operate context-free.
|
|
|
|
// TODO[marapongo/mu#4]: once we harden the marshalers, we should be able to largely eliminate this.
|
|
|
|
svcs.Public = make(ast.ServiceMap)
|
|
|
|
for _, name := range ast.StableUntypedServices(svcs.PublicUntyped) {
|
|
|
|
svcs.Public[name] = a.untypedServiceToTyped(parent, name, true, svcs.PublicUntyped[name])
|
|
|
|
}
|
|
|
|
svcs.Private = make(ast.ServiceMap)
|
|
|
|
for _, name := range ast.StableUntypedServices(svcs.PrivateUntyped) {
|
|
|
|
svcs.Private[name] = a.untypedServiceToTyped(parent, name, false, svcs.PrivateUntyped[name])
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (a *ptAnalyzer) untypedServiceToTyped(parent *ast.Stack, name ast.Name, public bool,
|
|
|
|
bag map[string]interface{}) *ast.Service {
|
|
|
|
var typ ast.Name
|
|
|
|
t, has := bag["type"]
|
|
|
|
if has {
|
|
|
|
// If the bag contains a type, ensure that it is a string.
|
|
|
|
ts, ok := t.(string)
|
|
|
|
if ok {
|
|
|
|
typ = ast.Name(ts)
|
|
|
|
} else {
|
|
|
|
a.Diag().Errorf(errors.ErrorIllegalMufileSyntax.At(parent), "service type must be a string")
|
|
|
|
}
|
|
|
|
|
|
|
|
// Delete the type property so it's not considered semantically meaningful for the target.
|
|
|
|
delete(bag, "type")
|
|
|
|
}
|
|
|
|
|
|
|
|
return &ast.Service{
|
|
|
|
Name: name,
|
|
|
|
Type: ast.Ref(typ),
|
|
|
|
Public: public,
|
|
|
|
Properties: bag,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (a *ptAnalyzer) VisitService(pstack *ast.Stack, parent *ast.Services, name ast.Name, public bool,
|
|
|
|
svc *ast.Service) {
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
}
|
|
|
|
|
Custom types, round 2
This checkin continues progress on marapongo/mu#9. It's still not
complete, however we're getting there. In particular, this includes:
* Rename of ComplexLiteral to SchemaLiteral, as it is used exclusively
for schematized types. Also includes a set of changes associated
with this, like deep value conversion to `map[string]interface{}`.
* Binding of schema types included within a Stack. This allows names in
type references to be bound to those schema types during typechecking.
This also includes binding schema properties, reusing all the existing
property binding logic for stacks. In this way, properties between
stacks and custom schema types are one and the same, which is nice.
* Enforcement for custom schema constraints; this includes Pattern,
MaxLength, MinLength, Maximum, and Minimum, as per the JSON Schema
specification.
2016-12-07 05:51:05 +01:00
|
|
|
// parseType produces an ast.Type. This will not have been bound yet, so for example, we won't know whether
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
// an arbitrary non-primitive reference name references a stack or a schema, however at least this is a start.
|
Custom types, round 2
This checkin continues progress on marapongo/mu#9. It's still not
complete, however we're getting there. In particular, this includes:
* Rename of ComplexLiteral to SchemaLiteral, as it is used exclusively
for schematized types. Also includes a set of changes associated
with this, like deep value conversion to `map[string]interface{}`.
* Binding of schema types included within a Stack. This allows names in
type references to be bound to those schema types during typechecking.
This also includes binding schema properties, reusing all the existing
property binding logic for stacks. In this way, properties between
stacks and custom schema types are one and the same, which is nice.
* Enforcement for custom schema constraints; this includes Pattern,
MaxLength, MinLength, Maximum, and Minimum, as per the JSON Schema
specification.
2016-12-07 05:51:05 +01:00
|
|
|
func (a *ptAnalyzer) parseType(ref ast.Ref) *ast.Type {
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
refs := string(ref)
|
|
|
|
|
|
|
|
mix := strings.Index(refs, ast.TypeDecorsMapPrefix)
|
|
|
|
if mix == 0 {
|
|
|
|
// If we have a map, find the separator, and then parse the key and value parts.
|
|
|
|
rest := refs[mix+len(ast.TypeDecorsMapPrefix):]
|
|
|
|
if sep := strings.Index(rest, ast.TypeDecorsMapSeparator); sep != -1 {
|
|
|
|
keyn := ast.Ref(rest[:sep])
|
2016-12-07 22:24:05 +01:00
|
|
|
valn := ast.Ref(rest[sep+len(ast.TypeDecorsMapSeparator):])
|
|
|
|
if keyn != "" && valn != "" {
|
|
|
|
keyt := a.parseType(keyn)
|
|
|
|
valt := a.parseType(valn)
|
|
|
|
if keyt != nil && valt != nil {
|
|
|
|
return ast.NewMapType(keyt, valt)
|
|
|
|
}
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
}
|
|
|
|
}
|
2016-12-07 22:24:05 +01:00
|
|
|
|
|
|
|
a.Diag().Errorf(errors.ErrorIllegalMapLikeSyntax, refs)
|
|
|
|
} else if aix := strings.LastIndex(refs, ast.TypeDecorsArraySuffix); aix != -1 {
|
|
|
|
if aix == len(refs)-len(ast.TypeDecorsArraySuffix) {
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
// If we have an array, peel off the front and keep going.
|
2016-12-07 22:24:05 +01:00
|
|
|
rest := refs[:aix]
|
|
|
|
if rest != "" {
|
|
|
|
if elem := a.parseType(ast.Ref(rest)); elem != nil {
|
|
|
|
return ast.NewArrayType(elem)
|
|
|
|
}
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
}
|
|
|
|
}
|
2016-12-07 22:24:05 +01:00
|
|
|
|
|
|
|
// The array part was in the wrong position. Issue an error. Maybe they did T[] instead of []T?
|
|
|
|
a.Diag().Errorf(errors.ErrorIllegalArrayLikeSyntax, refs)
|
Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler. In particular, we now have an ast.Type, and have begun
using its use where appropriate. An ast.Type is a union representing
precisely one of the possible sources of types in the system:
* Primitive type: any, bool, number, string, or service.
* Stack type: a resolved reference to an actual concrete stack.
* Schema type: a resolved reference to an actual concrete schema.
* Unresolved reference: a textual reference that hasn't yet been
resolved to a concrete artifact.
* Uninstantiated reference: a reference that has been resolved to
an uninstantiated stack, but hasn't been bound to a concrete
result yet. Right now, this can point to a stack, however
eventually we would imagine this supporting inter-stack schema
references also.
* Decorated type: either an array or a map; in the array case, there
is a single inner element type; in the map case, there are two,
the keys and values; in all cases, the type recurses to any of the
possibilities listed here.
All of the relevant AST nodes have been overhauled accordingly.
In addition to this, we now have an ast.Schema type. It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.
This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 23:49:47 +01:00
|
|
|
} else if mix != -1 {
|
|
|
|
// The map part was in the wrong position. Issue an error.
|
|
|
|
a.Diag().Errorf(errors.ErrorIllegalMapLikeSyntax, refs)
|
|
|
|
} else {
|
|
|
|
// Otherwise, there are no decorators. Parse the result as either a primitive type or unresolved name.
|
|
|
|
switch ast.PrimitiveType(refs) {
|
|
|
|
case ast.PrimitiveTypeAny:
|
|
|
|
return ast.NewAnyType()
|
|
|
|
case ast.PrimitiveTypeString:
|
|
|
|
return ast.NewStringType()
|
|
|
|
case ast.PrimitiveTypeNumber:
|
|
|
|
return ast.NewNumberType()
|
|
|
|
case ast.PrimitiveTypeBool:
|
|
|
|
return ast.NewBoolType()
|
|
|
|
case ast.PrimitiveTypeService:
|
|
|
|
return ast.NewServiceType()
|
|
|
|
}
|
|
|
|
|
|
|
|
// If we didn't recognize anything thus far, it's a simple name. We don't yet know what it references --
|
|
|
|
// it could be a stack, schema, or even a completely bogus, missing name -- so just store it as it is.
|
|
|
|
if _, err := ref.Parse(); err != nil {
|
|
|
|
a.Diag().Errorf(errors.ErrorIllegalNameLikeSyntax, refs, err)
|
|
|
|
} else {
|
|
|
|
return ast.NewUnresolvedRefType(&ref)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return nil
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
}
|