2016-11-15 20:30:34 +01:00
|
|
|
// Copyright 2016 Marapongo, Inc. All rights reserved.
|
|
|
|
|
|
|
|
package compiler
|
|
|
|
|
|
|
|
import (
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
"os"
|
|
|
|
|
2016-11-15 20:30:34 +01:00
|
|
|
"github.com/golang/glog"
|
|
|
|
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
"github.com/marapongo/mu/pkg/ast"
|
2016-11-17 17:52:54 +01:00
|
|
|
"github.com/marapongo/mu/pkg/compiler/core"
|
2016-11-15 20:30:34 +01:00
|
|
|
"github.com/marapongo/mu/pkg/diag"
|
|
|
|
"github.com/marapongo/mu/pkg/errors"
|
2016-11-23 16:26:45 +01:00
|
|
|
"github.com/marapongo/mu/pkg/util"
|
2016-11-15 20:30:34 +01:00
|
|
|
"github.com/marapongo/mu/pkg/workspace"
|
|
|
|
)
|
|
|
|
|
|
|
|
// Compiler provides an interface into the many phases of the Mu compilation process.
|
|
|
|
type Compiler interface {
|
2016-11-17 17:52:54 +01:00
|
|
|
core.Phase
|
2016-11-16 18:29:44 +01:00
|
|
|
|
2016-11-16 02:42:22 +01:00
|
|
|
// Context returns the current compiler context.
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
Context() *core.Context
|
2016-11-16 02:42:22 +01:00
|
|
|
|
2016-11-15 20:30:34 +01:00
|
|
|
// Build detects and compiles inputs from the given location, storing build artifacts in the given destination.
|
|
|
|
Build(inp string, outp string)
|
2016-11-18 00:13:36 +01:00
|
|
|
// BuildFile uses the given Mufile directly, and stores build artifacts in the given destination.
|
|
|
|
BuildFile(mufile []byte, ext string, outp string)
|
2016-11-15 20:30:34 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
// compiler is the canonical implementation of the Mu compiler.
|
|
|
|
type compiler struct {
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
ctx *core.Context
|
2016-11-15 20:30:34 +01:00
|
|
|
opts Options
|
2016-11-23 16:26:45 +01:00
|
|
|
deps map[ast.Ref]*ast.Stack // a cache of mapping names to loaded dependencies.
|
2016-11-15 20:30:34 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
// NewCompiler creates a new instance of the Mu compiler, with the given initialization settings.
|
|
|
|
func NewCompiler(opts Options) Compiler {
|
2016-11-16 02:42:22 +01:00
|
|
|
return &compiler{
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
ctx: &core.Context{},
|
2016-11-16 02:42:22 +01:00
|
|
|
opts: opts,
|
2016-11-23 16:26:45 +01:00
|
|
|
deps: make(map[ast.Ref]*ast.Stack),
|
2016-11-16 02:42:22 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
func (c *compiler) Context() *core.Context {
|
2016-11-16 02:42:22 +01:00
|
|
|
return c.ctx
|
2016-11-15 20:30:34 +01:00
|
|
|
}
|
|
|
|
|
2016-11-16 01:30:10 +01:00
|
|
|
func (c *compiler) Diag() diag.Sink {
|
|
|
|
return c.opts.Diag
|
2016-11-15 20:30:34 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
func (c *compiler) Build(inp string, outp string) {
|
2016-11-16 01:30:10 +01:00
|
|
|
glog.Infof("Building target '%v' (out='%v')", inp, outp)
|
2016-11-17 22:08:20 +01:00
|
|
|
|
|
|
|
// First find the root of the current package based on the location of its Mufile.
|
2016-11-21 18:23:39 +01:00
|
|
|
w, err := workspace.New(inp, c.Diag())
|
|
|
|
if err != nil {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
c.Diag().Errorf(errors.ErrorIO.WithFile(inp), err)
|
2016-11-21 18:23:39 +01:00
|
|
|
return
|
|
|
|
}
|
|
|
|
mufile, err := w.DetectMufile()
|
|
|
|
if err != nil {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
c.Diag().Errorf(errors.ErrorIO.WithFile(inp), err)
|
2016-11-21 18:23:39 +01:00
|
|
|
return
|
|
|
|
}
|
2016-11-17 22:08:20 +01:00
|
|
|
if mufile == "" {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
c.Diag().Errorf(errors.ErrorMissingMufile, inp)
|
2016-11-17 22:08:20 +01:00
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
// Read in the contents of the document and make it available to subsequent stages.
|
|
|
|
doc, err := diag.ReadDocument(mufile)
|
|
|
|
if err != nil {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
c.Diag().Errorf(errors.ErrorCouldNotReadMufile.WithFile(mufile), err)
|
2016-11-17 22:08:20 +01:00
|
|
|
return
|
|
|
|
}
|
|
|
|
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
c.buildDocument(w, doc, outp)
|
2016-11-17 22:08:20 +01:00
|
|
|
}
|
|
|
|
|
2016-11-18 00:13:36 +01:00
|
|
|
func (c *compiler) BuildFile(mufile []byte, ext string, outp string) {
|
|
|
|
glog.Infof("Building in-memory %v file (bytes=%v out='%v')", ext, len(mufile), outp)
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
|
|
|
|
// Default to the current working directory for the workspace.
|
|
|
|
dir, err := os.Getwd()
|
|
|
|
if err != nil {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
c.Diag().Errorf(errors.ErrorIO, err)
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
return
|
|
|
|
}
|
|
|
|
w, err := workspace.New(dir, c.Diag())
|
|
|
|
if err != nil {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
c.Diag().Errorf(errors.ErrorIO, err)
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
doc := &diag.Document{File: workspace.Mufile + ext, Body: mufile}
|
|
|
|
c.buildDocument(w, doc, outp)
|
2016-11-17 22:08:20 +01:00
|
|
|
}
|
|
|
|
|
Implement dependency resolution
This change includes logic to resolve dependencies declared by stacks. The design
is described in https://github.com/marapongo/mu/blob/master/docs/deps.md.
In summary, each stack may declare dependencies, which are name/semver pairs. A
new structure has been introduced, ast.Ref, to distinguish between ast.Names and
dependency names. An ast.Ref includes a protocol, base part, and a name part (the
latter being an ast.Name); for example, in "https://hub.mu.com/mu/container/",
"https://" is the protocol, "hub.mu.com/" is the base, and "mu/container" is the
name. This is used to resolve URL-like names to package manager-like artifacts.
The dependency resolution phase happens after parsing, but before semantic analysis.
This is because dependencies are "source-like" in that we must load and parse all
dependency metadata files. We stick the full transitive closure of dependencies
into a map attached to the compiler to avoid loading dependencies multiple times.
Note that, although dependencies prohibit cycles, this forms a DAG, meaning multiple
inbound edges to a single stack may come from multiple places.
From there, we rely on ordinary visitation to deal with dependencies further.
This includes inserting symbol entries into the symbol table, mapping names to the
loaded stacks, during the first phase of binding so that they may be found
subsequently when typechecking during the second phase and beyond.
2016-11-21 20:19:25 +01:00
|
|
|
func (c *compiler) buildDocument(w workspace.W, doc *diag.Document, outp string) {
|
2016-11-17 22:08:20 +01:00
|
|
|
glog.Infof("Building doc '%v' (bytes=%v out='%v')", doc.File, len(doc.Body), outp)
|
2016-11-16 01:30:10 +01:00
|
|
|
if glog.V(2) {
|
|
|
|
defer func() {
|
2016-11-17 22:08:20 +01:00
|
|
|
glog.V(2).Infof("Building doc '%v' completed w/ %v warnings and %v errors",
|
|
|
|
doc.File, c.Diag().Warnings(), c.Diag().Errors())
|
2016-11-16 01:30:10 +01:00
|
|
|
}()
|
|
|
|
}
|
2016-11-15 20:30:34 +01:00
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// Perform the front-end phases of the compiler.
|
2016-11-23 16:26:45 +01:00
|
|
|
stack := c.buildDocumentFE(w, doc)
|
|
|
|
if !c.Diag().Success() {
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
return
|
|
|
|
}
|
2016-11-23 16:26:45 +01:00
|
|
|
util.Assert(stack != nil)
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
// Next, perform the semantic analysis phases of the compiler.
|
2016-11-23 16:26:45 +01:00
|
|
|
c.buildDocumentSema(w, stack)
|
|
|
|
if !c.Diag().Success() {
|
Implement dependency versions
This change implements dependency versions, including semantic analysis, per the
checkin https://github.com/marapongo/mu/commit/83030685c3b8a3dbe96bd10ab055f029667a96b0.
There's quite a bit in here but at a top-level this parses and validates dependency
references of the form
[[proto://]base.url]namespace/.../name[@version]
and verifies that the components are correct, as well as binding them to symbols.
These references can appear in two places at the moment:
* Service types.
* Cluster dependencies.
As part of this change, a number of supporting changes have been made:
* Parse Workspaces using a full-blown parser, parser analysis, and semantic analysis.
This allows us to share logic around the validation of common AST types. This also
moves some of the logic around loading workspace.yaml files back to the parser, where
it can be unified with the way we load Mu.yaml files.
* New ast.Version and ast.VersionSpec types. The former represents a precise version
-- either a specific semantic version or a short or long Git SHA hash -- and the
latter represents a range -- either a Version, "latest", or a semantic range.
* New ast.Ref and ast.RefParts types. The former is an unparsed string that is
thought to contain a Ref, while the latter is a validated Ref that has been parsed
into its components (Proto, Base, Name, and Version).
* Added some type assertions to ensure certain structs implement certain interfaces,
to speed up finding errors. (And remove the coercions that zero-fill vtbl slots.)
* Be consistent about prefixing error types with Error or Warning.
* Organize the core compiler driver's logic into three methods, FE, sema, and BE.
* A bunch of tests for some of the above ... more to come in an upcoming change.
2016-11-23 01:58:23 +01:00
|
|
|
return
|
|
|
|
}
|
|
|
|
|
|
|
|
// Finally, perform the back-end phases of the compiler.
|
2016-11-23 16:26:45 +01:00
|
|
|
c.buildDocumentBE(w, stack)
|
Add cloud target and architecture detection
This change implements most of the cloud target and architecture detection
logic, along with associated verification and a bunch of new error messages.
There are two settings for picking a cloud destination:
* Architecture: this specifies the combination of cloud (e.g., AWS, GCP, etc)
plus scheduler (e.g., none, Swarm, ECS, etc).
* Target: a named, preconfigured entity that includes both an Architecture and
an assortment of extra default configuration options.
The general idea here is that you can preconfigure a set of Targets for
named environments like "prod", "stage", etc. Those can either exist in a
single Mufile, or the Mucluster file if they are shared amongst multiple
Mufiles. This can be specified at the command line as such:
$ mu build --target=stage
Furthermore, a given environment may be annointed the default, so that
$ mu build
selects that environment without needing to say so explicitly.
It is also possible to specify an architecture at the command line for
scenarios where you aren't intending to target an existing named environment.
This is good for "anonymous" testing scenarios or even just running locally:
$ mu build --arch=aws
$ mu build --arch=aws:ecs
$ mu build --arch=local:kubernetes
$ .. and so on ..
This change does little more than plumb these settings around, verify them,
etc., however it sets us up to actually start dispating to the right backend.
2016-11-17 19:30:37 +01:00
|
|
|
}
|