maxmustermann/pulumi
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
pulumi/pkg/pack/package.go

57 lines
2.6 KiB
Go
Raw Normal View History

Reclassify Lumi under the Apache 2.0 license This is part of pulumi/lumi#147.
2017-05-18 23:51:52 +02:00
// Licensed to Pulumi Corporation ("Pulumi") under one or more
// contributor license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// Pulumi licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Create MuPack and MuIL packages in our Go toolset This change carries over all of the metadata shapes in the MuPack and MuIL file formats to our Go toolset. This includes creating a proper discriminated AST type tree along with correct annotations so that the metadata will serialize and deserialize correctly.
2017-01-13 23:32:10 +01:00
Rename Coconut to Lumi This is part of pulumi/coconut#147. After it has landed, I will rename the repo on GitHub.
2017-05-18 20:38:28 +02:00
// Package pack contains the core LumiPack metadata types.
Create MuPack and MuIL packages in our Go toolset This change carries over all of the metadata shapes in the MuPack and MuIL file formats to our Go toolset. This includes creating a proper discriminated AST type tree along with correct annotations so that the metadata will serialize and deserialize correctly.
2017-01-13 23:32:10 +01:00
package pack
import (
Rename Coconut to Lumi This is part of pulumi/coconut#147. After it has landed, I will rename the repo on GitHub.
2017-05-18 20:38:28 +02:00
"github.com/pulumi/lumi/pkg/compiler/ast"
"github.com/pulumi/lumi/pkg/diag"
"github.com/pulumi/lumi/pkg/tokens"
Create MuPack and MuIL packages in our Go toolset This change carries over all of the metadata shapes in the MuPack and MuIL file formats to our Go toolset. This includes creating a proper discriminated AST type tree along with correct annotations so that the metadata will serialize and deserialize correctly.
2017-01-13 23:32:10 +01:00
)
Begin overhauling semantic phases This change further merges the new AST and MuPack/MuIL formats and abstractions into the core of the compiler. A good amount of the old code is gone now; I decided against ripping it all out in one fell swoop so that I can methodically check that we are preserving all relevant decisions and/or functionality we had in the old model. The changes are too numerous to outline in this commit message, however, here are the noteworthy ones: * Split up the notion of symbols and tokens, resulting in: - pkg/symbols for true compiler symbols (bound nodes) - pkg/tokens for name-based tokens, identifiers, constants * Several packages move underneath pkg/compiler: - pkg/ast becomes pkg/compiler/ast - pkg/errors becomes pkg/compiler/errors - pkg/symbols becomes pkg/compiler/symbols * pkg/ast/... becomes pkg/compiler/legacy/ast/... * pkg/pack/ast becomes pkg/compiler/ast. * pkg/options goes away, merged back into pkg/compiler. * All binding functionality moves underneath a dedicated package, pkg/compiler/binder. The legacy.go file contains cruft that will eventually go away, while the other files represent a halfway point between new and old, but are expected to stay roughly in the current shape. * All parsing functionality is moved underneath a new pkg/compiler/metadata namespace, and we adopt new terminology "metadata reading" since real parsing happens in the MetaMu compilers. Hence, Parser has become metadata.Reader. * In general phases of the compiler no longer share access to the actual compiler.Compiler object. Instead, shared state is moved to the core.Context object underneath pkg/compiler/core. * Dependency resolution during binding has been rewritten to the new model, including stashing bound package symbols in the context object, and detecting import cycles. * Compiler construction does not take a workspace object. Instead, creation of a workspace is entirely hidden inside of the compiler's constructor logic. * There are three Compile* functions on the Compiler interface, to support different styles of invoking compilation: Compile() auto- detects a Mu package, based on the workspace; CompilePath(string) loads the target as a Mu package and compiles it, regardless of the workspace settings; and, CompilePackage(*pack.Package) will compile a pre-loaded package AST, again regardless of workspace. * Delete the _fe, _sema, and parsetree phases. They are no longer relevant and the functionality is largely subsumed by the above. ...and so very much more. I'm surprised I ever got this to compile again!
2017-01-18 21:18:37 +01:00
// Package is a top-level package definition.
type Package struct {
Replace tree-walk-style type binding with token map lookups Now that we have introduced a full blown token map -- new as of just a few changes ago -- we can start using it for all of our symbol binding. This also addresses some order-dependent issues, like intra-module references looking up symbols that have been registered in the token map but not necessarily stored in the relevant parent symbols just yet. Plus, frankly, it's much simpler and uses a hashmap lookup instead of a fairly complex recursive tree walk. I've kept the tree walk case, however, to improve diagnostics upon failure. This allows us to tell developers, for example, that the reason a binding failed was due to a missing package.
2017-01-27 01:49:38 +01:00
Name tokens.PackageName `json:"name"` // a required fully qualified name.
Begin overhauling semantic phases This change further merges the new AST and MuPack/MuIL formats and abstractions into the core of the compiler. A good amount of the old code is gone now; I decided against ripping it all out in one fell swoop so that I can methodically check that we are preserving all relevant decisions and/or functionality we had in the old model. The changes are too numerous to outline in this commit message, however, here are the noteworthy ones: * Split up the notion of symbols and tokens, resulting in: - pkg/symbols for true compiler symbols (bound nodes) - pkg/tokens for name-based tokens, identifiers, constants * Several packages move underneath pkg/compiler: - pkg/ast becomes pkg/compiler/ast - pkg/errors becomes pkg/compiler/errors - pkg/symbols becomes pkg/compiler/symbols * pkg/ast/... becomes pkg/compiler/legacy/ast/... * pkg/pack/ast becomes pkg/compiler/ast. * pkg/options goes away, merged back into pkg/compiler. * All binding functionality moves underneath a dedicated package, pkg/compiler/binder. The legacy.go file contains cruft that will eventually go away, while the other files represent a halfway point between new and old, but are expected to stay roughly in the current shape. * All parsing functionality is moved underneath a new pkg/compiler/metadata namespace, and we adopt new terminology "metadata reading" since real parsing happens in the MetaMu compilers. Hence, Parser has become metadata.Reader. * In general phases of the compiler no longer share access to the actual compiler.Compiler object. Instead, shared state is moved to the core.Context object underneath pkg/compiler/core. * Dependency resolution during binding has been rewritten to the new model, including stashing bound package symbols in the context object, and detecting import cycles. * Compiler construction does not take a workspace object. Instead, creation of a workspace is entirely hidden inside of the compiler's constructor logic. * There are three Compile* functions on the Compiler interface, to support different styles of invoking compilation: Compile() auto- detects a Mu package, based on the workspace; CompilePath(string) loads the target as a Mu package and compiles it, regardless of the workspace settings; and, CompilePackage(*pack.Package) will compile a pre-loaded package AST, again regardless of workspace. * Delete the _fe, _sema, and parsetree phases. They are no longer relevant and the functionality is largely subsumed by the above. ...and so very much more. I'm surprised I ever got this to compile again!
2017-01-18 21:18:37 +01:00
Print descriptions as header comments
2017-01-16 23:45:32 +01:00
Description *string `json:"description,omitempty"` // an optional informational description.
Author *string `json:"author,omitempty"` // an optional author.
Website *string `json:"website,omitempty"` // an optional website for additional info.
License *string `json:"license,omitempty"` // an optional license governing this package's usage.
Begin overhauling semantic phases This change further merges the new AST and MuPack/MuIL formats and abstractions into the core of the compiler. A good amount of the old code is gone now; I decided against ripping it all out in one fell swoop so that I can methodically check that we are preserving all relevant decisions and/or functionality we had in the old model. The changes are too numerous to outline in this commit message, however, here are the noteworthy ones: * Split up the notion of symbols and tokens, resulting in: - pkg/symbols for true compiler symbols (bound nodes) - pkg/tokens for name-based tokens, identifiers, constants * Several packages move underneath pkg/compiler: - pkg/ast becomes pkg/compiler/ast - pkg/errors becomes pkg/compiler/errors - pkg/symbols becomes pkg/compiler/symbols * pkg/ast/... becomes pkg/compiler/legacy/ast/... * pkg/pack/ast becomes pkg/compiler/ast. * pkg/options goes away, merged back into pkg/compiler. * All binding functionality moves underneath a dedicated package, pkg/compiler/binder. The legacy.go file contains cruft that will eventually go away, while the other files represent a halfway point between new and old, but are expected to stay roughly in the current shape. * All parsing functionality is moved underneath a new pkg/compiler/metadata namespace, and we adopt new terminology "metadata reading" since real parsing happens in the MetaMu compilers. Hence, Parser has become metadata.Reader. * In general phases of the compiler no longer share access to the actual compiler.Compiler object. Instead, shared state is moved to the core.Context object underneath pkg/compiler/core. * Dependency resolution during binding has been rewritten to the new model, including stashing bound package symbols in the context object, and detecting import cycles. * Compiler construction does not take a workspace object. Instead, creation of a workspace is entirely hidden inside of the compiler's constructor logic. * There are three Compile* functions on the Compiler interface, to support different styles of invoking compilation: Compile() auto- detects a Mu package, based on the workspace; CompilePath(string) loads the target as a Mu package and compiles it, regardless of the workspace settings; and, CompilePackage(*pack.Package) will compile a pre-loaded package AST, again regardless of workspace. * Delete the _fe, _sema, and parsetree phases. They are no longer relevant and the functionality is largely subsumed by the above. ...and so very much more. I'm surprised I ever got this to compile again!
2017-01-18 21:18:37 +01:00
Add the ability to specify analyzers This change adds the ability to specify analyzers in two ways: 1) By listing them in the project file, for example: analyzers: - acmecorp/security - acmecorp/gitflow 2) By explicitly listing them on the CLI, as a "one off": $ coco deploy <env> \ --analyzer=acmecorp/security \ --analyzer=acmecorp/gitflow This closes out pulumi/coconut#119.
2017-03-11 19:07:34 +01:00
Analyzers *Analyzers `json:"analyzers,omitempty"` // any analyzers enabled for this project.
Generalize default modules This change generalizes the support for default modules (added as part of marapongo/mu#57), to use an "alias" table. This is just a table of alternative module names to the real defining module name. The default module is now just represented as a mapping from the special default module token, ".default", to the actual defining module. This generalization was necessary to support Node.js-style alternative names for modules, like "lib" mapping to "lib/index".
2017-02-15 21:53:36 +01:00
Dependencies *Dependencies `json:"dependencies,omitempty"` // all of the package dependencies.
Modules *ast.Modules `json:"modules,omitempty"` // a collection of top-level modules.
Aliases *ModuleAliases `json:"aliases,omitempty"` // an optional mapping of aliased module names.
Begin overhauling semantic phases This change further merges the new AST and MuPack/MuIL formats and abstractions into the core of the compiler. A good amount of the old code is gone now; I decided against ripping it all out in one fell swoop so that I can methodically check that we are preserving all relevant decisions and/or functionality we had in the old model. The changes are too numerous to outline in this commit message, however, here are the noteworthy ones: * Split up the notion of symbols and tokens, resulting in: - pkg/symbols for true compiler symbols (bound nodes) - pkg/tokens for name-based tokens, identifiers, constants * Several packages move underneath pkg/compiler: - pkg/ast becomes pkg/compiler/ast - pkg/errors becomes pkg/compiler/errors - pkg/symbols becomes pkg/compiler/symbols * pkg/ast/... becomes pkg/compiler/legacy/ast/... * pkg/pack/ast becomes pkg/compiler/ast. * pkg/options goes away, merged back into pkg/compiler. * All binding functionality moves underneath a dedicated package, pkg/compiler/binder. The legacy.go file contains cruft that will eventually go away, while the other files represent a halfway point between new and old, but are expected to stay roughly in the current shape. * All parsing functionality is moved underneath a new pkg/compiler/metadata namespace, and we adopt new terminology "metadata reading" since real parsing happens in the MetaMu compilers. Hence, Parser has become metadata.Reader. * In general phases of the compiler no longer share access to the actual compiler.Compiler object. Instead, shared state is moved to the core.Context object underneath pkg/compiler/core. * Dependency resolution during binding has been rewritten to the new model, including stashing bound package symbols in the context object, and detecting import cycles. * Compiler construction does not take a workspace object. Instead, creation of a workspace is entirely hidden inside of the compiler's constructor logic. * There are three Compile* functions on the Compiler interface, to support different styles of invoking compilation: Compile() auto- detects a Mu package, based on the workspace; CompilePath(string) loads the target as a Mu package and compiles it, regardless of the workspace settings; and, CompilePackage(*pack.Package) will compile a pre-loaded package AST, again regardless of workspace. * Delete the _fe, _sema, and parsetree phases. They are no longer relevant and the functionality is largely subsumed by the above. ...and so very much more. I'm surprised I ever got this to compile again!
2017-01-18 21:18:37 +01:00
Doc *diag.Document `json:"-"` // the document from which this package came.
Create MuPack and MuIL packages in our Go toolset This change carries over all of the metadata shapes in the MuPack and MuIL file formats to our Go toolset. This includes creating a proper discriminated AST type tree along with correct annotations so that the metadata will serialize and deserialize correctly.
2017-01-13 23:32:10 +01:00
}
Begin overhauling semantic phases This change further merges the new AST and MuPack/MuIL formats and abstractions into the core of the compiler. A good amount of the old code is gone now; I decided against ripping it all out in one fell swoop so that I can methodically check that we are preserving all relevant decisions and/or functionality we had in the old model. The changes are too numerous to outline in this commit message, however, here are the noteworthy ones: * Split up the notion of symbols and tokens, resulting in: - pkg/symbols for true compiler symbols (bound nodes) - pkg/tokens for name-based tokens, identifiers, constants * Several packages move underneath pkg/compiler: - pkg/ast becomes pkg/compiler/ast - pkg/errors becomes pkg/compiler/errors - pkg/symbols becomes pkg/compiler/symbols * pkg/ast/... becomes pkg/compiler/legacy/ast/... * pkg/pack/ast becomes pkg/compiler/ast. * pkg/options goes away, merged back into pkg/compiler. * All binding functionality moves underneath a dedicated package, pkg/compiler/binder. The legacy.go file contains cruft that will eventually go away, while the other files represent a halfway point between new and old, but are expected to stay roughly in the current shape. * All parsing functionality is moved underneath a new pkg/compiler/metadata namespace, and we adopt new terminology "metadata reading" since real parsing happens in the MetaMu compilers. Hence, Parser has become metadata.Reader. * In general phases of the compiler no longer share access to the actual compiler.Compiler object. Instead, shared state is moved to the core.Context object underneath pkg/compiler/core. * Dependency resolution during binding has been rewritten to the new model, including stashing bound package symbols in the context object, and detecting import cycles. * Compiler construction does not take a workspace object. Instead, creation of a workspace is entirely hidden inside of the compiler's constructor logic. * There are three Compile* functions on the Compiler interface, to support different styles of invoking compilation: Compile() auto- detects a Mu package, based on the workspace; CompilePath(string) loads the target as a Mu package and compiles it, regardless of the workspace settings; and, CompilePackage(*pack.Package) will compile a pre-loaded package AST, again regardless of workspace. * Delete the _fe, _sema, and parsetree phases. They are no longer relevant and the functionality is largely subsumed by the above. ...and so very much more. I'm surprised I ever got this to compile again!
2017-01-18 21:18:37 +01:00
var _ diag.Diagable = (*Package)(nil)
Create MuPack and MuIL packages in our Go toolset This change carries over all of the metadata shapes in the MuPack and MuIL file formats to our Go toolset. This includes creating a proper discriminated AST type tree along with correct annotations so that the metadata will serialize and deserialize correctly.
2017-01-13 23:32:10 +01:00
Begin overhauling semantic phases This change further merges the new AST and MuPack/MuIL formats and abstractions into the core of the compiler. A good amount of the old code is gone now; I decided against ripping it all out in one fell swoop so that I can methodically check that we are preserving all relevant decisions and/or functionality we had in the old model. The changes are too numerous to outline in this commit message, however, here are the noteworthy ones: * Split up the notion of symbols and tokens, resulting in: - pkg/symbols for true compiler symbols (bound nodes) - pkg/tokens for name-based tokens, identifiers, constants * Several packages move underneath pkg/compiler: - pkg/ast becomes pkg/compiler/ast - pkg/errors becomes pkg/compiler/errors - pkg/symbols becomes pkg/compiler/symbols * pkg/ast/... becomes pkg/compiler/legacy/ast/... * pkg/pack/ast becomes pkg/compiler/ast. * pkg/options goes away, merged back into pkg/compiler. * All binding functionality moves underneath a dedicated package, pkg/compiler/binder. The legacy.go file contains cruft that will eventually go away, while the other files represent a halfway point between new and old, but are expected to stay roughly in the current shape. * All parsing functionality is moved underneath a new pkg/compiler/metadata namespace, and we adopt new terminology "metadata reading" since real parsing happens in the MetaMu compilers. Hence, Parser has become metadata.Reader. * In general phases of the compiler no longer share access to the actual compiler.Compiler object. Instead, shared state is moved to the core.Context object underneath pkg/compiler/core. * Dependency resolution during binding has been rewritten to the new model, including stashing bound package symbols in the context object, and detecting import cycles. * Compiler construction does not take a workspace object. Instead, creation of a workspace is entirely hidden inside of the compiler's constructor logic. * There are three Compile* functions on the Compiler interface, to support different styles of invoking compilation: Compile() auto- detects a Mu package, based on the workspace; CompilePath(string) loads the target as a Mu package and compiles it, regardless of the workspace settings; and, CompilePackage(*pack.Package) will compile a pre-loaded package AST, again regardless of workspace. * Delete the _fe, _sema, and parsetree phases. They are no longer relevant and the functionality is largely subsumed by the above. ...and so very much more. I'm surprised I ever got this to compile again!
2017-01-18 21:18:37 +01:00
func (s *Package) Where() (*diag.Document, *diag.Location) {
return s.Doc, nil
Create MuPack and MuIL packages in our Go toolset This change carries over all of the metadata shapes in the MuPack and MuIL file formats to our Go toolset. This includes creating a proper discriminated AST type tree along with correct annotations so that the metadata will serialize and deserialize correctly.
2017-01-13 23:32:10 +01:00
}
Overhaul names versus tokens I was sloppy in my use of names versus tokens in the original AST. Now that we're actually binding things to concrete symbols, etc., we need to be more precise. In particular, names are just identifiers that must be "interpreted" in a given lexical context for them to make any sense; whereas, tokens stand alone and can be resolved without context other than the set of imported packages, modules, and overall module structure. As such, names are much simpler than tokens. As explained in the comments, tokens.Names are simple identifiers: Name = [A-Za-z_][A-Za-z0-9_]* and tokens.QNames are fully qualified identifiers delimited by "/": QName = [ <Name> "/" ]* <Name> The legal grammar for a token depends on the subset of symbols that token is meant to represent. However, the most general case, that accepts all specializations of tokens, is roughly as follows: Token = <Name> | <PackageName> [ ":" <ModuleName> [ "/" <ModuleMemberName> [ "." <Class MemberName> ] ] ] where: PackageName = <QName> ModuleName = <QName> ModuleMemberName = <Name> ClassMemberName = <Name> Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 02:57:20 +01:00
Add the ability to specify analyzers This change adds the ability to specify analyzers in two ways: 1) By listing them in the project file, for example: analyzers: - acmecorp/security - acmecorp/gitflow 2) By explicitly listing them on the CLI, as a "one off": $ coco deploy <env> \ --analyzer=acmecorp/security \ --analyzer=acmecorp/gitflow This closes out pulumi/coconut#119.
2017-03-11 19:07:34 +01:00
// Analyzers is a list of analyzers to run on this project.
type Analyzers []tokens.QName
Overhaul names versus tokens I was sloppy in my use of names versus tokens in the original AST. Now that we're actually binding things to concrete symbols, etc., we need to be more precise. In particular, names are just identifiers that must be "interpreted" in a given lexical context for them to make any sense; whereas, tokens stand alone and can be resolved without context other than the set of imported packages, modules, and overall module structure. As such, names are much simpler than tokens. As explained in the comments, tokens.Names are simple identifiers: Name = [A-Za-z_][A-Za-z0-9_]* and tokens.QNames are fully qualified identifiers delimited by "/": QName = [ <Name> "/" ]* <Name> The legal grammar for a token depends on the subset of symbols that token is meant to represent. However, the most general case, that accepts all specializations of tokens, is roughly as follows: Token = <Name> | <PackageName> [ ":" <ModuleName> [ "/" <ModuleMemberName> [ "." <Class MemberName> ] ] ] where: PackageName = <QName> ModuleName = <QName> ModuleMemberName = <Name> ClassMemberName = <Name> Please refer to the comments in pkg/tokens/tokens.go for more details.
2017-01-20 02:57:20 +01:00
// Dependencies maps dependency names to the full URL, including version, of the package.
Clean up package URL logic This change rearranges the old way we dealt with URLs. In the old system, virtually every reference to an element, including types, was fully qualified with a possible URL-like reference. (The old pkg/tokens/Ref type.) In the new model, only dependency references are URL-like. All maps and references within the MuPack/MuIL format are token and name based, using the new pkg/tokens/Token and pkg/tokens/Name family of related types. As such, this change renames Ref to PackageURLString, and RefParts to PackageURL. (The convenient name is given to the thing with "more" structure, since we prefer to deal with structured types and not strings.) It moves out of the pkg/tokens package and into pkg/pack, since it is exclusively there to support package resolution. Similarly, the Version, VersionSpec, and related types move out of pkg/tokens and into pkg/pack. This change cleans up the various binder, package, and workspace logic. Most of these changes are a natural fallout of this overall restructuring, although in a few places we remained sloppy about the difference between Token, Name, and URL. Now the type system supports these distinctions and forces us to be more methodical about any conversions that take place.
2017-01-20 20:46:36 +01:00
type Dependencies map[tokens.PackageName]PackageURLString
Generalize default modules This change generalizes the support for default modules (added as part of marapongo/mu#57), to use an "alias" table. This is just a table of alternative module names to the real defining module name. The default module is now just represented as a mapping from the special default module token, ".default", to the actual defining module. This generalization was necessary to support Node.js-style alternative names for modules, like "lib" mapping to "lib/index".
2017-02-15 21:53:36 +01:00
// ModuleAliases can be used to map module names to other module names during binding. This is useful for representing
// "default" modules in various forms; e.g., "index" as ".default"; "lib/index" as "lib"; and so on.
type ModuleAliases map[tokens.ModuleName]tokens.ModuleName
Reference in a new issue Copy permalink