* Initial support for passing URLs to `new` and `up` This PR adds initial support for `pulumi new` using Git under the covers to manage Pulumi templates, providing the same experience as before. You can now also optionally pass a URL to a Git repository, e.g. `pulumi new [<url>]`, including subdirectories within the repository, and arbitrary branches, tags, or commits. The following commands result in the same behavior from the user's perspective: - `pulumi new javascript` - `pulumi new https://github.com/pulumi/templates/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/master/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates/javascript` To specify an arbitrary branch, tag, or commit: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates/javascript` Branches and tags can include '/' separators, and `pulumi` will still find the right subdirectory. URLs to Gists are also supported, e.g.: `pulumi new https://gist.github.com/justinvp/6673959ceb9d2ac5a14c6d536cb871a6` If the specified subdirectory in the repository does not contain a `Pulumi.yaml`, it will look for subdirectories within containing `Pulumi.yaml` files, and prompt the user to choose a template, along the lines of how `pulumi new` behaves when no template is specified. The following commands result in the CLI prompting to choose a template: - `pulumi new` - `pulumi new https://github.com/pulumi/templates/templates` - `pulumi new https://github.com/pulumi/templates/tree/master/templates` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates` Of course, arbitrary branches, tags, or commits can be specified as well: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates` This PR also includes initial support for passing URLs to `pulumi up`, providing a streamlined way to deploy installable cloud applications with Pulumi, without having to manage source code locally before doing a deployment. For example, `pulumi up https://github.com/justinvp/aws` can be used to deploy a sample AWS app. The stack can be updated with different versions, e.g. `pulumi up https://github.com/justinvp/aws/tree/v2 -s <stack-to-update>` Config values can optionally be passed via command line flags, e.g. `pulumi up https://github.com/justinvp/aws -c aws:region=us-west-2 -c foo:bar=blah` Gists can also be used, e.g. `pulumi up https://gist.github.com/justinvp/62fde0463f243fcb49f5a7222e51bc76` * Fix panic when hitting ^C from "choose template" prompt * Add description to templates When running `pulumi new` without specifying a template, include the template description along with the name in the "choose template" display. ``` $ pulumi new Please choose a template: aws-go A minimal AWS Go program aws-javascript A minimal AWS JavaScript program aws-python A minimal AWS Python program aws-typescript A minimal AWS TypeScript program > go A minimal Go program hello-aws-javascript A simple AWS serverless JavaScript program javascript A minimal JavaScript program python A minimal Python program typescript A minimal TypeScript program ``` * React to changes to the pulumi/templates repo. We restructured the `pulumi/templates` repo to have all the templates in the root instead of in a `templates` subdirectory, so make the change here to no longer look for templates in `templates`. This also fixes an issue around using `Depth: 1` that I found while testing this. When a named template is used, we attempt to clone or pull from the `pulumi/templates` repo to `~/.pulumi/templates`. Having it go in this well-known directory allows us to maintain previous behavior around allowing offline use of templates. If we use `Depth: 1` for the initial clone, it will fail when attempting to pull when there are updates to the remote repository. Unfortunately, there's no built-in `--unshallow` support in `go-git` and setting a larger `Depth` doesn't appear to help. There may be a workaround, but for now, if we're cloning the pulumi templates directory to `~/.pulumi/templates`, we won't use `Depth: 1`. For template URLs, we will continue to use `Depth: 1` as we clone those to a temp directory (which gets deleted) that we'll never try to update. * List available templates in help text * Address PR Feedback * Don't show "Installing dependencies" message for `up` * Fix secrets handling When prompting for config, if the existing stack value is a secret, keep it a secret and mask the prompt. If the template says it should be secret, make it a secret. * Fix ${PROJECT} and ${DESCRIPTION} handling for `up` Templates used with `up` should already have a filled-in project name and description, but if it's a `new`-style template, that has `${PROJECT}` and/or `${DESCRIPTION}`, be helpful and just replace these with better values. * Fix stack handling Add a bool `setCurrent` param to `requireStack` to control whether the current stack should be saved in workspace settings. For the `up <url>` case, we don't want to save. Also, split the `up` code into two separate functions: one for the `up <url>` case and another for the normal `up` case where you have workspace in your current directory. While we may be able to combine them back into a single function, right now it's a bit cleaner being separate, even with some small amount of duplication. * Fix panic due to nil crypter Lazily get the crypter only if needed inside `promptForConfig`. * Embellish comment * Harden isPreconfiguredEmptyStack check Fix the code to check to make sure the URL specified on the command line matches the URL stored in the `pulumi:template` config value, and that the rest of the config from the stack satisfies the config requirements of the template.
294 lines
8.9 KiB
Go
294 lines
8.9 KiB
Go
// Copyright 2016-2018, Pulumi Corporation.
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package workspace
|
|
|
|
import (
|
|
"encoding/json"
|
|
"io/ioutil"
|
|
"os"
|
|
"path/filepath"
|
|
|
|
"github.com/pulumi/pulumi/pkg/resource/config"
|
|
"github.com/pulumi/pulumi/pkg/util/contract"
|
|
|
|
"github.com/pkg/errors"
|
|
|
|
"github.com/pulumi/pulumi/pkg/encoding"
|
|
"github.com/pulumi/pulumi/pkg/tokens"
|
|
)
|
|
|
|
// Analyzers is a list of analyzers to run on this project.
|
|
type Analyzers []tokens.QName
|
|
|
|
// ProjectTemplate is a Pulumi project template manifest.
|
|
// nolint: lll
|
|
type ProjectTemplate struct {
|
|
Description string `json:"description,omitempty" yaml:"description,omitempty"` // an optional description of the template.
|
|
Quickstart string `json:"quickstart,omitempty" yaml:"quickstart,omitempty"` // optional text to be displayed after template creation.
|
|
Config map[config.Key]ProjectTemplateConfigValue `json:"config,omitempty" yaml:"config,omitempty"` // optional template config.
|
|
}
|
|
|
|
// ProjectTemplateConfigValue is a config value included in the project template manifest.
|
|
// nolint: lll
|
|
type ProjectTemplateConfigValue struct {
|
|
Description string `json:"description,omitempty" yaml:"description,omitempty"` // an optional description for the config value.
|
|
Default string `json:"default,omitempty" yaml:"default,omitempty"` // an optional default value for the config value.
|
|
Secret bool `json:"secret,omitempty" yaml:"secret,omitempty"` // an optional value indicating whether the config value should be encrypted.
|
|
}
|
|
|
|
// Project is a Pulumi project manifest..
|
|
//
|
|
// We explicitly add yaml tags (instead of using the default behavior from https://github.com/ghodss/yaml which works
|
|
// in terms of the JSON tags) so we can directly marshall and unmarshall this struct using go-yaml an have the fields
|
|
// in the serialized object match the order they are defined in this struct.
|
|
//
|
|
// TODO[pulumi/pulumi#423]: use DOM based marshalling so we can roundtrip the seralized structure perfectly.
|
|
// nolint: lll
|
|
type Project struct {
|
|
Name tokens.PackageName `json:"name" yaml:"name"` // a required fully qualified name.
|
|
RuntimeInfo ProjectRuntimeInfo `json:"runtime" yaml:"runtime"` // a required runtime that executes code.
|
|
Main string `json:"main,omitempty" yaml:"main,omitempty"` // an optional override for the main program location.
|
|
|
|
Description *string `json:"description,omitempty" yaml:"description,omitempty"` // an optional informational description.
|
|
Author *string `json:"author,omitempty" yaml:"author,omitempty"` // an optional author.
|
|
Website *string `json:"website,omitempty" yaml:"website,omitempty"` // an optional website for additional info.
|
|
License *string `json:"license,omitempty" yaml:"license,omitempty"` // an optional license governing this project's usage.
|
|
|
|
Analyzers *Analyzers `json:"analyzers,omitempty" yaml:"analyzers,omitempty"` // any analyzers enabled for this project.
|
|
|
|
Context string `json:"context,omitempty" yaml:"context,omitempty"` // an optional path (combined with the on disk location of Pulumi.yaml) to control the data uploaded to the service.
|
|
NoDefaultIgnores *bool `json:"nodefaultignores,omitempty" yaml:"nodefaultignores,omitempty"` // true if we should only respect .pulumiignore when archiving
|
|
|
|
Config string `json:"config,omitempty" yaml:"config,omitempty"` // where to store Pulumi.<stack-name>.yaml files, this is combined with the folder Pulumi.yaml is in.
|
|
|
|
Template *ProjectTemplate `json:"template,omitempty" yaml:"template,omitempty"` // optional template manifest.
|
|
}
|
|
|
|
func (proj *Project) Validate() error {
|
|
if proj.Name == "" {
|
|
return errors.New("project is missing a 'name' attribute")
|
|
}
|
|
if proj.RuntimeInfo.Name() == "" {
|
|
return errors.New("project is missing a 'runtime' attribute")
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func (proj *Project) UseDefaultIgnores() bool {
|
|
if proj.NoDefaultIgnores == nil {
|
|
return true
|
|
}
|
|
|
|
return !(*proj.NoDefaultIgnores)
|
|
}
|
|
|
|
// Save writes a project definition to a file.
|
|
func (proj *Project) Save(path string) error {
|
|
contract.Require(path != "", "path")
|
|
contract.Require(proj != nil, "proj")
|
|
contract.Requiref(proj.Validate() == nil, "proj", "Validate()")
|
|
|
|
m, err := marshallerForPath(path)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
b, err := m.Marshal(proj)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
return ioutil.WriteFile(path, b, 0644)
|
|
}
|
|
|
|
// ProjectStack holds stack specific information about a project.
|
|
// nolint: lll
|
|
type ProjectStack struct {
|
|
EncryptionSalt string `json:"encryptionsalt,omitempty" yaml:"encryptionsalt,omitempty"` // base64 encoded encryption salt.
|
|
Config config.Map `json:"config,omitempty" yaml:"config,omitempty"` // optional config.
|
|
}
|
|
|
|
// Save writes a project definition to a file.
|
|
func (ps *ProjectStack) Save(path string) error {
|
|
contract.Require(path != "", "path")
|
|
contract.Require(ps != nil, "ps")
|
|
|
|
m, err := marshallerForPath(path)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
b, err := m.Marshal(ps)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
// nolint: gas, gas prefers 0700 for a directory, but 0755 (so group and world can read it) is what we prefer
|
|
if err := os.MkdirAll(filepath.Dir(path), 0755); err != nil {
|
|
return err
|
|
}
|
|
|
|
return ioutil.WriteFile(path, b, 0644)
|
|
}
|
|
|
|
type ProjectRuntimeInfo struct {
|
|
name string
|
|
options map[string]interface{}
|
|
}
|
|
|
|
func NewProjectRuntimeInfo(name string, options map[string]interface{}) ProjectRuntimeInfo {
|
|
return ProjectRuntimeInfo{
|
|
name: name,
|
|
options: options,
|
|
}
|
|
}
|
|
|
|
func (info *ProjectRuntimeInfo) Name() string {
|
|
return info.name
|
|
}
|
|
|
|
func (info *ProjectRuntimeInfo) Options() map[string]interface{} {
|
|
return info.options
|
|
}
|
|
|
|
func (info ProjectRuntimeInfo) MarshalYAML() (interface{}, error) {
|
|
if info.options == nil || len(info.options) == 0 {
|
|
return info.name, nil
|
|
}
|
|
|
|
return map[string]interface{}{
|
|
"name": info.name,
|
|
"options": info.options,
|
|
}, nil
|
|
}
|
|
|
|
func (info ProjectRuntimeInfo) MarshalJSON() ([]byte, error) {
|
|
if info.options == nil || len(info.options) == 0 {
|
|
return json.Marshal(info.name)
|
|
}
|
|
|
|
return json.Marshal(map[string]interface{}{
|
|
"name": info.name,
|
|
"options": info.options,
|
|
})
|
|
}
|
|
|
|
func (info *ProjectRuntimeInfo) UnmarshalJSON(data []byte) error {
|
|
if err := json.Unmarshal(data, &info.name); err == nil {
|
|
return nil
|
|
}
|
|
|
|
var payload struct {
|
|
Name string `json:"name"`
|
|
Options map[string]interface{} `json:"options"`
|
|
}
|
|
|
|
if err := json.Unmarshal(data, &payload); err == nil {
|
|
info.name = payload.Name
|
|
info.options = payload.Options
|
|
return nil
|
|
}
|
|
|
|
return errors.New("runtime section must be a string or an object with name and options attributes")
|
|
}
|
|
|
|
func (info *ProjectRuntimeInfo) UnmarshalYAML(unmarshal func(interface{}) error) error {
|
|
if err := unmarshal(&info.name); err == nil {
|
|
return nil
|
|
}
|
|
|
|
var payload struct {
|
|
Name string `yaml:"name"`
|
|
Options map[string]interface{} `yaml:"options"`
|
|
}
|
|
|
|
if err := unmarshal(&payload); err == nil {
|
|
info.name = payload.Name
|
|
info.options = payload.Options
|
|
return nil
|
|
}
|
|
|
|
return errors.New("runtime section must be a string or an object with name and options attributes")
|
|
}
|
|
|
|
// LoadProject reads a project definition from a file.
|
|
func LoadProject(path string) (*Project, error) {
|
|
contract.Require(path != "", "path")
|
|
|
|
m, err := marshallerForPath(path)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
b, err := ioutil.ReadFile(path)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
var proj Project
|
|
err = m.Unmarshal(b, &proj)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
err = proj.Validate()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
return &proj, err
|
|
}
|
|
|
|
// LoadProjectStack reads a stack definition from a file.
|
|
func LoadProjectStack(path string) (*ProjectStack, error) {
|
|
contract.Require(path != "", "path")
|
|
|
|
m, err := marshallerForPath(path)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
b, err := ioutil.ReadFile(path)
|
|
if os.IsNotExist(err) {
|
|
return &ProjectStack{
|
|
Config: make(config.Map),
|
|
}, nil
|
|
} else if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
var ps ProjectStack
|
|
err = m.Unmarshal(b, &ps)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
if ps.Config == nil {
|
|
ps.Config = make(config.Map)
|
|
}
|
|
|
|
return &ps, err
|
|
}
|
|
|
|
func marshallerForPath(path string) (encoding.Marshaler, error) {
|
|
ext := filepath.Ext(path)
|
|
m, has := encoding.Marshalers[ext]
|
|
if !has {
|
|
return nil, errors.Errorf("no marshaler found for file format '%v'", ext)
|
|
}
|
|
|
|
return m, nil
|
|
}
|