* Initial support for passing URLs to `new` and `up` This PR adds initial support for `pulumi new` using Git under the covers to manage Pulumi templates, providing the same experience as before. You can now also optionally pass a URL to a Git repository, e.g. `pulumi new [<url>]`, including subdirectories within the repository, and arbitrary branches, tags, or commits. The following commands result in the same behavior from the user's perspective: - `pulumi new javascript` - `pulumi new https://github.com/pulumi/templates/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/master/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates/javascript` To specify an arbitrary branch, tag, or commit: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates/javascript` Branches and tags can include '/' separators, and `pulumi` will still find the right subdirectory. URLs to Gists are also supported, e.g.: `pulumi new https://gist.github.com/justinvp/6673959ceb9d2ac5a14c6d536cb871a6` If the specified subdirectory in the repository does not contain a `Pulumi.yaml`, it will look for subdirectories within containing `Pulumi.yaml` files, and prompt the user to choose a template, along the lines of how `pulumi new` behaves when no template is specified. The following commands result in the CLI prompting to choose a template: - `pulumi new` - `pulumi new https://github.com/pulumi/templates/templates` - `pulumi new https://github.com/pulumi/templates/tree/master/templates` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates` Of course, arbitrary branches, tags, or commits can be specified as well: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates` This PR also includes initial support for passing URLs to `pulumi up`, providing a streamlined way to deploy installable cloud applications with Pulumi, without having to manage source code locally before doing a deployment. For example, `pulumi up https://github.com/justinvp/aws` can be used to deploy a sample AWS app. The stack can be updated with different versions, e.g. `pulumi up https://github.com/justinvp/aws/tree/v2 -s <stack-to-update>` Config values can optionally be passed via command line flags, e.g. `pulumi up https://github.com/justinvp/aws -c aws:region=us-west-2 -c foo:bar=blah` Gists can also be used, e.g. `pulumi up https://gist.github.com/justinvp/62fde0463f243fcb49f5a7222e51bc76` * Fix panic when hitting ^C from "choose template" prompt * Add description to templates When running `pulumi new` without specifying a template, include the template description along with the name in the "choose template" display. ``` $ pulumi new Please choose a template: aws-go A minimal AWS Go program aws-javascript A minimal AWS JavaScript program aws-python A minimal AWS Python program aws-typescript A minimal AWS TypeScript program > go A minimal Go program hello-aws-javascript A simple AWS serverless JavaScript program javascript A minimal JavaScript program python A minimal Python program typescript A minimal TypeScript program ``` * React to changes to the pulumi/templates repo. We restructured the `pulumi/templates` repo to have all the templates in the root instead of in a `templates` subdirectory, so make the change here to no longer look for templates in `templates`. This also fixes an issue around using `Depth: 1` that I found while testing this. When a named template is used, we attempt to clone or pull from the `pulumi/templates` repo to `~/.pulumi/templates`. Having it go in this well-known directory allows us to maintain previous behavior around allowing offline use of templates. If we use `Depth: 1` for the initial clone, it will fail when attempting to pull when there are updates to the remote repository. Unfortunately, there's no built-in `--unshallow` support in `go-git` and setting a larger `Depth` doesn't appear to help. There may be a workaround, but for now, if we're cloning the pulumi templates directory to `~/.pulumi/templates`, we won't use `Depth: 1`. For template URLs, we will continue to use `Depth: 1` as we clone those to a temp directory (which gets deleted) that we'll never try to update. * List available templates in help text * Address PR Feedback * Don't show "Installing dependencies" message for `up` * Fix secrets handling When prompting for config, if the existing stack value is a secret, keep it a secret and mask the prompt. If the template says it should be secret, make it a secret. * Fix ${PROJECT} and ${DESCRIPTION} handling for `up` Templates used with `up` should already have a filled-in project name and description, but if it's a `new`-style template, that has `${PROJECT}` and/or `${DESCRIPTION}`, be helpful and just replace these with better values. * Fix stack handling Add a bool `setCurrent` param to `requireStack` to control whether the current stack should be saved in workspace settings. For the `up <url>` case, we don't want to save. Also, split the `up` code into two separate functions: one for the `up <url>` case and another for the normal `up` case where you have workspace in your current directory. While we may be able to combine them back into a single function, right now it's a bit cleaner being separate, even with some small amount of duplication. * Fix panic due to nil crypter Lazily get the crypter only if needed inside `promptForConfig`. * Embellish comment * Harden isPreconfiguredEmptyStack check Fix the code to check to make sure the URL specified on the command line matches the URL stored in the `pulumi:template` config value, and that the rest of the config from the stack satisfies the config requirements of the template.
135 lines
4.3 KiB
Go
135 lines
4.3 KiB
Go
// Copyright 2016-2018, Pulumi Corporation.
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package testing
|
|
|
|
import (
|
|
"bytes"
|
|
"fmt"
|
|
"io/ioutil"
|
|
"os"
|
|
"os/exec"
|
|
"path"
|
|
"path/filepath"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/pulumi/pulumi/pkg/util/fsutil"
|
|
"github.com/stretchr/testify/assert"
|
|
)
|
|
|
|
const (
|
|
bookkeepingDir = ".pulumi"
|
|
pulumiCredentialsPathEnvVar = "PULUMI_CREDENTIALS_PATH"
|
|
)
|
|
|
|
// Environment is an extension of the testing.T type that provides support for a test environment
|
|
// on the local disk. The Environment has a root directory (e.g. a newly created temp folder) and
|
|
// a current working directory (to virtually change directories).
|
|
type Environment struct {
|
|
*testing.T
|
|
|
|
// RootPath is a new temp directory where the environment starts.
|
|
RootPath string
|
|
// Current working directory.
|
|
CWD string
|
|
}
|
|
|
|
// NewEnvironment returns a new Environment object, located in a temp directory.
|
|
func NewEnvironment(t *testing.T) *Environment {
|
|
root, err := ioutil.TempDir("", "test-env")
|
|
assert.NoError(t, err, "creating temp directory")
|
|
|
|
t.Logf("Created new test environment: %v", root)
|
|
return &Environment{
|
|
T: t,
|
|
RootPath: root,
|
|
CWD: root,
|
|
}
|
|
}
|
|
|
|
// ImportDirectory copies a folder into the test environment.
|
|
func (e *Environment) ImportDirectory(path string) {
|
|
err := fsutil.CopyFile(e.RootPath, path, nil)
|
|
if err != nil {
|
|
e.T.Fatalf("error importing directory: %v", err)
|
|
}
|
|
}
|
|
|
|
// DeleteEnvironment deletes the environment's RootPath, and everything underneath it.
|
|
func (e *Environment) DeleteEnvironment() {
|
|
e.Helper()
|
|
err := os.RemoveAll(e.RootPath)
|
|
assert.NoError(e, err, "cleaning up the test directory")
|
|
}
|
|
|
|
// PathExists returns whether or not a file or directory exists relative to Environment's working directory.
|
|
func (e *Environment) PathExists(p string) bool {
|
|
fullPath := path.Join(e.CWD, p)
|
|
_, err := os.Stat(fullPath)
|
|
return err == nil
|
|
}
|
|
|
|
// RunCommand runs the command expecting a zero exit code, returning stdout and stderr.
|
|
func (e *Environment) RunCommand(cmd string, args ...string) (string, string) {
|
|
e.Helper()
|
|
stdout, stderr, err := e.GetCommandResults(e.T, cmd, args...)
|
|
if err != nil {
|
|
e.Errorf("Ran command %v args %v and expected success. Instead got failure.", cmd, args)
|
|
e.Logf("Run Error: %v", err)
|
|
e.Logf("STDOUT: %v", stdout)
|
|
e.Logf("STDERR: %v", stderr)
|
|
}
|
|
return stdout, stderr
|
|
}
|
|
|
|
// RunCommandExpectError runs the command expecting a non-zero exit code, returning stdout and stderr.
|
|
func (e *Environment) RunCommandExpectError(cmd string, args ...string) (string, string) {
|
|
e.Helper()
|
|
stdout, stderr, err := e.GetCommandResults(e.T, cmd, args...)
|
|
if err == nil {
|
|
e.Errorf("Ran command %v args %v and expected failure. Instead got success.", cmd, args)
|
|
e.Logf("STDOUT: %v", stdout)
|
|
e.Logf("STDERR: %v", stderr)
|
|
}
|
|
return stdout, stderr
|
|
}
|
|
|
|
// LocalURL returns a URL that uses the "fire and forget", storing its data inside the test folder (so multiple tests)
|
|
// may reuse stack names.
|
|
func (e *Environment) LocalURL() string {
|
|
return "local://" + filepath.Join(e.RootPath, bookkeepingDir)
|
|
}
|
|
|
|
// GetCommandResults runs the given command and args in the Environments CWD, returning
|
|
// STDOUT, STDERR, and the result of os/exec.Command{}.Run.
|
|
func (e *Environment) GetCommandResults(t *testing.T, command string, args ...string) (string, string, error) {
|
|
t.Helper()
|
|
t.Logf("Running command %v %v", command, strings.Join(args, " "))
|
|
|
|
// Buffer STDOUT and STDERR so we can return them later.
|
|
var outBuffer bytes.Buffer
|
|
var errBuffer bytes.Buffer
|
|
|
|
// nolint: gas
|
|
cmd := exec.Command(command, args...)
|
|
cmd.Dir = e.CWD
|
|
cmd.Stdout = &outBuffer
|
|
cmd.Stderr = &errBuffer
|
|
cmd.Env = append(os.Environ(), fmt.Sprintf("%s=%s", pulumiCredentialsPathEnvVar, e.RootPath))
|
|
cmd.Env = append(cmd.Env, "PULUMI_DEBUG_COMMANDS=true")
|
|
|
|
runErr := cmd.Run()
|
|
return outBuffer.String(), errBuffer.String(), runErr
|
|
}
|