* Initial support for passing URLs to `new` and `up` This PR adds initial support for `pulumi new` using Git under the covers to manage Pulumi templates, providing the same experience as before. You can now also optionally pass a URL to a Git repository, e.g. `pulumi new [<url>]`, including subdirectories within the repository, and arbitrary branches, tags, or commits. The following commands result in the same behavior from the user's perspective: - `pulumi new javascript` - `pulumi new https://github.com/pulumi/templates/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/master/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates/javascript` To specify an arbitrary branch, tag, or commit: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates/javascript` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates/javascript` Branches and tags can include '/' separators, and `pulumi` will still find the right subdirectory. URLs to Gists are also supported, e.g.: `pulumi new https://gist.github.com/justinvp/6673959ceb9d2ac5a14c6d536cb871a6` If the specified subdirectory in the repository does not contain a `Pulumi.yaml`, it will look for subdirectories within containing `Pulumi.yaml` files, and prompt the user to choose a template, along the lines of how `pulumi new` behaves when no template is specified. The following commands result in the CLI prompting to choose a template: - `pulumi new` - `pulumi new https://github.com/pulumi/templates/templates` - `pulumi new https://github.com/pulumi/templates/tree/master/templates` - `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates` Of course, arbitrary branches, tags, or commits can be specified as well: - `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates` - `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates` This PR also includes initial support for passing URLs to `pulumi up`, providing a streamlined way to deploy installable cloud applications with Pulumi, without having to manage source code locally before doing a deployment. For example, `pulumi up https://github.com/justinvp/aws` can be used to deploy a sample AWS app. The stack can be updated with different versions, e.g. `pulumi up https://github.com/justinvp/aws/tree/v2 -s <stack-to-update>` Config values can optionally be passed via command line flags, e.g. `pulumi up https://github.com/justinvp/aws -c aws:region=us-west-2 -c foo:bar=blah` Gists can also be used, e.g. `pulumi up https://gist.github.com/justinvp/62fde0463f243fcb49f5a7222e51bc76` * Fix panic when hitting ^C from "choose template" prompt * Add description to templates When running `pulumi new` without specifying a template, include the template description along with the name in the "choose template" display. ``` $ pulumi new Please choose a template: aws-go A minimal AWS Go program aws-javascript A minimal AWS JavaScript program aws-python A minimal AWS Python program aws-typescript A minimal AWS TypeScript program > go A minimal Go program hello-aws-javascript A simple AWS serverless JavaScript program javascript A minimal JavaScript program python A minimal Python program typescript A minimal TypeScript program ``` * React to changes to the pulumi/templates repo. We restructured the `pulumi/templates` repo to have all the templates in the root instead of in a `templates` subdirectory, so make the change here to no longer look for templates in `templates`. This also fixes an issue around using `Depth: 1` that I found while testing this. When a named template is used, we attempt to clone or pull from the `pulumi/templates` repo to `~/.pulumi/templates`. Having it go in this well-known directory allows us to maintain previous behavior around allowing offline use of templates. If we use `Depth: 1` for the initial clone, it will fail when attempting to pull when there are updates to the remote repository. Unfortunately, there's no built-in `--unshallow` support in `go-git` and setting a larger `Depth` doesn't appear to help. There may be a workaround, but for now, if we're cloning the pulumi templates directory to `~/.pulumi/templates`, we won't use `Depth: 1`. For template URLs, we will continue to use `Depth: 1` as we clone those to a temp directory (which gets deleted) that we'll never try to update. * List available templates in help text * Address PR Feedback * Don't show "Installing dependencies" message for `up` * Fix secrets handling When prompting for config, if the existing stack value is a secret, keep it a secret and mask the prompt. If the template says it should be secret, make it a secret. * Fix ${PROJECT} and ${DESCRIPTION} handling for `up` Templates used with `up` should already have a filled-in project name and description, but if it's a `new`-style template, that has `${PROJECT}` and/or `${DESCRIPTION}`, be helpful and just replace these with better values. * Fix stack handling Add a bool `setCurrent` param to `requireStack` to control whether the current stack should be saved in workspace settings. For the `up <url>` case, we don't want to save. Also, split the `up` code into two separate functions: one for the `up <url>` case and another for the normal `up` case where you have workspace in your current directory. While we may be able to combine them back into a single function, right now it's a bit cleaner being separate, even with some small amount of duplication. * Fix panic due to nil crypter Lazily get the crypter only if needed inside `promptForConfig`. * Embellish comment * Harden isPreconfiguredEmptyStack check Fix the code to check to make sure the URL specified on the command line matches the URL stored in the `pulumi:template` config value, and that the rest of the config from the stack satisfies the config requirements of the template.
451 lines
12 KiB
Go
451 lines
12 KiB
Go
// Copyright 2016-2018, Pulumi Corporation.
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package cmd
|
|
|
|
import (
|
|
"fmt"
|
|
"io/ioutil"
|
|
"os"
|
|
"regexp"
|
|
"sort"
|
|
"strconv"
|
|
"strings"
|
|
|
|
zxcvbn "github.com/nbutton23/zxcvbn-go"
|
|
"github.com/pkg/errors"
|
|
"github.com/spf13/cobra"
|
|
"golang.org/x/crypto/ssh/terminal"
|
|
|
|
"github.com/pulumi/pulumi/pkg/backend"
|
|
"github.com/pulumi/pulumi/pkg/resource/config"
|
|
"github.com/pulumi/pulumi/pkg/tokens"
|
|
"github.com/pulumi/pulumi/pkg/util/cmdutil"
|
|
"github.com/pulumi/pulumi/pkg/workspace"
|
|
)
|
|
|
|
func newConfigCmd() *cobra.Command {
|
|
var stack string
|
|
var showSecrets bool
|
|
|
|
cmd := &cobra.Command{
|
|
Use: "config",
|
|
Short: "Manage configuration",
|
|
Long: "Lists all configuration values for a specific stack. To add a new configuration value, run\n" +
|
|
"'pulumi config set', to remove and existing value run 'pulumi config rm'. To get the value of\n" +
|
|
"for a specific configuration key, use 'pulumi config get <key-name>'.",
|
|
Args: cmdutil.NoArgs,
|
|
Run: cmdutil.RunFunc(func(cmd *cobra.Command, args []string) error {
|
|
opts := backend.DisplayOptions{
|
|
Color: cmdutil.GetGlobalColorization(),
|
|
}
|
|
|
|
stack, err := requireStack(stack, true, opts, true /*setCurrent*/)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
return listConfig(stack, showSecrets)
|
|
}),
|
|
}
|
|
|
|
cmd.Flags().BoolVar(
|
|
&showSecrets, "show-secrets", false,
|
|
"Show secret values when listing config instead of displaying blinded values")
|
|
cmd.PersistentFlags().StringVarP(
|
|
&stack, "stack", "s", "",
|
|
"The name of the stack to operate on. Defaults to the current stack")
|
|
|
|
cmd.AddCommand(newConfigGetCmd(&stack))
|
|
cmd.AddCommand(newConfigRmCmd(&stack))
|
|
cmd.AddCommand(newConfigSetCmd(&stack))
|
|
cmd.AddCommand(newConfigRefreshCmd(&stack))
|
|
|
|
return cmd
|
|
}
|
|
|
|
func newConfigGetCmd(stack *string) *cobra.Command {
|
|
getCmd := &cobra.Command{
|
|
Use: "get <key>",
|
|
Short: "Get a single configuration value",
|
|
Args: cmdutil.SpecificArgs([]string{"key"}),
|
|
Run: cmdutil.RunFunc(func(cmd *cobra.Command, args []string) error {
|
|
opts := backend.DisplayOptions{
|
|
Color: cmdutil.GetGlobalColorization(),
|
|
}
|
|
|
|
s, err := requireStack(*stack, true, opts, true /*setCurrent*/)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
key, err := parseConfigKey(args[0])
|
|
if err != nil {
|
|
return errors.Wrap(err, "invalid configuration key")
|
|
}
|
|
|
|
return getConfig(s, key)
|
|
}),
|
|
}
|
|
|
|
return getCmd
|
|
}
|
|
|
|
func newConfigRmCmd(stack *string) *cobra.Command {
|
|
rmCmd := &cobra.Command{
|
|
Use: "rm <key>",
|
|
Short: "Remove configuration value",
|
|
Args: cmdutil.SpecificArgs([]string{"key"}),
|
|
Run: cmdutil.RunFunc(func(cmd *cobra.Command, args []string) error {
|
|
opts := backend.DisplayOptions{
|
|
Color: cmdutil.GetGlobalColorization(),
|
|
}
|
|
|
|
s, err := requireStack(*stack, true, opts, true /*setCurrent*/)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
key, err := parseConfigKey(args[0])
|
|
if err != nil {
|
|
return errors.Wrap(err, "invalid configuration key")
|
|
}
|
|
|
|
ps, err := workspace.DetectProjectStack(s.Name().StackName())
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
if ps.Config != nil {
|
|
delete(ps.Config, key)
|
|
}
|
|
|
|
return workspace.SaveProjectStack(s.Name().StackName(), ps)
|
|
}),
|
|
}
|
|
|
|
return rmCmd
|
|
}
|
|
|
|
func newConfigRefreshCmd(stack *string) *cobra.Command {
|
|
var force bool
|
|
refreshCmd := &cobra.Command{
|
|
Use: "refresh",
|
|
Short: "Update the local configuration based on the most recent deployment of the stack",
|
|
Args: cmdutil.NoArgs,
|
|
Run: cmdutil.RunFunc(func(cmd *cobra.Command, args []string) error {
|
|
opts := backend.DisplayOptions{
|
|
Color: cmdutil.GetGlobalColorization(),
|
|
}
|
|
|
|
// Ensure the stack exists.
|
|
s, err := requireStack(*stack, false, opts, true /*setCurrent*/)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
c, err := backend.GetLatestConfiguration(commandContext(), s)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
configPath, err := workspace.DetectProjectStackPath(s.Name().StackName())
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
ps, err := workspace.LoadProjectStack(configPath)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
ps.Config = c
|
|
|
|
// If the configuration file doesn't exist, or force has been passed, save it in place.
|
|
if _, err = os.Stat(configPath); os.IsNotExist(err) || force {
|
|
return ps.Save(configPath)
|
|
}
|
|
|
|
// Otherwise we'll create a backup, let's figure out what name to use by adding ".bak" over and over
|
|
// until we get to a name not in use.
|
|
backupFile := configPath + ".bak"
|
|
for {
|
|
_, err = os.Stat(backupFile)
|
|
if os.IsNotExist(err) {
|
|
if err = os.Rename(configPath, backupFile); err != nil {
|
|
return errors.Wrap(err, "backing up existing configuration file")
|
|
}
|
|
|
|
fmt.Printf("backed up existing configuration file to %s\n", backupFile)
|
|
break
|
|
} else if err != nil {
|
|
return errors.Wrap(err, "backing up existing configuration file")
|
|
}
|
|
|
|
backupFile = backupFile + ".bak"
|
|
}
|
|
|
|
err = ps.Save(configPath)
|
|
if err == nil {
|
|
fmt.Printf("refreshed configuration for stack '%s'\n", s.Name().String())
|
|
}
|
|
return err
|
|
}),
|
|
}
|
|
refreshCmd.PersistentFlags().BoolVarP(
|
|
&force, "force", "f", false, "Overwrite configuration file, if it exists, without creating a backup")
|
|
|
|
return refreshCmd
|
|
}
|
|
|
|
func newConfigSetCmd(stack *string) *cobra.Command {
|
|
var plaintext bool
|
|
var secret bool
|
|
|
|
setCmd := &cobra.Command{
|
|
Use: "set <key> [value]",
|
|
Short: "Set configuration value",
|
|
Long: "Configuration values can be accessed when a stack is being deployed and used to configure behavior. \n" +
|
|
"If a value is not present on the command line, pulumi will prompt for the value. Multi-line values\n" +
|
|
"may be set by piping a file to standard in.",
|
|
Args: cmdutil.RangeArgs(1, 2),
|
|
Run: cmdutil.RunFunc(func(cmd *cobra.Command, args []string) error {
|
|
opts := backend.DisplayOptions{
|
|
Color: cmdutil.GetGlobalColorization(),
|
|
}
|
|
|
|
// Ensure the stack exists.
|
|
s, err := requireStack(*stack, true, opts, true /*setCurrent*/)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
key, err := parseConfigKey(args[0])
|
|
if err != nil {
|
|
return errors.Wrap(err, "invalid configuration key")
|
|
}
|
|
|
|
var value string
|
|
switch {
|
|
case len(args) == 2:
|
|
value = args[1]
|
|
case !terminal.IsTerminal(int(os.Stdin.Fd())):
|
|
b, readerr := ioutil.ReadAll(os.Stdin)
|
|
if readerr != nil {
|
|
return readerr
|
|
}
|
|
value = cmdutil.RemoveTralingNewline(string(b))
|
|
case secret:
|
|
value, err = cmdutil.ReadConsoleNoEcho("value")
|
|
if err != nil {
|
|
return err
|
|
}
|
|
default:
|
|
value, err = cmdutil.ReadConsole("value")
|
|
if err != nil {
|
|
return err
|
|
}
|
|
}
|
|
|
|
// Encrypt the config value if needed.
|
|
var v config.Value
|
|
if secret {
|
|
c, cerr := backend.GetStackCrypter(s)
|
|
if cerr != nil {
|
|
return cerr
|
|
}
|
|
enc, eerr := c.EncryptValue(value)
|
|
if eerr != nil {
|
|
return eerr
|
|
}
|
|
v = config.NewSecureValue(enc)
|
|
} else {
|
|
v = config.NewValue(value)
|
|
|
|
// If we saved a plaintext configuration value, and --plaintext was not passed, warn the user.
|
|
if !plaintext && looksLikeSecret(key, value) {
|
|
return errors.Errorf(
|
|
"config value '%s' looks like a secret; "+
|
|
"rerun with --secret to encrypt it, or --plaintext if you meant to store in plaintext",
|
|
value)
|
|
}
|
|
}
|
|
|
|
ps, err := workspace.DetectProjectStack(s.Name().StackName())
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
ps.Config[key] = v
|
|
|
|
return workspace.SaveProjectStack(s.Name().StackName(), ps)
|
|
}),
|
|
}
|
|
|
|
setCmd.PersistentFlags().BoolVar(
|
|
&plaintext, "plaintext", false,
|
|
"Save the value as plaintext (unencrypted)")
|
|
setCmd.PersistentFlags().BoolVar(
|
|
&secret, "secret", false,
|
|
"Encrypt the value instead of storing it in plaintext")
|
|
|
|
return setCmd
|
|
}
|
|
|
|
func parseConfigKey(key string) (config.Key, error) {
|
|
// As a convience, we'll treat any key with no delimiter as if:
|
|
// <program-name>:config:<key> had been written instead
|
|
if !strings.Contains(key, tokens.TokenDelimiter) {
|
|
proj, err := workspace.DetectProject()
|
|
if err != nil {
|
|
return config.Key{}, err
|
|
}
|
|
|
|
return config.ParseKey(fmt.Sprintf("%s:config:%s", proj.Name, key))
|
|
}
|
|
|
|
return config.ParseKey(key)
|
|
}
|
|
|
|
func prettyKey(k config.Key) string {
|
|
proj, err := workspace.DetectProject()
|
|
if err != nil {
|
|
return fmt.Sprintf("%s:%s", k.Namespace(), k.Name())
|
|
}
|
|
|
|
return prettyKeyForProject(k, proj)
|
|
}
|
|
|
|
func prettyKeyForProject(k config.Key, proj *workspace.Project) string {
|
|
if k.Namespace() == string(proj.Name) {
|
|
return k.Name()
|
|
}
|
|
|
|
return fmt.Sprintf("%s:%s", k.Namespace(), k.Name())
|
|
}
|
|
|
|
func listConfig(stack backend.Stack, showSecrets bool) error {
|
|
ps, err := workspace.DetectProjectStack(stack.Name().StackName())
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
cfg := ps.Config
|
|
|
|
// By default, we will use a blinding decrypter to show '******'. If requested, display secrets in plaintext.
|
|
var decrypter config.Decrypter
|
|
if cfg.HasSecureValue() && showSecrets {
|
|
decrypter, err = backend.GetStackCrypter(stack)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
} else {
|
|
decrypter = config.NewBlindingDecrypter()
|
|
}
|
|
|
|
fullKey := func(k config.Key) string {
|
|
return fmt.Sprintf("%s:%s", k.Namespace(), k.Name())
|
|
}
|
|
|
|
// Devote 48 characters to the config key, unless there's a key longer, in which case use that.
|
|
maxkey := 48
|
|
for key := range cfg {
|
|
if len(fullKey(key)) > maxkey {
|
|
maxkey = len(fullKey(key))
|
|
}
|
|
}
|
|
|
|
fmt.Printf("%-"+strconv.Itoa(maxkey)+"s %-48s\n", "KEY", "VALUE")
|
|
var keys config.KeyArray
|
|
for key := range cfg {
|
|
// Note that we use the fully qualified module member here instead of a `prettyKey`, this lets us ensure
|
|
// that all the config values for the current program are displayed next to one another in the output.
|
|
keys = append(keys, key)
|
|
}
|
|
sort.Sort(keys)
|
|
for _, key := range keys {
|
|
decrypted, err := cfg[key].Value(decrypter)
|
|
if err != nil {
|
|
return errors.Wrap(err, "could not decrypt configuration value")
|
|
}
|
|
|
|
fmt.Printf("%-"+strconv.Itoa(maxkey)+"s %-48s\n", prettyKey(key), decrypted)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
func getConfig(stack backend.Stack, key config.Key) error {
|
|
ps, err := workspace.DetectProjectStack(stack.Name().StackName())
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
cfg := ps.Config
|
|
|
|
if v, ok := cfg[key]; ok {
|
|
var d config.Decrypter
|
|
if v.Secure() {
|
|
var err error
|
|
if d, err = backend.GetStackCrypter(stack); err != nil {
|
|
return errors.Wrap(err, "could not create a decrypter")
|
|
}
|
|
} else {
|
|
d = config.NewPanicCrypter()
|
|
}
|
|
raw, err := v.Value(d)
|
|
if err != nil {
|
|
return errors.Wrap(err, "could not decrypt configuration value")
|
|
}
|
|
fmt.Printf("%v\n", raw)
|
|
return nil
|
|
}
|
|
|
|
return errors.Errorf(
|
|
"configuration key '%s' not found for stack '%s'", prettyKey(key), stack.Name())
|
|
}
|
|
|
|
var (
|
|
// keyPattern is the regular expression a configuration key must match before we check (and error) if we think
|
|
// it is a password
|
|
keyPattern = regexp.MustCompile("(?i)passwd|pass|password|pwd|secret|token")
|
|
)
|
|
|
|
const (
|
|
// maxEntropyCheckLength is the maximum length of a possible secret for entropy checking.
|
|
maxEntropyCheckLength = 16
|
|
// entropyThreshold is the total entropy threshold a potential secret needs to pass before being flagged.
|
|
entropyThreshold = 80.0
|
|
// entropyCharThreshold is the per-char entropy threshold a potential secret needs to pass before being flagged.
|
|
entropyPerCharThreshold = 3.0
|
|
)
|
|
|
|
// looksLikeSecret returns true if a configuration value "looks" like a secret. This is always going to be a heuristic
|
|
// that suffers from false positives, but is better (a) than our prior approach of unconditionally printing a warning
|
|
// for all plaintext values, and (b) to be paranoid about such things. Inspired by the gas linter and securego project.
|
|
func looksLikeSecret(k config.Key, v string) bool {
|
|
if !keyPattern.MatchString(k.Name()) {
|
|
return false
|
|
}
|
|
|
|
if len(v) > maxEntropyCheckLength {
|
|
v = v[:maxEntropyCheckLength]
|
|
}
|
|
|
|
// Compute the strength use the resulting entropy to flag whether this looks like a secret.
|
|
info := zxcvbn.PasswordStrength(v, nil)
|
|
entropyPerChar := info.Entropy / float64(len(v))
|
|
return (info.Entropy >= entropyThreshold ||
|
|
(info.Entropy >= (entropyThreshold/2) && entropyPerChar >= entropyPerCharThreshold))
|
|
}
|