This change adds support for setting `PULUMI_PREFER_YARN` to true to opt-in to preferring `yarn` over `npm` when installing Node.js dependencies (and publishing Policy Packs). If `PULUMI_PREFER_YARN` is truthy, but `yarn` cannot be found on `$PATH`, we fallback to using `npm`. If `npm` can't be found on `$PATH`, we provide a more helpful error message.
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).
For example:
```yaml
config:
proj:blah:
- a
- b
- c
proj:hello: world
proj:outer:
inner: value
proj:servers:
- port: 80
```
While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.
As always, you can specify single values with:
```shell
$ pulumi config set hello world
```
Which results in the following YAML:
```yaml
proj:hello world
```
And single value secrets via:
```shell
$ pulumi config set --secret token shhh
```
Which results in the following YAML:
```yaml
proj:token:
secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```
Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:
```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```
Which results in:
```yaml
proj:names
- a
- b
- c
```
Values can be obtained similarly:
```shell
$ pulumi config get --path names[1]
b
```
Or setting values in a map:
```shell
$ pulumi config set --path outer.inner value
```
Which results in:
```yaml
proj:outer:
inner: value
```
Of course, setting values in nested structures is supported:
```shell
$ pulumi config set --path servers[0].port 80
```
Which results in:
```yaml
proj:servers:
- port: 80
```
If you want to include a period in the name of a property, it can be
specified as:
```
$ pulumi config set --path 'nested["foo.bar"]' baz
```
Which results in:
```yaml
proj:nested:
foo.bar: baz
```
Examples of valid paths:
- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'
Note: paths that contain quotes can be surrounded by single quotes.
When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.
Secure values are supported in lists/maps as well:
```shell
$ pulumi config set --path --secret tokens[0] shh
```
Will result in:
```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```
Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:
```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```
**Accessing config values from the command line with JSON**
```shell
$ pulumi config --json
```
Will output:
```json
{
"proj:hello": {
"value": "world",
"secret": false,
"object": false
},
"proj:names": {
"value": "[\"a\",\"b\",\"c\"]",
"secret": false,
"object": true,
"objectValue": [
"a",
"b",
"c"
]
},
"proj:nested": {
"value": "{\"foo.bar\":\"baz\"}",
"secret": false,
"object": true,
"objectValue": {
"foo.bar": "baz"
}
},
"proj:outer": {
"value": "{\"inner\":\"value\"}",
"secret": false,
"object": true,
"objectValue": {
"inner": "value"
}
},
"proj:servers": {
"value": "[{\"port\":80}]",
"secret": false,
"object": true,
"objectValue": [
{
"port": 80
}
]
},
"proj:token": {
"secret": true,
"object": false
},
"proj:tokens": {
"secret": true,
"object": true
}
}
```
If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.
If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.
**Accessing config values from Pulumi programs**
Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:
```typescript
import * as pulumi from "@pulumi/pulumi";
interface Server {
port: number;
}
const config = new pulumi.Config();
const names = config.requireObject<string[]>("names");
for (const n of names) {
console.log(n);
}
const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
console.log(s.port);
}
```
If any templates are marked as `Important: true` then by default show only those templates along with an option to see additional templates.
Fixes#3094.
The linter had been warning me for a while that some comments we had
confused it. I fixed this. Then the linter found a place where we
were ignoring a return value. Looking at it, it feels like we want to
continue in this case, so I just `contract.IgnoreError`'d it.
Validate the value is well formed much earlier so you don't end up
seeing you've picked a bad value in the middle of trying to create
your new stack. Update the helptext to list currently supported
values.
Fixes#2727
This change allows using the passphrase secrets manager when creating
a stack managed by the Pulumi service. `pulumi stack init`, `pulumi
new` and `pulumi up` all learned a new optional argument
`--secrets-provider` which can be set to "passphrase" to force the
passphrase based secrets provider to be used. When unset the default
secrets provider is used based on the backend (for local stacks this
is passphrase, for remote stacks, it is the key managed by the pulumi
service).
As part of this change, we also initialize the secrets manager when a
stack is created, instead of waiting for the first time a secret
config value is stored. We do this so that if an update is run using
`pulumi.secret` before any secret configuration values are used, we
already have the correct encryption method selected for a stack.
As part of the pluggable secrets work, the crypter's used for secrets
are no longer tied to a backend. To enforce this, we remove the
`backend.GetStackCrypter` function and then have the relevent logic to
construct one live inside the CLI itself.
Right now the CLI still uses the backend type to decide what Crypter
to build, but we'll change that shortly.
Right now, when we run `npm install` as part of `pulumi new`, the
following warnings are emitted:
```
node-pre-gyp WARN Using needle for node-pre-gyp https download
...
npm WARN aws-typescript@ No description
npm WARN aws-typescript@ No repository field.
npm WARN aws-typescript@ No license field.
```
We can suppress these warnings by specifying `--loglevel=error` to the
`npm install` command.
Originally, `pulumi new` did not run `up` after generating a project. To
help users get a deployed stack as quickly as possible, we changed
`pulumi new` to run an initial deployment at the end of its operation.
Users would first see a preview and get to decide whether to proceed
with an initial deployment, and then continue to iterate from there.
Note that this would only happen for nodejs projects
(TypeScript/JavaScript). We would not run `up` for Python projects as we
require the user to run `pip install` in a virtualenv, so we'd print
instructions with the necessary commands the user must run instead.
Running `up` as part of `pulumi new` for nodejs projects has ended up
being more confusing than helpful for new users, and annoying for
experienced users. New users aren't expecting `pulumi new` to run an
initial deployment after generating the project (they haven't even
looked at the project source yet). Experienced users find it frustrating
as you typically want to just generate the project, and don't want to
have to wait for the preview just to decline running the update.
This change reverts `pulumi new` back to not running `up` automatically
for nodejs projects. Instead, like with Python projects, at the end of
the operation, we print instructions to the user to run `pulumi up` to
deploy the project.
When stack names had to be unique across an entire organization, we
had a convention that stack names would be prefixed by the project
name, to prevent collisions.
Now that stack names are scoped within a project, we no longer need to
include the project name in the stack name. So when running `pulumi
new` to scaffhold a new project, just recommend the name `dev` for the
stack to create instead of `<project-name>-dev`.
Fixes#1417
Some changes to `pulumi new` to improve the experience:
- Color the default values for config differently to make them stand
out better
- Mention that `new` will also perform an initial deployment
- Add more vertical whitespace between steps in the process
- Print message indicating the "Installing dependencies" step is
complete
- After "project is ready to go", add a note about doing an initial
deployment
- Output follow-up command to run when an update fails
- Go back to showing the `npm install` output as `npm` doesn't always
return an error code when it runs into problems
- Ensure new projects have a project name in line with what we'd like
to enforce going forward
- Do more aggresive validation during the interactive prompts during
`pulumi new`
- Fix an issue where the interactive prompt rendered weridly when
there was a validation error
Contributes to #1988Fixes#1441
We currently leave behind the template section inside Pulumi.yaml after
`pulumi new`. While we may eventually make use of it, we're not
currently using it, so it's cleaner to just remove it for now.
This option allows the user to override the file used to fetch and store
configuration information for a stack. It is available for the config,
destroy, logs, preview, refresh, and up commands.
Note that this option is not persistent: if it is not specified, the
stack's default configuration will be used. If an alternate config file
is used exclusively for a stack, it must be specified to all commands
that interact with that stack.
This option can be used to share plaintext configuration across multiple
stacks. It cannot be used to share secret configuration, as secrets are
associated with a particular stack and cannot be decryptex by other
stacks.
* Don't attempt to install packages for Python new
Global installation of packages is almost always not what a user will
want when running 'pulumi new'. This commit instead prints out the
commands that a user should run in order to create a new virtualenv and
install the required Pulumi packages within it.
* CR feedback
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
`pulumi up <arg>` does not currently support template names like `pulumi
new`; `up` is hard-coded to only support URLs to templates. This
prevents us from displaying shorter `$ pulumi up ...` commands in the
Pulumi Console when the URL is to one of our standard Pulumi templates.
In such cases, we'd like to be able to show the command with just the
template name instead of a full URL to the template in the
pulumi/templates repo.
This changes `up` to support the standard Pulumi template names
just like `new`.
Also, while making changes here, if a URL specified to `up` contains
multiple templates in subdirectories, allow the user to choose the
template, just like with `new`.
Whenever we need to display a template description, if the Pulumi.yaml
doesn't have it, but has a project description, just use the project
description. This will allow us to avoid having the same description for
both the project and template in our examples.
The new command shouldn't show you the "this command will walk you
through" prelude when using the `-g` command -- it's not helpful and
generally looks confusing.
In preparation for some workspace restructuring, I decided to scratch a
few itches of my own in the code:
* Change project's RuntimeInfo field to just Runtime, to match the
serialized name in JSON/YAML.
* Eliminate the no-longer-used Context and NoDefaultIgnores fields on
project, and all of the associated legacy PPC-related code.
* Eliminate the no-longer-used IgnoreFile constant.
* Remove a bunch of "// nolint: lll" annotations, and simply format
the structures with comments on dedicated lines, to avoid overly
lengthy lines and lint suppressions.
* Mark Dependencies and InitErrors as `omitempty` in the JSON
serialization directives for CheckpointV2 files. This was done for
the YAML directives, but (presumably accidentally) omitted for JSON.
This adds an option, --suppress-outputs, to many commands, to
avoid printing stack outputs for stacks that might contain sensitive
information in their outputs (like key material, and whatnot).
Fixespulumi/pulumi#2028.
Right now, we only support --non-interactive in a few places (up,
refresh, destroy, etc). Over time, we've added it to more (like new).
And now, as we're working on better Docker support (pulumi/pulumi#1991),
we want to support this more globally, so we can, for example, avoid
popping up a web browser inside a Docker contain for logging in.
So, this change makes --non-interactive a global flag. Because it is
a persistent flag, it still works in the old positions, so this isn't
a breaking change to existing commands that use it.
Previously `new` was operating under the assumption that it was always
going to be creating a new project/stack, and would always prompt for
these values. However, we want to be able to use `new` to pull down the
source for an existing stack. This change adds a `--stack` flag to `new`
that can be used to specify an existing stack. If the specified stack
already exists, we won't prompt for the project name/description, and
instead just use the existing stack's values. If `--stack` is specified,
but doesn't already exist, it will just use that as the stack name
(instead of prompting) when creating the stack. `new` also now handles
configuration like `up <url>`: if the stack is a preconfigured empty
stack (i.e. it was created/configured in the Pulumi Console via Pulumi
Button or New Project), we will use the existing stack's config without
prompting. Otherwise we will prompt for config, and just like `up
<url>`, we'll use the existing stack's config values as defaults when
prompting, or if the stack didn't exist, use the defaults from the
template.
Previously `up <url>`'s handling of the project name/description wasn't
correct: it would always automatically use the values from the template
without prompting. Now, just like `new`:
- When updating an existing stack, it will not prompt, and just use the
existing stack's values.
- When creating a new stack, it will prompt for the project
name/description, using the defaults from the template.
This PR consolidates some of the `new`/`up` implementation so it shares
code for this functionality. There's definitely opportunities for a lot
more code reuse, but that cleanup can happen down the line if/when we
have the cycles.
We generally want examples and apps to be authored such that they are
clonable/deployable as-is without using new/up (and want to
encourage this). That means no longer using the ${PROJECT} and
${DESCRIPTION} replacement strings in Pulumi.yaml and other text files.
Instead, good default project names and descriptions should be specified
in Pulumi.yaml and elsewhere.
We'll use the specified values as defaults when prompting the user, and
then directly serialize/save the values to Pulumi.yaml when configuring
the user's project. This does mean that name in package.json (for nodejs
projects) won't be updated if it isn't using ${PROJECT}, but that's OK.
Our templates in the pulumi/templates repo will still use
${PROJECT}/${DESCRIPTION} for now, to continue to work well with v0.15
of the CLI. After that version is no longer in use, we can update the
templates to no longer use the replacement strings and delete the code
in the CLI that deals with it.
This change implements the same preview behavior we have for
cloud stacks, in pkg/backend/httpbe, for local stacks, in
pkg/backend/filebe. This mostly required just refactoring bits
and pieces so that we can share more of the code, although it
does still entail quite a bit of redundancy. In particular, the
apply functions for both backends are now so close to being
unified, but still require enough custom logic that it warrants
keeping them separate (for now...)
This simply refactors all the display logic out of the
pkg/backend/filestate package. This helps to gear us up to better unify
this logic between the filestate and httpstate backends.
Furthermore, this really ought to be in its own non-backend,
CLI-specific package, but I'm taking one step at a time here.
* Fix new's auto-up to display things interactively
This change fixes `new` to check whether things should be done
interactively, and passes the information along when auto running `up`
so that the standard interactive output is displayed.
When running non-interactively, we'll now auto-accept all prompts as if
`--yes` was passed.
* Add --non-interactive flag
Previously, we only supported config keys that included a ':' delimiter
in config keys specified in the template manifest and in `-c` flags to
`new` and `up`. This prevented the use of project keys in the template
manifest and made it more difficult to pass such keys with `-c`,
effectively preventing the use of `new pulumi.Config()` in project code.
This change fixes this by allowing config keys that don't have a
delimiter in the template manifest and `-c` flags. In such cases, the
project name is automatically prepended behind the scenes, the same as
what `pulumi config set` does.
We already walk through creating a stack and prompting for required
config, and then tell the user to run `pulumi up` to do an initial
deployment. Instead, just proceed with the `up` automatically.
This allows us to get rid of the `mkdir <dir> && cd <dir>` instructions in all our tutorials before `pulumi new`, because anyone who runs `pulumi new` in a non-empty directory will be forced to create a new directory in order to proceed.
