We require configuration to preform updates (as well as previews,
destroys and refreshes). Because of how everything evolved, loading
this configuration (and finding the coresponding decrypter) was
implemented in both the file and http backends, which wasn't great.
Refactor things such that the CLI itself builds out this information
and passes it along to the backend to preform operations. This means
less code duplicated between backends and less places the backend
assume things about the existence of `Pulumi.yaml` files and in
general makes the interface more plesent to use for others uses.
For cloud backed stacks, this was already returning nil and due to the
fact that we no longer include config in the checkpoint for local
stacks, it was nil there as well.
Removing this helps clean stuff up and is should make some future
refactorings around custom secret managers easier to land.
We can always add it back later if we miss it (and make it actually do
the right thing!)
When constructing a Deployment (which is a plaintext representation of
a Snapshot), ensure that we encrypt secret values. To do so, we
introduce a new type `secrets.Manager` which is able to encrypt and
decrypt values. In addition, it is able to reflect information about
itself that can be stored in the deployment such that we can
deserialize the deployment into a snapshot (decrypting the values in
the process) without external knowledge about how it was encrypted.
The ability to do this is import for allowing stack references to
work, since two stacks may not use the same manager (or they will use
the same type of manager, but have different state).
The state value is stored in plaintext in the deployment, so it **must
not** contain sensitive data.
A sample manager, which just base64 encodes and decodes strings is
provided, as it useful for testing. We will allow it to be varried
soon.
* Add a var for PRNumber. Add an environment metadata key for PR number.
* Move the detection of PULUMI_CI_SYSTEM into vars.DetectVars(). Set the PRNumber CI property based on respective env vars from each CI system.
* Add Azure Pipelines build variables.
* Add tests for DetectVars.
* Added changelog entry for Azure Pipelines.
* Capture the value of env var being modified for the ciutil unit test, and restore their values at the end of them.
* Simplify the DetectVars function by moving the Pulumi CI system code into the switch-case expression.
* Rename the Pulumi CI system to Generic CI. Include the GenericCI system in the test case for DetectVars.
We changed the `pulumi update` command to be `pulumi up` a while back
(`update` is an alias of `up`). This change just makes it so we refer to
the actual command, `pulumi up`, instead of the older `pulumi update`.
This commit implements the `pulumi query` as a dedicated command in the
CLI. The semantics of this command are to run a program in "query mode",
which disallows all resource operations, but allows access to stack
snapshots, as well as various query primitives.
By default `query` will run the query program in the currend directory,
but as with `up`, you can us the `--cwd` or `-C` flags to customize
this.
The Pulumi CLI currently has two "display modes" -- one for rendering
diffs and one for rendering program updates. Neither of these is
particularly well-suited to `pulumi query`, which essentially needs to
render stdout from the query program verbatim.
This commit will add a separate display mode for this purpose:
* In interactive mode, `pulumi query` will use the display spinner to
say "running query". In non-interactive mode, this will be omitted.
* Query mode will print stdout from the query program verbatim, but
ignore `diag.Infoerr` so that they're not double-printed when they're
emitted again as error events.
* Query mode also does not double-print newlines at the end of diag
events.
Right now, when we run `npm install` as part of `pulumi new`, the
following warnings are emitted:
```
node-pre-gyp WARN Using needle for node-pre-gyp https download
...
npm WARN aws-typescript@ No description
npm WARN aws-typescript@ No repository field.
npm WARN aws-typescript@ No license field.
```
We can suppress these warnings by specifying `--loglevel=error` to the
`npm install` command.
For users in secure environments without internet access the update
check in pulumi causes a significant hitch on running any pulumi
command, as pulumi tries to access pulumi.com to get the latest version
and after a while times out.
This commit adds an envvar (PULUMI_SKIP_UPDATE_CHECK) that if set to "1"
or "true" will cause pulumi to skip the update check.
This change adds a --json flag to the preview command, enabling
basic JSON serialization of preview plans. This effectively flattens
the engine event stream into a preview structure that contains a list
of steps, diagnostics, and summary information. Each step contains
the deep serialization of resource state, in addition to metadata about
the step, such as what kind of operation it entails.
This is a partial implementation of pulumi/pulumi#2390. In particular,
we only support --json on the `preview` command itself, and not `up`,
meaning that it isn't possible to serialize the result of an actual
deployment yet (thereby limiting what you can do with outputs, etc).
Use `result.Result` in more places, so when a confirmation prompt is
declined, we just return `result.Bail()` after printing a message
without the `error: ` prefix.
Fixes#2070
`pulumi stack rename` allows you to change the name of an existing
stack. This operation is non-distructive, however it is possible that
the next update will show additional changes to resources, if the
pulumi program uses the value of `getStack()` as part of a resource
name.
Originally, `pulumi new` did not run `up` after generating a project. To
help users get a deployed stack as quickly as possible, we changed
`pulumi new` to run an initial deployment at the end of its operation.
Users would first see a preview and get to decide whether to proceed
with an initial deployment, and then continue to iterate from there.
Note that this would only happen for nodejs projects
(TypeScript/JavaScript). We would not run `up` for Python projects as we
require the user to run `pip install` in a virtualenv, so we'd print
instructions with the necessary commands the user must run instead.
Running `up` as part of `pulumi new` for nodejs projects has ended up
being more confusing than helpful for new users, and annoying for
experienced users. New users aren't expecting `pulumi new` to run an
initial deployment after generating the project (they haven't even
looked at the project source yet). Experienced users find it frustrating
as you typically want to just generate the project, and don't want to
have to wait for the preview just to decline running the update.
This change reverts `pulumi new` back to not running `up` automatically
for nodejs projects. Instead, like with Python projects, at the end of
the operation, we print instructions to the user to run `pulumi up` to
deploy the project.
Show `brew upgrade pulumi` as the upgrade message when the currently
running pulumi executable is on macOS and running from the brew install
directory.
When stack names had to be unique across an entire organization, we
had a convention that stack names would be prefixed by the project
name, to prevent collisions.
Now that stack names are scoped within a project, we no longer need to
include the project name in the stack name. So when running `pulumi
new` to scaffhold a new project, just recommend the name `dev` for the
stack to create instead of `<project-name>-dev`.
Fixes#1417
Some changes to `pulumi new` to improve the experience:
- Color the default values for config differently to make them stand
out better
- Mention that `new` will also perform an initial deployment
- Add more vertical whitespace between steps in the process
- Print message indicating the "Installing dependencies" step is
complete
- After "project is ready to go", add a note about doing an initial
deployment
- Output follow-up command to run when an update fails
- Go back to showing the `npm install` output as `npm` doesn't always
return an error code when it runs into problems
We haven't shipped this yet, and it feels like returning a dictionary
where the keys are config keys and the value is an object that tells
you if it is a secret or not (and its value if it is secret and you
have passed `--show-secrets`) is going to be better overall.
This supports using `--json` to get configuration information in a
structured way.
The objects we return have the following schema:
```
{
name: string;
value: string?;
secret: bool;
}
```
In the case of `pulumi config` when --show-secrets is not passed, and
there are secret values, the `value` property of the object for that
configuration value will not be set. This differs from the normal
rendering where we show `[secret]`.
Contributes To #496
- Ensure new projects have a project name in line with what we'd like
to enforce going forward
- Do more aggresive validation during the interactive prompts during
`pulumi new`
- Fix an issue where the interactive prompt rendered weridly when
there was a validation error
Contributes to #1988Fixes#1441
This reverts commit fe6567a9a7.
We've decided to change get.pulumi.com to not require this, so
reverting the change to make the already long command no longer than
needed.
We currently leave behind the template section inside Pulumi.yaml after
`pulumi new`. While we may eventually make use of it, we're not
currently using it, so it's cleaner to just remove it for now.
We upgraded get.pulumi.com to require TLS1.2. Because powershell does
not support this out of the box, we had to augment the install command
to also enable support for TLS1.2. While we did this on pulumi.io in
our documentation, we also print a similar message on Windows when the
CLI is out of date.
This change augments the message to include enabiling TLS 1.2 during
the invocation.
* Initial stack history command
* Adding use of color pkg, adding background colors to color pkg, and removing extra stack output
* gofmt-ed colors file
* Fixing format and removing JSON output
* Fixing nits, changing output for environment, and adding some tests
* fixing failing history test
As of this change, all of the stack specific commands for `pulumi` now
allow passing `--stack` to operate on a different stack from the
default one.
Fixes#1648
Before the windows console will understand ANSI colorization codes the
terminal must be put in a special mode by passing
ENABLE_VIRTUAL_TERMINAL_PROCESSING to SetConsoleMode. This was
happening as a side effect of term.StdStreams() from a docker package,
which we did before displaying the update data. However, any
colorization done before that call would just show the raw ANSI escape
codes.
Call this helper much earlier (i.e. as soon as the CLI starts up) so
any messages that we print will have the correct colorization applied
to them.
Fixes#2214
This option allows the user to override the file used to fetch and store
configuration information for a stack. It is available for the config,
destroy, logs, preview, refresh, and up commands.
Note that this option is not persistent: if it is not specified, the
stack's default configuration will be used. If an alternate config file
is used exclusively for a stack, it must be specified to all commands
that interact with that stack.
This option can be used to share plaintext configuration across multiple
stacks. It cannot be used to share secret configuration, as secrets are
associated with a particular stack and cannot be decryptex by other
stacks.
* Don't attempt to install packages for Python new
Global installation of packages is almost always not what a user will
want when running 'pulumi new'. This commit instead prints out the
commands that a user should run in order to create a new virtualenv and
install the required Pulumi packages within it.
* CR feedback
When an update is in progress, `pulumi stack ls` was showing the LAST
UPDATE time as "a long while ago" because the service API returns 0 as
the last update time.
Handle this case correctly, displaying "in progress" for the update
time. When using JSON output, we don't include the update time (just
like a stack that has never been updated) but we do set the
`updateInProgress` property of the returned object
Fixes#2042
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
`pulumi up <arg>` does not currently support template names like `pulumi
new`; `up` is hard-coded to only support URLs to templates. This
prevents us from displaying shorter `$ pulumi up ...` commands in the
Pulumi Console when the URL is to one of our standard Pulumi templates.
In such cases, we'd like to be able to show the command with just the
template name instead of a full URL to the template in the
pulumi/templates repo.
This changes `up` to support the standard Pulumi template names
just like `new`.
Also, while making changes here, if a URL specified to `up` contains
multiple templates in subdirectories, allow the user to choose the
template, just like with `new`.
Whenever we need to display a template description, if the Pulumi.yaml
doesn't have it, but has a project description, just use the project
description. This will allow us to avoid having the same description for
both the project and template in our examples.
The new command shouldn't show you the "this command will walk you
through" prelude when using the `-g` command -- it's not helpful and
generally looks confusing.
In preparation for some workspace restructuring, I decided to scratch a
few itches of my own in the code:
* Change project's RuntimeInfo field to just Runtime, to match the
serialized name in JSON/YAML.
* Eliminate the no-longer-used Context and NoDefaultIgnores fields on
project, and all of the associated legacy PPC-related code.
* Eliminate the no-longer-used IgnoreFile constant.
* Remove a bunch of "// nolint: lll" annotations, and simply format
the structures with comments on dedicated lines, to avoid overly
lengthy lines and lint suppressions.
* Mark Dependencies and InitErrors as `omitempty` in the JSON
serialization directives for CheckpointV2 files. This was done for
the YAML directives, but (presumably accidentally) omitted for JSON.
When outputing JSON, if we have a fixed number of log entries (i.e. we
are not `--follow`'ing, we wrap each entry in array. Otherwise, we
just emit each log entry as an object at top level.
As part of this change, I've adopted a slightly more precise time
output format in `pulumi stack ls` when using JSON output. These times
now match the default output from `console.log(new Date())`
Downlevel versions of the Pulumi Node SDK assumed that a parallelism
level of zero implied serial execution, which current CLIs use to signal
unbounded parallelism. This commit works around the downlevel issue by
using math.MaxInt32 to signal unbounded parallelism.
In the past, we had a mode where the CLI would upload the Pulumi
program, as well as its contents and do the execution remotely.
We've since stopped supporting that, but all the supporting code has
been left in the CLI.
This change removes the code we had to support the above case,
including the `pulumi archive` command, which was a debugging tool to
generate the archive we would have uploaded (which was helpful in the
past to understand why behavior differed between local execution and
remote execution.)
We used to issue an error when you ran `pulumi stack output` for a
stack with with no root resource (e.g. one that hadn't been stood up
yet or one that had been stood up but then destoryed).
Instead, just follow the normal case for a stack that has a root
resource, but has no outputs.
Fixes#2016
* Remove TODO for issue since fixed in PPCs.
* Update issue reference to source
* Update comment wording
* Remove --ppc arg of stack init
* Remove PPC references in int. testing fx
* Remove vestigial PPC API types
* Introduce new metadata keys `vcs.repo`, `vcs.kind` and `vcs.owner` to keep the keys generic for any vcs. Expanded the git SSH regex to account for bitbucket's .org domain.
* Introduce new stack tags keys with the same theme of detecting the vcs.
Some providers (namely Kubernetes) require unbounded parallelism in
order to function correctly. This commit enables the engine to operate
in a mode with unbounded parallelism and switches to that mode by
default.
* Add new 'pulumi state' command for editing state
This commit adds 'pulumi state unprotect' and 'pulumi state delete', two
commands that can be used to unprotect and delete resources from a
stack's state, respectively.
* Simplify LocateResource
* CR: Print yellow 'warning' before editing state
* Lots of CR feedback
* CR: Only delete protected resources when asked with --force
This adds an option, --suppress-outputs, to many commands, to
avoid printing stack outputs for stacks that might contain sensitive
information in their outputs (like key material, and whatnot).
Fixespulumi/pulumi#2028.
This change adds GitLab CI support, by sniffing out the right
variables (equivalent to what we already do for Travis).
I've also restructured the code to share more logic with our
existing CI detection code, now moved to the pkg/util/ciutil
package, and will be fleshing this out more in the days to come.
There is a seldom-used capability in our CLI, the ability to pass
-m to specify an update message, which we will then show prominently.
At the same time, we already scrape some interesting information from
the Git repo from which an update is performed, like the SHA hash,
committer, and author information. We explicitly didn't want to scrape
the entire message just in case someone put sensitive info inside of it.
It seems safe -- indeed, appealing -- to use just the title portion
as the default update message when no other has been provided (the
majority case). We'll work on displaying it in a better way, but this
strengthens our GitOps/CI/CD story.
Fixespulumi/pulumi#2008.
It is easy to continually type `pulumi apply` from years of muscle memory using
Terraform - this commit makes the CLI suggest `up` when the `apply` subcommand
is used, similar to how `plan` suggests `preview`.
Right now, we only support --non-interactive in a few places (up,
refresh, destroy, etc). Over time, we've added it to more (like new).
And now, as we're working on better Docker support (pulumi/pulumi#1991),
we want to support this more globally, so we can, for example, avoid
popping up a web browser inside a Docker contain for logging in.
So, this change makes --non-interactive a global flag. Because it is
a persistent flag, it still works in the old positions, so this isn't
a breaking change to existing commands that use it.
This change adds a --json (short -j) flag for `pulumi stack output`
that prints the results as JSON, rather than our ad-hoc format.
Fixespulumi/pulumi#1863.
These commands ought to work even when you don't have a Pulumi.yaml:
$ pulumi stack ls --all
$ pulumi stack rm some-random-stack
They didn't previously, now they do. This fixespulumi/pulumi#1556.
API calls agains the Pulumi service may start setting a new header,
`X-Pulumi-Warning`. The value of this header should be presented to
the user as a warning.
The Service will use this to provide additional information to the
user without having the CLI have to know about every specific warning
path.
## Why ?
I'm using Zsh (and I'm not the only one 🤣). Pulumi having Zsh completions is great. I will also add completions to the Homebrew Formula when this is merged.
## Why not use Cobra `GenZshCompletion`
It's currently not good enough. Maybe it will be when spf13/cobra#646 is done.
## Implementation
I did the same thing `kubectl` does for Zsh completion. Meaning using the bash completion generated by Cobra and adapting it to a zsh format. The resulting zsh completion file is not perfect (compared to one's where you have a short command description in the output) but it's good enough I think.
I also changed the file output to a stdout output. I think it's better than outputting to a file and it will make adding completions in Homebrew straightforward. I don't know if the previous `gen-bash-completion` is used in any Pulumi project so this may break things.
If you run an operation that requires a stack, but you don't have
one selected, you'll be prompted. This happens all over the place.
Sadly, your selection at this prompt is not remembered (unless you
opt to create a new one), meaning you'll just keep getting prompted.
The fix is simple: we just ignored the setCurrent bool previously;
we need to respect it and call the SetCurrentStack function.
This fixespulumi/pulumi#1831.
This commit adds checks for a set of predefined environment variables:
- PULUMI_CI_SYSTEM
- PULUMI_CI_BUILD_ID
- PULUMI_CI_BUILD_TYPE
- PULUMI_CI_BUILD_URL
- PULUMI_CI_PULL_REQUEST_SHA
If PULUMI_CI_SYSTEM is set in the environment, CI configuration is
extracted from the remaining variables for sending to the backend, and
disables the checks for supported systems (currently only Travis CI).
This increases the flexibility of the Pulumi CLI by not requiring
specific support for particular CI systems to be added, provided the
necessary environment variables are configured for the job - this should
be possible for at least TeamCity, Jenkins, AWS CodeBuild, Azure DevOps
Pipelines, and likely most other systems.
This should not replace native support for detecting more CI systems in
future, however, since it requires more work of the user.
* Have backend.ListStacks return a new StackSummary interface
* Update filestake backend to use new type
* Update httpstate backend to use new type
* Update commands to use new type
* lint
* Address PR feedback
* Lint
* Close cancellation source before closing events
The cancellation source logs cancellation messages to the engine event
channel, so we must first close the cancellation source before closing
the channel.
* CR: Fix race in shutdown of signal goroutine
This change improves the root command help text for the CLI. It
advertises common commands and includes a more prominent link to
our project website. Fixespulumi/pulumi#1652.
Previously `new` was operating under the assumption that it was always
going to be creating a new project/stack, and would always prompt for
these values. However, we want to be able to use `new` to pull down the
source for an existing stack. This change adds a `--stack` flag to `new`
that can be used to specify an existing stack. If the specified stack
already exists, we won't prompt for the project name/description, and
instead just use the existing stack's values. If `--stack` is specified,
but doesn't already exist, it will just use that as the stack name
(instead of prompting) when creating the stack. `new` also now handles
configuration like `up <url>`: if the stack is a preconfigured empty
stack (i.e. it was created/configured in the Pulumi Console via Pulumi
Button or New Project), we will use the existing stack's config without
prompting. Otherwise we will prompt for config, and just like `up
<url>`, we'll use the existing stack's config values as defaults when
prompting, or if the stack didn't exist, use the defaults from the
template.
Previously `up <url>`'s handling of the project name/description wasn't
correct: it would always automatically use the values from the template
without prompting. Now, just like `new`:
- When updating an existing stack, it will not prompt, and just use the
existing stack's values.
- When creating a new stack, it will prompt for the project
name/description, using the defaults from the template.
This PR consolidates some of the `new`/`up` implementation so it shares
code for this functionality. There's definitely opportunities for a lot
more code reuse, but that cleanup can happen down the line if/when we
have the cycles.
We generally want examples and apps to be authored such that they are
clonable/deployable as-is without using new/up (and want to
encourage this). That means no longer using the ${PROJECT} and
${DESCRIPTION} replacement strings in Pulumi.yaml and other text files.
Instead, good default project names and descriptions should be specified
in Pulumi.yaml and elsewhere.
We'll use the specified values as defaults when prompting the user, and
then directly serialize/save the values to Pulumi.yaml when configuring
the user's project. This does mean that name in package.json (for nodejs
projects) won't be updated if it isn't using ${PROJECT}, but that's OK.
Our templates in the pulumi/templates repo will still use
${PROJECT}/${DESCRIPTION} for now, to continue to work well with v0.15
of the CLI. After that version is no longer in use, we can update the
templates to no longer use the replacement strings and delete the code
in the CLI that deals with it.
This change implements the same preview behavior we have for
cloud stacks, in pkg/backend/httpbe, for local stacks, in
pkg/backend/filebe. This mostly required just refactoring bits
and pieces so that we can share more of the code, although it
does still entail quite a bit of redundancy. In particular, the
apply functions for both backends are now so close to being
unified, but still require enough custom logic that it warrants
keeping them separate (for now...)
This simply refactors all the display logic out of the
pkg/backend/filestate package. This helps to gear us up to better unify
this logic between the filestate and httpstate backends.
Furthermore, this really ought to be in its own non-backend,
CLI-specific package, but I'm taking one step at a time here.
This change alters the login prompt slightly, so that it is more
obvious that alternative methods exist.
Before this change, we would say:
$ pulumi login
We need your Pulumi account to identify you.
Enter your access token from https://app.pulumi.com/account
or hit <ENTER> to log in using your browser :
After this change, we say this instead:
$ pulumi login
Manage your Pulumi stacks by logging in.
Run `pulumi login --help` for alternative login options.
Enter your access token from https://app.pulumi.com/account
or hit <ENTER> to log in using your browser :
Also updated the help text to advertise this a bit more prominently.
This renames the backend packages to more closely align with the
new direction for them. Namely, pkg/backend/cloud becomes
pkg/backend/httpstate and pkg/backend/local becomes
pkg/backend/filestate. This also helps to clarify that these are meant
to be around state management and so the upcoming refactoring required
to split out (e.g.) the display logic (amongst other things) will make
more sense, and we'll need better package names for those too.
As part of making the local backend more prominent, this changes a few
aspects of how you use it:
* Simplify how you log into a specific cloud; rather than
`pulumi login --cloud-url <url>`, just say `pulumi login <url>`.
* Use a proper URL scheme to denote local backend usage. We have chosen
file://, since the REST API backend is of course always https://.
This means that you can say `pulumi login file://~` to use the local
backend, with state files stored in your home directory. Similarly,
we support `pulumi login file://.` for the current directory.
* Add a --local flag to the login command, to make local logins a
bit easier in the common case of using your home directory. Just say
`pulumi login --local` and it is sugar for `pulumi login file://~`.
* Print the URL for the backend after logging in; for the cloud,
this is just the user's stacks page, and for the local backend,
this is the path to the user's stacks directory on disk.
* Tidy up the documentation for login a bit to be clearer about this.
This is part of pulumi/pulumi#1818.
* Fix new's auto-up to display things interactively
This change fixes `new` to check whether things should be done
interactively, and passes the information along when auto running `up`
so that the standard interactive output is displayed.
When running non-interactively, we'll now auto-accept all prompts as if
`--yes` was passed.
* Add --non-interactive flag
Previously, we were only saving config values specified on the command line (via --config/-c) for the URL case. This change fixes things to save these config values for the non-URL path.
This commit reverts most of #1853 and replaces it with functionally
identical logic, using the notion of status message-specific sinks.
In other words, where the original commit implemented ephemeral status
messages by adding an `isStatus` parameter to most of the logging
methdos in pulumi/pulumi, this implements ephemeral status messages as a
parallel logging sink, which emits _only_ ephemeral status messages.
The original commit message in that PR was:
> Allow log events to be marked "status" events
>
> This commit will introduce a field, IsStatus to LogRequest. A "status"
> logging event will be displayed in the Info column of the main
> display, but will not be printed out at the end, when resource
> operations complete.
>
> For example, for complex resource initialization, we'd like to display
> a series of intermediate results: [1/4] Service object created, for
> example. We'd like these to appear in the Info column, but not at the
> end, where they are not helpful to the user.
