It is easy to continually type `pulumi apply` from years of muscle memory using
Terraform - this commit makes the CLI suggest `up` when the `apply` subcommand
is used, similar to how `plan` suggests `preview`.
Right now, we only support --non-interactive in a few places (up,
refresh, destroy, etc). Over time, we've added it to more (like new).
And now, as we're working on better Docker support (pulumi/pulumi#1991),
we want to support this more globally, so we can, for example, avoid
popping up a web browser inside a Docker contain for logging in.
So, this change makes --non-interactive a global flag. Because it is
a persistent flag, it still works in the old positions, so this isn't
a breaking change to existing commands that use it.
This change adds a --json (short -j) flag for `pulumi stack output`
that prints the results as JSON, rather than our ad-hoc format.
Fixespulumi/pulumi#1863.
These commands ought to work even when you don't have a Pulumi.yaml:
$ pulumi stack ls --all
$ pulumi stack rm some-random-stack
They didn't previously, now they do. This fixespulumi/pulumi#1556.
API calls agains the Pulumi service may start setting a new header,
`X-Pulumi-Warning`. The value of this header should be presented to
the user as a warning.
The Service will use this to provide additional information to the
user without having the CLI have to know about every specific warning
path.
## Why ?
I'm using Zsh (and I'm not the only one 🤣). Pulumi having Zsh completions is great. I will also add completions to the Homebrew Formula when this is merged.
## Why not use Cobra `GenZshCompletion`
It's currently not good enough. Maybe it will be when spf13/cobra#646 is done.
## Implementation
I did the same thing `kubectl` does for Zsh completion. Meaning using the bash completion generated by Cobra and adapting it to a zsh format. The resulting zsh completion file is not perfect (compared to one's where you have a short command description in the output) but it's good enough I think.
I also changed the file output to a stdout output. I think it's better than outputting to a file and it will make adding completions in Homebrew straightforward. I don't know if the previous `gen-bash-completion` is used in any Pulumi project so this may break things.
If you run an operation that requires a stack, but you don't have
one selected, you'll be prompted. This happens all over the place.
Sadly, your selection at this prompt is not remembered (unless you
opt to create a new one), meaning you'll just keep getting prompted.
The fix is simple: we just ignored the setCurrent bool previously;
we need to respect it and call the SetCurrentStack function.
This fixespulumi/pulumi#1831.
This commit adds checks for a set of predefined environment variables:
- PULUMI_CI_SYSTEM
- PULUMI_CI_BUILD_ID
- PULUMI_CI_BUILD_TYPE
- PULUMI_CI_BUILD_URL
- PULUMI_CI_PULL_REQUEST_SHA
If PULUMI_CI_SYSTEM is set in the environment, CI configuration is
extracted from the remaining variables for sending to the backend, and
disables the checks for supported systems (currently only Travis CI).
This increases the flexibility of the Pulumi CLI by not requiring
specific support for particular CI systems to be added, provided the
necessary environment variables are configured for the job - this should
be possible for at least TeamCity, Jenkins, AWS CodeBuild, Azure DevOps
Pipelines, and likely most other systems.
This should not replace native support for detecting more CI systems in
future, however, since it requires more work of the user.
* Have backend.ListStacks return a new StackSummary interface
* Update filestake backend to use new type
* Update httpstate backend to use new type
* Update commands to use new type
* lint
* Address PR feedback
* Lint
* Close cancellation source before closing events
The cancellation source logs cancellation messages to the engine event
channel, so we must first close the cancellation source before closing
the channel.
* CR: Fix race in shutdown of signal goroutine
This change improves the root command help text for the CLI. It
advertises common commands and includes a more prominent link to
our project website. Fixespulumi/pulumi#1652.
Previously `new` was operating under the assumption that it was always
going to be creating a new project/stack, and would always prompt for
these values. However, we want to be able to use `new` to pull down the
source for an existing stack. This change adds a `--stack` flag to `new`
that can be used to specify an existing stack. If the specified stack
already exists, we won't prompt for the project name/description, and
instead just use the existing stack's values. If `--stack` is specified,
but doesn't already exist, it will just use that as the stack name
(instead of prompting) when creating the stack. `new` also now handles
configuration like `up <url>`: if the stack is a preconfigured empty
stack (i.e. it was created/configured in the Pulumi Console via Pulumi
Button or New Project), we will use the existing stack's config without
prompting. Otherwise we will prompt for config, and just like `up
<url>`, we'll use the existing stack's config values as defaults when
prompting, or if the stack didn't exist, use the defaults from the
template.
Previously `up <url>`'s handling of the project name/description wasn't
correct: it would always automatically use the values from the template
without prompting. Now, just like `new`:
- When updating an existing stack, it will not prompt, and just use the
existing stack's values.
- When creating a new stack, it will prompt for the project
name/description, using the defaults from the template.
This PR consolidates some of the `new`/`up` implementation so it shares
code for this functionality. There's definitely opportunities for a lot
more code reuse, but that cleanup can happen down the line if/when we
have the cycles.
We generally want examples and apps to be authored such that they are
clonable/deployable as-is without using new/up (and want to
encourage this). That means no longer using the ${PROJECT} and
${DESCRIPTION} replacement strings in Pulumi.yaml and other text files.
Instead, good default project names and descriptions should be specified
in Pulumi.yaml and elsewhere.
We'll use the specified values as defaults when prompting the user, and
then directly serialize/save the values to Pulumi.yaml when configuring
the user's project. This does mean that name in package.json (for nodejs
projects) won't be updated if it isn't using ${PROJECT}, but that's OK.
Our templates in the pulumi/templates repo will still use
${PROJECT}/${DESCRIPTION} for now, to continue to work well with v0.15
of the CLI. After that version is no longer in use, we can update the
templates to no longer use the replacement strings and delete the code
in the CLI that deals with it.
This change implements the same preview behavior we have for
cloud stacks, in pkg/backend/httpbe, for local stacks, in
pkg/backend/filebe. This mostly required just refactoring bits
and pieces so that we can share more of the code, although it
does still entail quite a bit of redundancy. In particular, the
apply functions for both backends are now so close to being
unified, but still require enough custom logic that it warrants
keeping them separate (for now...)
This simply refactors all the display logic out of the
pkg/backend/filestate package. This helps to gear us up to better unify
this logic between the filestate and httpstate backends.
Furthermore, this really ought to be in its own non-backend,
CLI-specific package, but I'm taking one step at a time here.
This change alters the login prompt slightly, so that it is more
obvious that alternative methods exist.
Before this change, we would say:
$ pulumi login
We need your Pulumi account to identify you.
Enter your access token from https://app.pulumi.com/account
or hit <ENTER> to log in using your browser :
After this change, we say this instead:
$ pulumi login
Manage your Pulumi stacks by logging in.
Run `pulumi login --help` for alternative login options.
Enter your access token from https://app.pulumi.com/account
or hit <ENTER> to log in using your browser :
Also updated the help text to advertise this a bit more prominently.
This renames the backend packages to more closely align with the
new direction for them. Namely, pkg/backend/cloud becomes
pkg/backend/httpstate and pkg/backend/local becomes
pkg/backend/filestate. This also helps to clarify that these are meant
to be around state management and so the upcoming refactoring required
to split out (e.g.) the display logic (amongst other things) will make
more sense, and we'll need better package names for those too.
As part of making the local backend more prominent, this changes a few
aspects of how you use it:
* Simplify how you log into a specific cloud; rather than
`pulumi login --cloud-url <url>`, just say `pulumi login <url>`.
* Use a proper URL scheme to denote local backend usage. We have chosen
file://, since the REST API backend is of course always https://.
This means that you can say `pulumi login file://~` to use the local
backend, with state files stored in your home directory. Similarly,
we support `pulumi login file://.` for the current directory.
* Add a --local flag to the login command, to make local logins a
bit easier in the common case of using your home directory. Just say
`pulumi login --local` and it is sugar for `pulumi login file://~`.
* Print the URL for the backend after logging in; for the cloud,
this is just the user's stacks page, and for the local backend,
this is the path to the user's stacks directory on disk.
* Tidy up the documentation for login a bit to be clearer about this.
This is part of pulumi/pulumi#1818.
* Fix new's auto-up to display things interactively
This change fixes `new` to check whether things should be done
interactively, and passes the information along when auto running `up`
so that the standard interactive output is displayed.
When running non-interactively, we'll now auto-accept all prompts as if
`--yes` was passed.
* Add --non-interactive flag
Previously, we were only saving config values specified on the command line (via --config/-c) for the URL case. This change fixes things to save these config values for the non-URL path.
This commit reverts most of #1853 and replaces it with functionally
identical logic, using the notion of status message-specific sinks.
In other words, where the original commit implemented ephemeral status
messages by adding an `isStatus` parameter to most of the logging
methdos in pulumi/pulumi, this implements ephemeral status messages as a
parallel logging sink, which emits _only_ ephemeral status messages.
The original commit message in that PR was:
> Allow log events to be marked "status" events
>
> This commit will introduce a field, IsStatus to LogRequest. A "status"
> logging event will be displayed in the Info column of the main
> display, but will not be printed out at the end, when resource
> operations complete.
>
> For example, for complex resource initialization, we'd like to display
> a series of intermediate results: [1/4] Service object created, for
> example. We'd like these to appear in the Info column, but not at the
> end, where they are not helpful to the user.
Previously, we only supported config keys that included a ':' delimiter
in config keys specified in the template manifest and in `-c` flags to
`new` and `up`. This prevented the use of project keys in the template
manifest and made it more difficult to pass such keys with `-c`,
effectively preventing the use of `new pulumi.Config()` in project code.
This change fixes this by allowing config keys that don't have a
delimiter in the template manifest and `-c` flags. In such cases, the
project name is automatically prepended behind the scenes, the same as
what `pulumi config set` does.
We already walk through creating a stack and prompting for required
config, and then tell the user to run `pulumi up` to do an initial
deployment. Instead, just proceed with the `up` automatically.
This allows us to get rid of the `mkdir <dir> && cd <dir>` instructions in all our tutorials before `pulumi new`, because anyone who runs `pulumi new` in a non-empty directory will be forced to create a new directory in order to proceed.
Replace the Source-based implementation of refresh with a phase that
runs as the first part of plan execution and rewrites the snapshot in-memory.
In order to fit neatly within the existing framework for resource operations,
these changes introduce a new kind of step, RefreshStep, to represent
refreshes. RefreshSteps operate similar to ReadSteps but do not imply that
the resource being read is not managed by Pulumi.
In addition to the refresh reimplementation, these changes incorporate those
from #1394 to run refresh in the integration test framework.
Fixes#1598.
Fixespulumi/pulumi-terraform#165.
Contributes to #1449.
The fact that we include the `:config:` portion of a configuration
name when sending to the service is an artifact of history. It was
needed back when we needed to support deploying using a new CLI
against a PPC that had older packages embeded in it.
We don't have that sort of problem anymore, so we can stop sending
this data.
* Show a better error message when decrypting fails
It is most often the case that failing to decrypt a secret implies that
the secret was transferred from one stack to another via copying the
configuration. This commit introduces a better error message for this
case and instructs users to explicitly re-encrypt their encrypted keys
in the context of the new stack.
* Spelling
* CR: Grammar fixes
The glog package force the use of golang's underyling flag package,
which Cobra does not use. To work around this, we had a complicated
dance around defining flags in multiple places, calling flag.Parse
explicitly and then stomping values in the flag package with values we
got from Cobra.
Because we ended up parsing parts of the command line twice, each with
a different set of semantics, we ended up with bad UX in some
cases. For example:
`$ pulumi -v=10 --logflow update`
Would fail with an error message that looked nothing like normal CLI
errors, where as:
`$ pulumi -v=10 update --logflow`
Would behave as you expect. To address this, we now do two things:
- We never call flag.Parse() anymore. Wacking the flags with values we
got from Cobra is sufficent for what we care about.
- We use a forked copy of glog which does not complain when
flag.Parse() is not called before logging.
Fixes#301Fixes#710Fixes#968
* Add a list of in-flight operations to the deployment
This commit augments 'DeploymentV2' with a list of operations that are
currently in flight. This information is used by the engine to keep
track of whether or not a particular deployment is in a valid state.
The SnapshotManager is responsible for inserting and removing operations
from the in-flight operation list. When the engine registers an intent
to perform an operation, SnapshotManager inserts an Operation into this
list and saves it to the snapshot. When an operation completes, the
SnapshotManager removes it from the snapshot. From this, the engine can
infer that if it ever sees a deployment with pending operations, the
Pulumi CLI must have crashed or otherwise abnormally terminated before
seeing whether or not an operation completed successfully.
To remedy this state, this commit also adds code to 'pulumi stack
import' that clears all pending operations from a deployment, as well as
code to plan generation that will reject any deployments that have
pending operations present.
At the CLI level, if we see that we are in a state where pending
operations were in-flight when the engine died, we'll issue a
human-friendly error message that indicates which resources are in a bad
state and how to recover their stack.
* CR: Multi-line string literals, renaming in-flight -> pending
* CR: Add enum to apitype for operation type, also name status -> type for clarity
* Fix the yaml type
* Fix missed renames
* Add implementation for lifecycle_test.go
* Rebase against master
* Initial support for passing URLs to `new` and `up`
This PR adds initial support for `pulumi new` using Git under the covers
to manage Pulumi templates, providing the same experience as before.
You can now also optionally pass a URL to a Git repository, e.g.
`pulumi new [<url>]`, including subdirectories within the repository,
and arbitrary branches, tags, or commits.
The following commands result in the same behavior from the user's
perspective:
- `pulumi new javascript`
- `pulumi new https://github.com/pulumi/templates/templates/javascript`
- `pulumi new https://github.com/pulumi/templates/tree/master/templates/javascript`
- `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates/javascript`
To specify an arbitrary branch, tag, or commit:
- `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates/javascript`
- `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates/javascript`
- `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates/javascript`
Branches and tags can include '/' separators, and `pulumi` will still
find the right subdirectory.
URLs to Gists are also supported, e.g.:
`pulumi new https://gist.github.com/justinvp/6673959ceb9d2ac5a14c6d536cb871a6`
If the specified subdirectory in the repository does not contain a
`Pulumi.yaml`, it will look for subdirectories within containing
`Pulumi.yaml` files, and prompt the user to choose a template, along the
lines of how `pulumi new` behaves when no template is specified.
The following commands result in the CLI prompting to choose a template:
- `pulumi new`
- `pulumi new https://github.com/pulumi/templates/templates`
- `pulumi new https://github.com/pulumi/templates/tree/master/templates`
- `pulumi new https://github.com/pulumi/templates/tree/HEAD/templates`
Of course, arbitrary branches, tags, or commits can be specified as well:
- `pulumi new https://github.com/pulumi/templates/tree/<branch>/templates`
- `pulumi new https://github.com/pulumi/templates/tree/<tag>/templates`
- `pulumi new https://github.com/pulumi/templates/tree/<commit>/templates`
This PR also includes initial support for passing URLs to `pulumi up`,
providing a streamlined way to deploy installable cloud applications
with Pulumi, without having to manage source code locally before doing
a deployment.
For example, `pulumi up https://github.com/justinvp/aws` can be used to
deploy a sample AWS app. The stack can be updated with different
versions, e.g.
`pulumi up https://github.com/justinvp/aws/tree/v2 -s <stack-to-update>`
Config values can optionally be passed via command line flags, e.g.
`pulumi up https://github.com/justinvp/aws -c aws:region=us-west-2 -c foo:bar=blah`
Gists can also be used, e.g.
`pulumi up https://gist.github.com/justinvp/62fde0463f243fcb49f5a7222e51bc76`
* Fix panic when hitting ^C from "choose template" prompt
* Add description to templates
When running `pulumi new` without specifying a template, include the template description along with the name in the "choose template" display.
```
$ pulumi new
Please choose a template:
aws-go A minimal AWS Go program
aws-javascript A minimal AWS JavaScript program
aws-python A minimal AWS Python program
aws-typescript A minimal AWS TypeScript program
> go A minimal Go program
hello-aws-javascript A simple AWS serverless JavaScript program
javascript A minimal JavaScript program
python A minimal Python program
typescript A minimal TypeScript program
```
* React to changes to the pulumi/templates repo.
We restructured the `pulumi/templates` repo to have all the templates in the root instead of in a `templates` subdirectory, so make the change here to no longer look for templates in `templates`.
This also fixes an issue around using `Depth: 1` that I found while testing this. When a named template is used, we attempt to clone or pull from the `pulumi/templates` repo to `~/.pulumi/templates`. Having it go in this well-known directory allows us to maintain previous behavior around allowing offline use of templates. If we use `Depth: 1` for the initial clone, it will fail when attempting to pull when there are updates to the remote repository. Unfortunately, there's no built-in `--unshallow` support in `go-git` and setting a larger `Depth` doesn't appear to help. There may be a workaround, but for now, if we're cloning the pulumi templates directory to `~/.pulumi/templates`, we won't use `Depth: 1`. For template URLs, we will continue to use `Depth: 1` as we clone those to a temp directory (which gets deleted) that we'll never try to update.
* List available templates in help text
* Address PR Feedback
* Don't show "Installing dependencies" message for `up`
* Fix secrets handling
When prompting for config, if the existing stack value is a secret, keep it a secret and mask the prompt. If the template says it should be secret, make it a secret.
* Fix ${PROJECT} and ${DESCRIPTION} handling for `up`
Templates used with `up` should already have a filled-in project name and description, but if it's a `new`-style template, that has `${PROJECT}` and/or `${DESCRIPTION}`, be helpful and just replace these with better values.
* Fix stack handling
Add a bool `setCurrent` param to `requireStack` to control whether the current stack should be saved in workspace settings. For the `up <url>` case, we don't want to save. Also, split the `up` code into two separate functions: one for the `up <url>` case and another for the normal `up` case where you have workspace in your current directory. While we may be able to combine them back into a single function, right now it's a bit cleaner being separate, even with some small amount of duplication.
* Fix panic due to nil crypter
Lazily get the crypter only if needed inside `promptForConfig`.
* Embellish comment
* Harden isPreconfiguredEmptyStack check
Fix the code to check to make sure the URL specified on the command line matches the URL stored in the `pulumi:template` config value, and that the rest of the config from the stack satisfies the config requirements of the template.
Many non-secrets are actually pretty high entropy, at least according
to `zxcvbn`. For example: "Hello, Pulumi Timers!" would actually cause
us to say: "this looks like a secret", much in the same way that
"correct horse battery staple" is high entropy according to that
package.
In addition to considering the entropy of the value, gosec (the linter
we copied this logic from) also considers the name of the value that
is being assigned to.
In that spirit, let's only do this check when the config key name
actually looks like it is something we'd want to warn the user about.
We use the same regular expression as `gosec`.
Fixes#1732
In #1341 we promoted a class of errors in fetching git metadata from
glog messages to warnings printed by the CLI. On the asumption that
when we got warnings here they would be actionable.
The major impact here is that when you are working in a repository
which does not have a remote set to GitHub (common if you have just
`git init`'d a repository for a new project) or you don't call your
remote `origin` or you use some other code provider, we end up
printing a warning during every update.
This change does two things:
- Restructure the way we detect metadata to attempt to make progress
when it can. We bias towards returning some metadata even when we
can't determine the complete set of metadata.
- Use a multierror to track all the underlying failures from our
metadata probing and move it back to a glog message.
Overall, this feels like the right balance to me. We are retaining the
rich diagnostics information for when things go wrong, but we aren't
warning about common cases.
We could, of course, try to tighten our huristics (e.g. don't warn if
we can't find a GitHub remote but do warn if we can't compute if the
worktree is dirty) but it feels like that will be a game of
whack-a-mole over time and when warnings do fire its unlikely they
will be actionable.
Fixes#1443
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
This change lets us set runtime specific options in Pulumi.yaml, which
will flow as arguments to the language hosts. We then teach the nodejs
host that when the `typescript` is set to `true` that it should load
ts-node before calling into user code. This allows using typescript
natively without an explicit compile step outside of Pulumi.
This works even when a tsconfig.json file is not present in the
application and should provide a nicer inner loop for folks writing
typescript (I'm pretty sure everyone has run into the "but I fixed
that bug! Why isn't it getting picked up? Oh, I forgot to run tsc"
problem.
Fixes#958
* Protobuf changes to record dependencies for read resources
* Add a number of tests for read resources, especially around replacement
* Place read resources in the snapshot with "external" bit set
Fixespulumi/pulumi#1521. This commit introduces two new step ops: Read
and ReadReplacement. The engine generates Read and ReadReplacement steps
when servicing ReadResource RPC calls from the language host.
* Fix an omission of OpReadReplace from the step list
* Rebase against master
* Transition to use V2 Resources by default
* Add a semantic "relinquish" operation to the engine
If the engine observes that a resource is read and also that the
resource exists in the snapshot as a non-external resource, it will not
delete the resource if the IDs of the old and new resources match.
* Typo fix
* CR: add missing comments, DeserializeDeployment -> DeserializeDeploymentV2, ID check
Previously, we would unconditionally warn anytime you added a non-secret
config:
$ pulumi config set aws:region us-west-2
warning: saved config key '%s' value '%s' as plaintext;
re-run with --secret to encrypt the value instead.
Use --plaintext to avoid this warning
This was particularly annoying, since it is very common to store
non-secret config. For instance, the AWS region. And it was easy to tune
out because it wasn't actually warning about anything interesting.
This change, which resolvespulumi/pulumi#570, uses an approach similar
to Go's gas linter, to detect high entropy values, and issue an error.
This ensures that we only make noise on things we suspect are actually
secrets being stored in plaintext, and forces the user to pass
--plaintext. For instance, the common case issues no errors:
$ pulumi config set aws:region us-west-2
And in the event that you store something that is secret-like:
$ pulumi config set aws:region nq8r4B4xslzrtj0a3
error: config value 'nq8r4B4xslzrtj0a3' looks like a secret;
rerun with --secret to encrypt it, or --plaintext if you meant
to store in plaintext
To suppress this, simply pass --secret (to encrypt) or --plaintext (to
override the warning).
After running `pulumi new`, we print a message to let the user
know the project has been created, along with next steps to actually
perform a deployment. It's easy for this to get lost among the rest
of the output, however. So, wordsmith it a little, and add some color,
to make it pop a little bit more.
Also add SuggestFor annotations so that `init` and `create` direct
you to run the `new` command (I often accidentally type `init`).
Previously, it would only show the first 7 templates (we currently have 9 total), and you'd have to move the cursor down to the bottom to show the last 2.
Instead of needing you to first select the current stack to use any of
these commands, allow passing `-s <stack-name>` or `--stack
<stack-name>` to say what stack you want to operate on.
These commands still require a `Pulumi.yaml` file to be present, which
is not ideal, but would require a larger refactoring to fix. That
refactoring will happen as part of #1556.
Fixes#1370
This was an artifact of history. Since we'll be supporting the local
backend, we don't need yet another flag guarding it (you already have
to opt in with -c local:// which is enough of a hoop).
This change fixes the following warning issued by using `go get` with
the Pulumi repository:
package github.com/moby/moby/api/types/time: code in directory
/Users/James/Code/go/src/github.com/moby/moby/api/types/time expects
import "github.com/docker/docker/api/types/time"
Moby defines canonical import paths [1] reflecting the lineage back to
the `docker/docker` repository.
[1]: https://github.com/moby/moby/blob/master/api/types/time/duration_convert.go#L1
Usage:
```
pulumi new <template> --dir folderName
```
Used to specify the directory where to place the generated project.
If the directory does not exist, it will be created.
* Error when loading a deployment that is not a version that the CLI understands
* Add a test for 'pulumi stack import' on a badly-versioned deployment
* Move current deployment version to 'apitype'
* Rebase against master
* CR: emit CLI-friendly error message at the two points outside of the engine calling 'DeserializeDeployment'
This change supports displaying manifest information for a stack and
changes the way we handle Snapshots in our backend.
Previously, every call to GetStack would synthesize a Snapshot by
taking the set of resources returned from the
`/api/stacks/<owner>/<name>` endpoint, combined with an empty
manfiest (since the service was not returning the manifest).
This wasn't great for two reasons:
1. We didn't have manifest information, so we couldn't display any of
its information (most important the last updated time).
2. This strategy required that the service return all the resources
for a stack anytime GetStack was called. While the CLI did not
often need this detailed information the fact that we forced the
Service to produce it (which in the case of stack managed PPC would
require the service to talk to yet another service) creates a bunch
of work that we end up ignoring.
I've refactored the code such that `backend.Stack`'s `Snapshot()` method
now lazily requests the information from the service such that we can
construct a `Snapshot()` on demand and only pay the cost when we
actually need it.
I think making more of this stuff lazy is the long term direction we
want to follow.
Unfortunately, right now, it means in cases where we do need this data
we end up fetching it twice. The service does it once when we call
GetStack and then we do it again when we actually need to get at the
Snapshot. However, once we land this change, we can update the
service to no longer return resources on the apistack.Stack type. The
CLI no longer needs this property. We'll likely want to continue in a
direction where `apistack.Stack` can be created quickly by the
service (without expensive database queries or fetching remote
resources) and just add additional endpoints that let us get at the
specific information we want in the specific cases when we want it
instead of forcing us to return a bunch of data that we often ignore.
Fixespulumi/pulumi-service#371
While we wait for upstream to fix the security issue, we'll just pass
`--no-audit` to `npm install` when creating a new project. Since the
warning is against an indirect dependency of @pulumi/pulumi, we can't
actually address the issue ourselves.
Mitigates #1350
In pulumi/pulumi#1356, we observed that we can fail during a destroy
because we attempt to load the language plugin, which now eagerly looks
for the @pulumi/pulumi package.
This is also blocking ingestion of the latest engine bits into the PPC.
It turns out that for destroy (and refresh), we have no need for the
language plugin. So, let's skip loading it when appropriate.
These changes add support for adding a tracing header to API requests
made to the Pulumi service. Setting the `PULUMI_TRACING_HEADER`
environment variable or enabling debug commands and passing the
`--tracing-header` will change the value sent in this header. Setting
this value to `1` will request that the service enable distributed
tracing for all requests made by a particular CLI invocation.
The newly added `pulumi config refresh` updates your local copy of the
Pulumi.<stack-name>.yaml file to have the same configuration as the
most recent deployment in the cloud.
This can be used in a varirty of ways. One place we plan to use it is
in automation to clean up "leaked" stacks we have in CI. With the
changes you'll now be able to do the following:
```
$ cd $(mktemp -d)
$ echo -e "name: who-cares\nruntime: nodejs" > Pulumi.yaml
$ pulumi stack select <leaked-stack-name>
$ pulumi config refresh -f
$ pulumi destroy --force
```
Having a simpler gesture for the above is something we'll want to do
long term (we should be able to support `pulumi destory <stack-name>`
from a completely empty folder, today you need a Pulumi.yaml file
present, even if the contents don't matter).
But this gets us a little closer to where we want to be and introduces
a helpful primitive in the system.
Contributes to #814
These changes add support for injecting client tracing spans into HTTP
requests to the Pulumi API. The server can then rematerialize these span
references and attach its own spans for distributed tracing.
This change moves Git failures from glog.Warnings, which we don't
really pay attention, to true CLI warnings.
This will ensure we at least get bug reports in the event that this
fails on some user machine out in the wild. They are still non-fatal,
of course, since such a failure needn't prevent an update from happening.
This change captures the Git committer and author's login and email
addresses, so that we can display them prominently in the service. At
the moment, we only attribute updates to the identity that performed the
update which, in CI scenarios, is often always the same person for an
organization. This makes Pulumi look like a needlessly lonely place.
These changes enable tracing of Pulumi API calls.
The span with which to associate an API call is passed via a
`context.Context` parameter. This required plumbing a
`context.Context` parameter through a rather large number of APIs,
especially in the backend.
In general, all API calls are associated with a new root span that
exists for essentially the entire lifetime of an invocation of the
Pulumi CLI. There were a few places where the plumbing got a bit hairier
than I was willing to address with these changes; I've used
`context.Background()` in these instances. API calls that receive this
context will create new root spans, but will still be traced.
* Initialize a new stack as part of `pulumi new`
* Prompt for values with defaults preselected
* Install dependencies
* Prompt for default config values
This changes the CLI interface in a few ways:
* `pulumi preview` is back! The alternative of saying
`pulumi update --preview` just felt awkward, and it's a common
operation to want to perform. Let's just make it work.
* There are two flags consistent across all update commands,
`update`, `refresh`, and `destroy`:
- `--skip-preview` will skip the preview step. Note that this
does *not* skip the prompt to confirm that you'd like to proceed.
Indeed, it will still prompt, with a little warning text about
the fact that the preview has been skipped.
* `--yes` will auto-approve the updates.
This lands us in a simpler and more intuitive spot for common scenarios.
I found the flag --force to be a strange name for skipping a preview,
since that name is usually reserved for operations that might be harmful
and yet you're coercing a tool to do it anyway, knowing there's a chance
you're going to shoot yourself in the foot.
I also found that what I almost always want in the situation where
--force was being used is to actually just run a preview and have the
confirmation auto-accepted. Going straight to --force isn't the right
thing in a CI scenario, where you actually want to run a preview first,
just to ensure there aren't any issues, before doing the update.
In a sense, there are four options here:
1. Run a preview, ask for confirmation, then do an update (the default).
2. Run a preview, auto-accept, and then do an update (the CI scenario).
3. Just run a preview with neither a confirmation nor an update (dry run).
4. Just do an update, without performing a preview beforehand (rare).
This change enables all four workflows in our CLI.
Rather than have an explosion of flags, we have a single flag,
--preview, which can specify the mode that we're operating in. The
following are the values which correlate to the above four modes:
1. "": default (no --preview specified)
2. "auto": auto-accept preview confirmation
3. "only": only run a preview, don't confirm or update
4. "skip": skip the preview altogether
As part of this change, I redid a bit of how the preview modes
were specified. Rather than booleans, which had some illegal
combinations, this change introduces a new enum type. Furthermore,
because the engine is wholly ignorant of these flags -- and only the
backend understands them -- it was confusing to me that
engine.UpdateOptions stored this flag, especially given that all
interesting engine options _also_ accepted a dryRun boolean. As of
this change, the backend.PreviewBehavior controls the preview options.
The `go-git` implementation of `git status` is outrageously expensive,
as it performs a hash-based comparision of the working tree against the
committed state with no caching. In some example runs, this takes
upwards of 15 seconds. Because this is on the startup path for updates,
this results in a rather poor user experience.
These changes replace the `go-git` implementation with a call to `git
status --porcelain -z`, which only writes data to stdout if the working
tree is dirty.
Note that these changes also make all git-related update metadata
best-effort.
These changes add a new flag to the CLI, `--profiling`, that enables
CPU and heap profiling as well as execution tracing of the CLI itself.
The argument to this flag serves as a prefix for the profile outputs;
the CPU and heap profiles and execution trace are written to
`[filename].[pid].{cpu,mem,trace}`, respectively.
These changes also fix an issue with `cmdutil.RunFunc` wherein any error
would prevent the command's post-run hooks from executing.
We already have a great history viewing experience on
Pulumi.com. `pulumi history` in the CLI today is basically unuseable,
and instead of working on trying to improve it, let's just remove it
for now.
Fixes#965
This change adds two new (hidden) CLI commands:
* `gen-bash-completion`: This command generates a bash completion
script for the CLI, storing it in the file specified by the 1st arg.
This fixespulumi/pulumi#1172.
* `gen-markdown`: This command generates a directory of Markdown files,
one per command, documenting the CLI commands and their usage.
I originally did these as separate scripts that we can use in our
build processes, but it was actually even easier to make `pulumi` able
to generate them for itself. The nice part about this is that we don't
even need to bundle additional assets in order to distribute e.g. the
bash completion scripts, we can simply tell people to run
$ pulumi gen-bash-completion /etc/bash_completion.d/pulumi
This can also be used in our upcoming Brew installer.
Pat ran into a weird error when trying to do some development agains
the testing cloud:
```
$ pulumi logout
$ pulumi login --cloud-url [test-cloud-url]
Logged into [test-cloud-url]
$ pulumi stack ls
Enter your Pulumi access token from https://pulumi.com/account:
```
In his case, we did not have `current` set in our credentials.json
file (likely due to him calling `pulumi logout` at some point) but we
did have stored credentials for that cloud. When he logged in the CLI
noticed we could reuse the stored credentials but did not update the
the current setting to set the current cloud.
While investigating, I also noticed that `logout` did not always do
the right thing when you were logged into a different backend than
pulumi.com
* Refactor the SnapshotManager interface
Lift snapshot management out of the engine by delegating it to the
SnapshotManager implementation in pkg/backend.
* Add a event interface for plugin loads and use that interface to record plugins in the snapshot
* Remove dead code
* Add comments to Events
* Add a number of tests for SnapshotManager
* CR feedback: use a successful bit on 'End' instead of having a separate 'Abort' API
* CR feedback
* CR feedback: register plugins one-at-a-time instead of the entire state at once
- Show Emojis on non-Windows platforms, instead of just macOS
- Change help text for `pulumi logs` to clarify the logs are specific
to a stack, not a project
- Display stack name when showing logs
- Have preamble text show to users for engine operations read a little
nicer
PPCs are no longer a central concept to our model, but instead a
feature that that pulumi.com provides to some organizations. Let's
remove most mentions of PPCs except for cases where we really need to
talk about them (e.g. when a stack is actually hosted in a PPC instead
of just via the normal pulumi.com service)
Also remove some "in the Pulumi Cloud" messages from the CLI, as using
the Pulumi Cloud is now the only real way to use pulumi.
Fixespulumi/pulumi-service#1117
hese changes plumb basic support for cancellation through the engine.
Two types of cancellation are supported for all engine operations:
- Cancellation, which waits for the operation to drive itself to a safe
point before the operation returns, and
- Termination, which does not wait for the operation to drive itself
to a safe opint for the operation returns.
When updating local or managed stacks, a single ^C triggers cancellation
of any running operation; a second ^C will trigger termination.
Fixes#513, #1077.
This command cancels a stack's currently running update, if any. It can
be used to recover from the scenario in which an update is aborted
without marking the running update as complete. Once an update has been
cancelled, it is likely that the affected stack will need to be repaired
via an pair of export/import commands before future updates can succeed.
This is part of #1077.
This change implements a `pulumi refresh` command. It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.
It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events. This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state. This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion). The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.
Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint. That
will be coming soon ...
This change introduces support for using the cloud backend when
`pulumi init` has not been run. When this is the case, we use the new
identity model, where a stack is referenced by an owner and a stack
name only.
There are a few things going on here:
- We add a new `--owner` flag to `pulumi stack init` that lets you
control what account a stack is created in.
- When listing stacks, we show stacks owned by you and any
organizations you are a member of. So, for example, I can do:
* `pulumi stack init my-great-stack`
* `pulumi stack init --owner pulumi my-great-stack`
To create a stack owned by my user and one owned by my
organization. When `pulumi stack ls` is run, you'll see both
stacks (since they are part of the same project).
- When spelling a stack on the CLI, an owner can be optionally
specified by prefixing the stack name with an owner name. For
example `my-great-stack` means the stack `my-great-stack` owned by
the current logged in user, where-as `pulumi/my-great-stack` would
be the stack owned by the `pulumi` organization
- `--all` can be passed to `pulumi stack ls` to see *all* stacks you
have access to, not just stacks tied to the current project.
Long term, a stack name alone will not be sufficent to address a
stack. Introduce a new `backend.StackReference` interface that allows
each backend to give an opaque stack reference that can be used across
operations.
This change removes the need to `pulumi init` when targeting the local
backend. A fair amount of the change lays the foundation that the next
set of changes to stop having `pulumi init` be used for cloud stacks
as well.
Previously, `pulumi init` logically did two things:
1. It created the bookkeeping directory for local stacks, this was
stored in `<repository-root>/.pulumi`, where `<repository-root>` was
the path to what we belived the "root" of your project was. In the
case of git repositories, this was the directory that contained your
`.git` folder.
2. It recorded repository information in
`<repository-root>/.pulumi/repository.json`. This was used by the
cloud backend when computing what project to interact with on
Pulumi.com
The new identity model will remove the need for (2), since we only
need an owner and stack name to fully qualify a stack on
pulumi.com, so it's easy enough to stop creating a folder just for
that.
However, for the local backend, we need to continue to retain some
information about stacks (e.g. checkpoints, history, etc). In
addition, we need to store our workspace settings (which today just
contains the selected stack) somehere.
For state stored by the local backend, we change the URL scheme from
`local://` to `local://<optional-root-path>`. When
`<optional-root-path>` is unset, it defaults to `$HOME`. We create our
`.pulumi` folder in that directory. This is important because stack
names now must be unique within the backend, but we have some tests
using local stacks which use fixed stack names, so each integration
test really wants its own "view" of the world.
For the workspace settings, we introduce a new `workspaces` directory
in `~/.pulumi`. In this folder we write the workspace settings file
for each project. The file name is the name of the project, combined
with the SHA1 of the path of the project file on disk, to ensure that
multiple pulumi programs with the same project name have different
workspace settings.
This does mean that moving a project's location on disk will cause the
CLI to "forget" what the selected stack was, which is unfortunate, but
not the end of the world. If this ends up being a big pain point, we
can certianly try to play games in the future (for example, if we saw
a .git folder in a parent folder, we could store data in there).
With respect to compatibility, we don't attempt to migrate older files
to their newer locations. For long lived stacks managed using the
local backend, we can provide information on where to move things
to. For all stacks (regardless of backend) we'll require the user to
`pulumi stack select` their stack again, but that seems like the
correct trade-off vs writing complicated upgrade code.
Upcoming work to remove the need for `pulumi init` makes testing the
upgrade code much harder than it did in the past (since workspace data
is moving to a different location on the file system, as well as some
other changes).
Instead of trying to maintain the code and test, let's just remove
it. Folks who haven't migrated (LM and the PPC deployment in the
service) should use the 0.11.3 or earlier CLI to migrate their
projects (simply by logging in and running a pulumi command) or move
things forward by hand.
This field indicates the schema of the serialized deployment. This field
behaves identically to the `Version` field of
`PatchUpdateCheckpointRequest`.
This is part of pulumi/pulumi-service#1046
We already had logic to skip prompting a user to create a stack,
when a stack-specific command was run but none was found, but we
only heeded it in one of two cases. This fixes the other case.
To help us diagnose various issues -- and also just as a nice status
reporting thing -- we'll add a --verbose flag to the pulumi plugin
install command. This will get used in the package scripts.
We have some code that deals with upgrading legacy projects (which had
workspace level configuration) to the new format where configuration
information was stored SxS with the application.
This code requires us to get a list of stacks from the backend (which
for hosted stacks means hitting api.pulumi.com) as part of the upgrade
process, so we knew all the stacks the user's project has. This is a
somewhat slow operation (which we will make faster regardless) but we
can structure things such that we don't need to do this often.
In the common case, we don't need to actually do upgrading at
all (new projects won't need it and once a project is upgraded that
project won't need it either) so update the code first to check if we
would need to do any work and if so, do the expensive operation of
getting stacks from a backend.
This should help with the slight pauses we've seen on the command line
since the work to default to folks logging in has landed.
The code that calculated plugin sizes was incorrect; it would show the
total size consumed by all plugins, for each plugin, which is clearly
busted. We should compute each plugin's size from its own directory.
This clashes with the existing --emoji/-e flag for the pulumi-wide
flags. I doubt --exact is going to be very common, so rather than
trying to invent a shorthand for it, we can just support the long form.
This change lets plugin versions to float in two ways:
1) If a `pulumi plugin install` detects a newer version is available
already, there's no need to download and install the older version.
2) If the engine attempts to load a plugin at a particular version,
if a newer version is available, it will be accepted without error.
As part of this, we permit $PATH to have the final say when determining
which version to accept. That is, it can always override the choice.
Note that I highly suspect, in the limit, that we'll want to stop doing
this for major version incompatibilities. For now, since we don't
envision any such version changes imminently, this will suffice.
This change does three major things:
1. Removes the ability to be logged into multiple clouds at the same
time. Previously, we supported being logged into multiple clouds at
the same time and the CLI would fan out requests and join responses
when needed. In general, this was only useful for Pulumi employees
that wanted run against multiple copies of the service (say production
and staging) but overall was very confusing (for example in the old
world a stack with the same identity could appear twice (since it was
in two backends) which the CLI didn't handle very well).
2. Stops treating the "local" backend as a special thing, from the
point of view of the CLI. Previouly we'd always connect to the local
backend and merge that data with whatever was in clouds we were
connected to. We had gestures like `--local` in `pulumi stack init`
that meant "use the local mode". Instead, to use the local mode now
you run `pulumi login --cloud-url local://` and then you are logged in
the local backend. Since you can only ever be logged into a single
backend, we can remove the `--local` and `--remote` flags from `pulumi
stack init`, it just now requires you to be logged in and creates a
stack in whatever back end you were logged into. When logging into the
local backend, you are not prompted for an access key.
3. Prompt for login in places where you have to log in, if you are not
already logged in.
Tests now target managed stacks instead of local stacks.
The existing logged in user and target backend API are used unless PULUMI_ACCES_TOKEN is defined, in which case tests are run under that access token and against the PULUMI_API backend.
For developer machines, we will now need to be logged in to Pulumi to run tests, and whichever default API backend is logged in (the one listed as current in ~/.pulumi/credentials.json) will be used. If you need to override these, provide PULUMI_ACCESS_TOKEN and possibly PULUMI_API.
For Travis, we currently target the staging service using the Pulumi Bot user.
We have decided to run tests in the pulumi organization. This can be overridden for local testing (or in Travis in the future) by defining PULUMI_API_OWNER_ORGANIZATION and using an access token with access to that organization.
Part of pulumi/home#195.
This change includes a bunch of refactorings I made in prep for
doing refresh (first, the command, see pulumi/pulumi#1081):
* The primary change is to change the way the engine's core update
functionality works with respect to deploy.Source. This is the
way we can plug in new sources of resource information during
planning (and, soon, diffing). The way I intend to model refresh
is by having a new kind of source, deploy.RefreshSource, which
will let us do virtually everything about an update/diff the same
way with refreshes, which avoid otherwise duplicative effort.
This includes changing the planOptions (nee deployOptions) to
take a new SourceFunc callback, which is responsible for creating
a source specific to the kind of plan being requested.
Preview, Update, and Destroy now are primarily differentiated by
the kind of deploy.Source that they return, rather than sprinkling
things like `if Destroying` throughout. This tidies up some logic
and, more importantly, gives us precisely the refresh hook we need.
* Originally, we used the deploy.NullSource for Destroy operations.
This simply returns nothing, which is how Destroy works. For some
reason, we were no longer doing this, and instead had some
`if Destroying` cases sprinkled throughout the deploy.EvalSource.
I think this is a vestige of some old way we did configuration, at
least judging by a comment, which is apparently no longer relevant.
* Move diff and diff-printing logic within the engine into its own
pkg/engine/diff.go file, to prepare for upcoming work.
* I keep noticing benign diffs anytime I regenerate protobufs. I
suspect this is because we're also on different versions. I changed
generate.sh to also dump the version into grpc_version.txt. At
least we can understand where the diffs are coming from, decide
whether to take them (i.e., a newer version), and ensure that as
a team we are monotonically increasing, and not going backwards.
* I also tidied up some tiny things I noticed while in there, like
comments, incorrect types, lint suppressions, and so on.
These changes refactor direct interactions with the Pulumi API out of
the cloud backend and into a subpackage, `pkg/backend/cloud/client`.
This package exposes a slightly higher-level API that takes care of
calculating paths, performing HTTP calls, and occasionally wrapping
multiple physical calls into a single logical call (notably the creation
of an update and the upload of its program).
This is primarily intended as preparation for some of the changes
suggested in the feedback for #1067.
If you use --cloud-url, as in
$ pulumi stack init foo --cloud-url https://x.io
we would silently fall back to logic that creates local stacks.
I realize all of this will get better with the new stack identity
model, however in the meantime, let's infer that the user wanted
--remote when --cloud-url is non-"".
When run without a `--plaintext` or `--secret` argument, `pulumi
config` warns that the value is stored unecrypted and that you can
pass `--secret` to encrypt it. Now, we also mention that `--plaintext`
can be pased explicity on the command line to avoid the warning.
Fixes#752
Note: This is a minor issue that I didn't get to for M11 that isn't
required for M11 and would be fine merging for post-M11.
When you specify a template name explicitly (e.g.
`pulumi new typescript`), we'll try to download the template tarball
without first downloading the JSON list of available templates. The JSON
includes a description used when replacing the `${DESCRIPTION}` string
in template files. Since we didn't download the JSON, we won't have a
description, so we fallback to a default value (`"A Pulumi project."`).
This also happens when specifying `--offline` to use an existing
template under `~/.pulumi/templates`; we won't have a description for
the template, so we fallback to a default description. The fallback
value happens to be the same as the description for each of our current
templates, so noone will currently notice an issue.
For M11, I included initial support for a template manifest file where
the description (and any future metadata) could be stored, but didn't go
as far as actually reading the file.
This change makes it so the CLI actually reads the description from the
manifest file (if it exists), otherwise falling back to the default
value as is done currently. Some minor related cleanup is included in
this change.
This fixes two different DST bugs in the logs test, one of which
led to the failure in pulumi/pulumi#1041:
1) The assertion was using ANSIC formatting, which does not contain
a time-zone, and because we are now in DST, the resulting time
didn't parse and format in a round-trippable way. Instead, let's
use RFC3339, and let's be explicit about the timezone.
2) The parseSince("1m30s") would in theory fail if the DST changed
at just the right moment, since the reference time could end up
different from the reference time that the test is using. Let's
expose the reference time to the caller, so that they may decide
how to deal with these situations, and let's use UTC here.
This adds a `pulumi new` command which makes it easy to quickly
automatically create the handful of needed files to get started building
an empty Pulumi project.
Usage:
```
$ pulumi new typescript
```
Or you can leave off the template name, and it will ask you to choose
one:
```
$ pulumi new
Please choose a template:
> javascript
python
typescript
```
The engine now emits events with richer metadata during the
ResourceOutputs and ResourcePre callbacks. The CLI can then use this
information to decide if it should display the event or not and how
much of the event to display.
Options dealing with what to display and how to display it have moved
into the CLI and the engine now emits all information for each event.
The engine now unconditionally emits a new type of event, a
PreludeEvent, which contains the configuration for a stack as well as
an indication if the stack is being previewed or updated. The
responsibility for interpreting the --show-config flag on the command
line is now handled by the CLI, which uses this to decide if it should
print the configuration or not, and then writes the "Previewing
changes" or "Deploying chanages" header.
This helper method is only really used for testing, but we should not
allow it to create a Key who's namespace has a colon (as ParseKey
would not build something like this).
config.Key has become a pair of namespace and name. Because the whole
world has not changed yet, there continues to be a way to convert
between a tokens.ModuleMember and config.Key, however now sometime the
conversion from tokens.ModuleMember can fail (when the module member
is not of the form `<package>:config:<name>`).
Right now, config.Key is a type alias for tokens.ModuleMember. I did a
pass over the codebase such that we use config.Key everywhere it
looked like the value did not leak to some external process (e.g a
resource provider or a langhost).
Doing this makes it a little clearer (hopefully) where code is
depending on a module member structure (e.g. <package>:config:<value>)
instead of just an opaque type.
As it stands, we only configure those providers for which configuration
is present. This can lead to surprising failure modes if those providers
are then used to create resources. These changes ensure that all
resource providers that are not configured during plan initialization
are configured upon first load.
Fixes#758.
