Commit graph

1499 commits

Author SHA1 Message Date
Sean Gillespie f83d32390c
Don't close eventChannel when panicking (#1891)
* Don't close eventChannel when panicking

The state of the system is completely unknown when panicking and in
general it's not safe to infer whether or not it is safe to close a
channel when in this tate.

* CR feedback

* Spelling
2018-09-05 17:50:48 -07:00
Sean Gillespie ca58b8117f
Clarify control flow in step generator (#1843)
* Introduce Result type to engine

The Result type can be used to signal the failure of a computation due
to both internal and non-internal reasons. If a computation failed due
to an internal error, the Result type carries that error with it and
provides it when the 'Error' method on a Result is called. If a
computation failed gracefully, but wished to bail instead of continue a
doomed plan, the 'Error' method provides a value of null.

* CR feedback
2018-09-05 15:08:09 -07:00
Pat Gavlin df1a5e653d
Fail refreshes with init errors. (#1882)
And ensure that refreshes continue on errors.

Fixes #1881.
2018-09-05 14:00:28 -07:00
Joe Duffy 373bc25cfd
Merge pull request #1875 from pulumi/joeduffy/1818_local_backend
Improve the local backend
2018-09-05 13:00:26 -07:00
joeduffy c1967129e7 Fix integration tests
This fixes the integration tests:

* Expect and allow the update header.

* Don't print the local permalink if there's an error.
2018-09-05 11:39:58 -07:00
joeduffy b1f7cf7050 Fix a few lint warnings 2018-09-05 08:25:23 -07:00
joeduffy db48f10412 Update test frameworks to new packages 2018-09-05 08:16:14 -07:00
Justin Van Patten b46820dbef
Move away from ${PROJECT} and ${DESCRIPTION} (#1873)
We generally want examples and apps to be authored such that they are
clonable/deployable as-is without using new/up (and want to
encourage this). That means no longer using the ${PROJECT} and
${DESCRIPTION} replacement strings in Pulumi.yaml and other text files.
Instead, good default project names and descriptions should be specified
in Pulumi.yaml and elsewhere.

We'll use the specified values as defaults when prompting the user, and
then directly serialize/save the values to Pulumi.yaml when configuring
the user's project. This does mean that name in package.json (for nodejs
projects) won't be updated if it isn't using ${PROJECT}, but that's OK.

Our templates in the pulumi/templates repo will still use
${PROJECT}/${DESCRIPTION} for now, to continue to work well with v0.15
of the CLI. After that version is no longer in use, we can update the
templates to no longer use the replacement strings and delete the code
in the CLI that deals with it.
2018-09-05 08:00:57 -07:00
joeduffy 7ecc08d75d Print a link to the local stack's checkpoint file
Similar to how we print a URL at the end of each cloud-based
update, we now print a link to the stack's checkpoint file after
each update.
2018-09-05 07:33:18 -07:00
joeduffy 95e917441a Implement preview-then-update for local stacks
This change implements the same preview behavior we have for
cloud stacks, in pkg/backend/httpbe, for local stacks, in
pkg/backend/filebe. This mostly required just refactoring bits
and pieces so that we can share more of the code, although it
does still entail quite a bit of redundancy. In particular, the
apply functions for both backends are now so close to being
unified, but still require enough custom logic that it warrants
keeping them separate (for now...)
2018-09-05 07:33:18 -07:00
joeduffy bf51d7594a Refactor display logic out of pkg/backend/filestate
This simply refactors all the display logic out of the
pkg/backend/filestate package. This helps to gear us up to better unify
this logic between the filestate and httpstate backends.

Furthermore, this really ought to be in its own non-backend,
CLI-specific package, but I'm taking one step at a time here.
2018-09-05 07:33:18 -07:00
joeduffy d2d24612e8 Fix test import issue 2018-09-05 07:32:42 -07:00
joeduffy d28f0cb7b6 Alter the login prompt
This change alters the login prompt slightly, so that it is more
obvious that alternative methods exist.

Before this change, we would say:

    $ pulumi login
    We need your Pulumi account to identify you.
    Enter your access token from https://app.pulumi.com/account
        or hit <ENTER> to log in using your browser            :

After this change, we say this instead:

    $ pulumi login
    Manage your Pulumi stacks by logging in.
    Run `pulumi login --help` for alternative login options.
    Enter your access token from https://app.pulumi.com/account
        or hit <ENTER> to log in using your browser            :

Also updated the help text to advertise this a bit more prominently.
2018-09-05 07:32:42 -07:00
joeduffy feaea31f7b Rename backend packages
This renames the backend packages to more closely align with the
new direction for them. Namely, pkg/backend/cloud becomes
pkg/backend/httpstate and pkg/backend/local becomes
pkg/backend/filestate. This also helps to clarify that these are meant
to be around state management and so the upcoming refactoring required
to split out (e.g.) the display logic (amongst other things) will make
more sense, and we'll need better package names for those too.
2018-09-05 07:32:42 -07:00
joeduffy 126d31c9c2 Simplify logging into the local backend
As part of making the local backend more prominent, this changes a few
aspects of how you use it:

* Simplify how you log into a specific cloud; rather than
  `pulumi login --cloud-url <url>`, just say `pulumi login <url>`.

* Use a proper URL scheme to denote local backend usage. We have chosen
  file://, since the REST API backend is of course always https://.
  This means that you can say `pulumi login file://~` to use the local
  backend, with state files stored in your home directory. Similarly,
  we support `pulumi login file://.` for the current directory.

* Add a --local flag to the login command, to make local logins a
  bit easier in the common case of using your home directory. Just say
  `pulumi login --local` and it is sugar for `pulumi login file://~`.

* Print the URL for the backend after logging in; for the cloud,
  this is just the user's stacks page, and for the local backend,
  this is the path to the user's stacks directory on disk.

* Tidy up the documentation for login a bit to be clearer about this.

This is part of pulumi/pulumi#1818.
2018-09-05 07:32:42 -07:00
CyrusNajmabadi 90ce5231ab
Disable colors if we're not in an interactive session. (#1878) 2018-09-05 00:31:15 -07:00
Matt Ellis 9ec695dda0 Reformat call to action when we fail to launch browser for login
The existing message put the URL to visit and some explanation text on
the same line, which makes it a little harder to copy only the URL
into a browser. If this extra text ends up being copied as well as the
URL it can lead to failures later, when we try to decode the query
string as part of the OAuth flow.

It's easy enough to fix by just putting the URL on its own line, split
off from the text itself.

Fixes #1832
2018-09-04 14:28:29 -07:00
Justin Van Patten 5586f4ecad
Improve the error message for pulumi new when the template doesn't exist (#1823)
And offer distance-based suggestions.
2018-09-04 08:40:59 -07:00
Joe Duffy fd852754fd
Merge pull request #1848 from pulumi/joeduffy/1551_refresh_text_clarification
Clarify refresh text
2018-09-02 11:58:50 -07:00
joeduffy 4827c7fb29 Update some text/colors
This changes two things:

1) Per PR feedback, change the text to "N changes found during refresh."

2) Colorize the parenthetical "No resources will be modified" message;
   it looked a little odd being plain colored.
2018-09-02 11:26:39 -07:00
Alex Clemmer dea68b8b37 Implement status sinks
This commit reverts most of #1853 and replaces it with functionally
identical logic, using the notion of status message-specific sinks.

In other words, where the original commit implemented ephemeral status
messages by adding an `isStatus` parameter to most of the logging
methdos in pulumi/pulumi, this implements ephemeral status messages as a
parallel logging sink, which emits _only_ ephemeral status messages.

The original commit message in that PR was:

> Allow log events to be marked "status" events
>
> This commit will introduce a field, IsStatus to LogRequest. A "status"
> logging event will be displayed in the Info column of the main
> display, but will not be printed out at the end, when resource
> operations complete.
>
> For example, for complex resource initialization, we'd like to display
> a series of intermediate results: [1/4] Service object created, for
> example. We'd like these to appear in the Info column, but not at the
> end, where they are not helpful to the user.
2018-08-31 15:56:53 -07:00
Alex Clemmer 9e58fd1aaa Revert "Plumb LogRequest.IsStatus through the logging subsystem"
This reverts commit 3066cbcbd7.
2018-08-31 15:56:53 -07:00
Luke Hoban b9c64b1e70
Allow skipping refresh step in integration tests (#1852)
Allows working around bugs in refresh (such as https://github.com/pulumi/pulumi-terraform/issues/241).
2018-08-31 11:34:26 -05:00
Alex Clemmer 3066cbcbd7 Plumb LogRequest.IsStatus through the logging subsystem 2018-08-30 17:17:20 -07:00
Justin Van Patten c29776e561
Support project config keys in templates and -c (#1841)
Previously, we only supported config keys that included a ':' delimiter
in config keys specified in the template manifest and in `-c` flags to
`new` and `up`. This prevented the use of project keys in the template
manifest and made it more difficult to pass such keys with `-c`,
effectively preventing the use of `new pulumi.Config()` in project code.

This change fixes this by allowing config keys that don't have a
delimiter in the template manifest and `-c` flags. In such cases, the
project name is automatically prepended behind the scenes, the same as
what `pulumi config set` does.
2018-08-30 14:56:13 -07:00
Luke Hoban 1ebd698a7a
Support for more ProgramTestOptions in With (#1844)
The pattern we are using here is generally prone to error, so I hope we find a way to move away from this more generally - but for now we need to be able to configure more of these in places we are using `With`.

Also remove some `+""` that are tripping up the linter for me locally.
2018-08-30 11:08:19 -05:00
Pat Gavlin 0634c1852a
Fix some potential bugs in the refresher. (#1845)
- Create all refresh steps before issuing any. This is important as the
  state update loop expects all steps to exist.
- Check for cancellation later in the refresher.

This also fixes races in the SnapshotManager and the test journal that
could cause panics during cancellation.
2018-08-29 21:00:05 -07:00
joeduffy cd5707f483 Clarify refresh text
The wording for refresh doesn't accurately convey that the operations
aren't actually mutating your resources, but instead are simply changing
your checkpoint state. This change (hopefully) helps in two ways:

First, put text just before the prompt:

Do you want to perform this refresh?
No resources will be modified as part of this refresh; just your stack's state will be.

Second, alter the summary ever-so-slightly, from:

info: 2 changes performed:
    ~ 2 resources updated
      3 resources unchanged

to:

info: 2 changes refreshed:
    ~ 2 resources updated
      3 resources unchanged

This reads just slightly better, and removes any sense of panic I might
have otherwise had that my refresh just did something wrong.

As I was in here, since I had to pass UpdateKind information to new
places, I cleaned up the situation where we had three mostly-similar
enums (but which actually diverged) and several areas where we were
using untyped strings for this same information. Now there's just one.

This fixes pulumi/pulumi#1551.
2018-08-29 17:09:14 -07:00
Pat Gavlin 85db1e6ea9
Add a WriteString method to tools/GenWriter. (#1839)
This method writes a string to the underlying bufio.Writer without
formatting it first.
2018-08-29 14:43:20 -07:00
Alex Clemmer 2baf920c96 Remove unused print statements 2018-08-28 18:00:35 -07:00
Alex Clemmer 2fa98a8dad Generate empty update steps for partial failures
This commit will greatly improve the experience of dealing with partial
failures by simply re-trying to initialize the relevant resources on
every subsequent `pulumi up`, instead of printing a list of reasons the
resource had previously failed to initialize.

As motivation, consider our behavior in the following common, painful
scenario:

  * The user creates a `Service` and a `Deployment`.
  * The `Pod`s in the `Deployment` fail to become live. This causes the
    `Service` to fail, since it does not target any live `Pod`s.
  * The user fixes the `Deployment`. A run of `pulumi up` sees the
    `Pod`s successfully initialize.
  * Users will expect that the `Service` is now in a state of success,
    as the `Pod`s it targets are alive. But, because we don't update the
    `Service` by default, it perpetually exists in a state of error.
  * The user is now required to change some trivial feature of the
    `Service` just to trigger an update, so that we can see it succeed.

There are many situations like this. Another very common one is waiting
for test `Pod`s that are meant to successfully complete when some object
becomes live.

By triggering an empty update step for all resources that have any
initialization errors, we avoid all problems like this.

This commit will implement this empty-update semantics for partial
failures, as well as fix the display UX to correctly render the diff in
these cases.
2018-08-28 18:00:35 -07:00
Sean Gillespie 1624d4a1fe
Remove dead code (#1833)
Large parts of pkg/tokens are not used anymore and can be deleted.
2018-08-28 14:43:41 -07:00
Matt Ellis 8f5def8453 Do not delete failed stacks from integration tests
Often when the test fails, you'll want to use the web console to
investigate the stack, looking at detailed log messages, for
example. But since we always delete the stack today, you often can't.

Stop running `pulumi stack rm` when the test fails, however do run a
`pulumi destroy` so we at least clean up any resources the stack may
have allocated during testing.

Fixes #1722
2018-08-27 13:28:46 -07:00
Pat Gavlin abfdf69a9c
Indent stack outputs by 1. (#1822)
* Indent stack outputs by 1.

Also:
- Show stack output diffs during preview
- Fix a bug where deletes in stack outputs were not displayed
2018-08-24 15:36:55 -07:00
Pat Gavlin 73f4f2c464
Reimplement refresh. (#1814)
Replace the Source-based implementation of refresh with a phase that
runs as the first part of plan execution and rewrites the snapshot in-memory.

In order to fit neatly within the existing framework for resource operations,
these changes introduce a new kind of step, RefreshStep, to represent
refreshes. RefreshSteps operate similar to ReadSteps but do not imply that
the resource being read is not managed by Pulumi.

In addition to the refresh reimplementation, these changes incorporate those
from #1394 to run refresh in the integration test framework.

Fixes #1598.
Fixes pulumi/pulumi-terraform#165.
Contributes to #1449.
2018-08-22 17:52:46 -07:00
Matt Ellis ad2d51898c Don't use :config: token when sending values to the service
The fact that we include the `:config:` portion of a configuration
name when sending to the service is an artifact of history. It was
needed back when we needed to support deploying using a new CLI
against a PPC that had older packages embeded in it.

We don't have that sort of problem anymore, so we can stop sending
this data.
2018-08-22 16:41:46 -07:00
Sean Gillespie 3cba431ada
Show a better error message when decrypting fails (#1815)
* Show a better error message when decrypting fails

It is most often the case that failing to decrypt a secret implies that
the secret was transferred from one stack to another via copying the
configuration. This commit introduces a better error message for this
case and instructs users to explicitly re-encrypt their encrypted keys
in the context of the new stack.

* Spelling

* CR: Grammar fixes
2018-08-22 15:32:54 -07:00
Pat Gavlin 91e20289a8
Remove deploy.PlanSummary. (#1809)
Nothing actually uses this information.
2018-08-21 19:33:59 -07:00
Pat Gavlin 734f7beba7
Refactor plan execution in preparation for refresh. (#1801)
These changes simplify a couple aspects of plan execution in the hopes of
clarifying some responsibilities and preparing the code for changes to the
implementation of refresh.

1. All aspects of plan execution are now managed by the plan executor,
   which is no longer exported. Instead, it is abstracted behind
   `Plan.Execute`.
2. The plan executor's error-handling and reporting have been unified
   and simplified somewhat.
2018-08-21 14:05:00 -07:00
Sean Gillespie d1524e1081
Log errors coming from the language host (#1780)
* Log errors coming from the language host

Similar to pulumi/pulumi#1762, fixes pulumi/pulumi#1775. The language
host can fail without issuing any diagnostics and it is very unclear
what happens if the engine does not log the error.

* CR feedback
2018-08-20 16:41:07 -07:00
Pat Gavlin effff869a1
Elide unnecessary checkpoint writes. (#1798)
A checkpoint write is unnecessary if it does not change the semantics of
the data currently stored in the checkpoint. We currently perform
unnecessary checkpoint writes in two cases:
- Same steps where no aspect of the resource's state has changed
- Replace steps, which exist solely for display purposes

The former case is particularly bothersome, as it is rather common to
run updates--especially in CI--that consist largely/entirely of these
same steps.

These changes eliminate the checkpoint writes we perform in these two
cases. Some care is needed to ensure that we continue to write the
checkpoint in the case of same steps that do represent meaningful
changes (e.g. changes to a resource's output properties or
dependencies).

Fixes #1769.
2018-08-20 14:14:50 -07:00
Matt Ellis acf0fb278a Fix wierd interactions due to Cobra and glog
The glog package force the use of golang's underyling flag package,
which Cobra does not use. To work around this, we had a complicated
dance around defining flags in multiple places, calling flag.Parse
explicitly and then stomping values in the flag package with values we
got from Cobra.

Because we ended up parsing parts of the command line twice, each with
a different set of semantics, we ended up with bad UX in some
cases. For example:

`$ pulumi -v=10 --logflow update`

Would fail with an error message that looked nothing like normal CLI
errors, where as:

`$ pulumi -v=10 update --logflow`

Would behave as you expect. To address this, we now do two things:

- We never call flag.Parse() anymore. Wacking the flags with values we
  got from Cobra is sufficent for what we care about.

- We use a forked copy of glog which does not complain when
  flag.Parse() is not called before logging.

Fixes #301
Fixes #710
Fixes #968
2018-08-20 14:08:40 -07:00
Sean Gillespie 347148c374
Don't try to convert a checkpoint's Latest field if nil (#1800)
Not all checkpoints have a Latest field and it is not guaranteed to be
non-nil. Fixes pulumi/pulumi#1788.
2018-08-20 13:47:49 -07:00
CyrusNajmabadi 89d5dd004e
Do not add the same file multiple times to a zip or tar file. (#1783) 2018-08-15 22:44:55 -07:00
Sean Gillespie 3a26ec382a
Add test for pulumi/pulumi#1770 (#1773)
* Add test for pulumi/pulumi#1770

* CR: Add parallel test, do an update before refresh
2018-08-14 15:32:23 -07:00
Sean Gillespie d83774ca96
Fix two issues that can cause hangs during refresh (#1770)
1. 'readID' was never assigned to and was always the default value,
leading the refresh source to believe a resource was deleted
2. The refresh source could hang when a resource is deleted.
2018-08-13 21:43:10 -07:00
Pat Gavlin 0caba6b990
Elide resource pre events for default providers in preview. (#1767)
Just what it says on the tin.
2018-08-13 15:45:08 -07:00
Sean Gillespie d9c56eab23
Emit diagnostics for errors coming from step generation (#1766)
The plan executor assumed that the step generator was responsible for
logging its own diagnostics, which it sort-of is but also doesn't log a
majority of the diagnositcs that come out of it. This commit logs all
errors coming out of step generation so that we don't unintentionally
drop errors.
2018-08-13 13:41:01 -07:00
Matt Ellis e51e159d18 Warn when pulumi is out of date
Before running commands, ask the service for version information about
the CLI and warn the user to upgrade in some cases.

Fixes #1686
2018-08-13 13:18:02 -07:00
CyrusNajmabadi da616eccca
Print out an archive diff even in the event of resource replacement. (#1760) 2018-08-11 18:33:27 -07:00