Commit graph

493 commits

Author SHA1 Message Date
CyrusNajmabadi
3b13803c71
Add a hidden --no-interactive flag so that we can reduce interactive noise when running jenkins. (#1262) 2018-04-24 14:23:08 -07:00
Chris Smith
6a0718bcff
Revert adding HEAD commit info to update env (#1257) 2018-04-23 14:01:28 -07:00
Matt Ellis
fe8bad70d1 Don't mention PPC unless needed in the CLI
PPCs are no longer a central concept to our model, but instead a
feature that that pulumi.com provides to some organizations. Let's
remove most mentions of PPCs except for cases where we really need to
talk about them (e.g. when a stack is actually hosted in a PPC instead
of just via the normal pulumi.com service)

Also remove some "in the Pulumi Cloud" messages from the CLI, as using
the Pulumi Cloud is now the only real way to use pulumi.

Fixes pulumi/pulumi-service#1117
2018-04-23 16:50:48 -04:00
Chris Smith
4651419f7f
Add Commit message, committer, and author to update env (#1249)
* Add Commit message, committer, and author to update env

* Remove low value comments
2018-04-23 11:39:14 -07:00
pat@pulumi.com
42b67a4dbc Fix a couple of typos.
- Restore a dropped "update"
- %s/Resouce/Resource/g
2018-04-20 11:09:54 -07:00
Matt Ellis
15e2ad27fe Address code review feedback 2018-04-20 02:34:10 -04:00
Matt Ellis
cc938a3bc8 Merge remote-tracking branch 'origin/master' into ellismg/identity 2018-04-20 01:56:41 -04:00
Matt Ellis
04e5dfde5f Address code review feedback 2018-04-20 01:31:14 -04:00
Pat Gavlin
4fa69bfd72
Plumb basic cancellation through the engine. (#1231)
hese changes plumb basic support for cancellation through the engine.
Two types of cancellation are supported for all engine operations:
- Cancellation, which waits for the operation to drive itself to a safe
  point before the operation returns, and
- Termination, which does not wait for the operation to drive itself
  to a safe opint for the operation returns.

When updating local or managed stacks, a single ^C triggers cancellation
of any running operation; a second ^C will trigger termination.

Fixes #513, #1077.
2018-04-19 18:59:14 -07:00
Pat Gavlin
d1c547524d
Add a pulumi cancel command. (#1230)
This command cancels a stack's currently running update, if any. It can
be used to recover from the scenario in which an update is aborted
without marking the running update as complete. Once an update has been
cancelled, it is likely that the affected stack will need to be repaired
via an pair of export/import commands before future updates can succeed.

This is part of #1077.
2018-04-19 10:09:32 -07:00
Joe Duffy
3b8d0e6d96
Merge pull request #1148 from pulumi/1081_refresh_cmd
Implement a refresh command
2018-04-18 12:12:33 -07:00
joeduffy
bac58d7922 Respond to CR feedback
Incorporate feedback from @swgillespie and @pgavlin.
2018-04-18 11:46:37 -07:00
joeduffy
b77403b4bb Implement a refresh command
This change implements a `pulumi refresh` command.  It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.

It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events.  This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state.  This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion).  The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.

Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint.  That
will be coming soon ...
2018-04-18 10:57:16 -07:00
Matt Ellis
56d7f8eb24 Support new stack identity for the cloud backend
This change introduces support for using the cloud backend when
`pulumi init` has not been run. When this is the case, we use the new
identity model, where a stack is referenced by an owner and a stack
name only.

There are a few things going on here:

- We add a new `--owner` flag to `pulumi stack init` that lets you
  control what account a stack is created in.

- When listing stacks, we show stacks owned by you and any
  organizations you are a member of. So, for example, I can do:

  * `pulumi stack init my-great-stack`
  * `pulumi stack init --owner pulumi my-great-stack`

  To create a stack owned by my user and one owned by my
  organization. When `pulumi stack ls` is run, you'll see both
  stacks (since they are part of the same project).

- When spelling a stack on the CLI, an owner can be optionally
  specified by prefixing the stack name with an owner name. For
  example `my-great-stack` means the stack `my-great-stack` owned by
  the current logged in user, where-as `pulumi/my-great-stack` would
  be the stack owned by the `pulumi` organization

- `--all` can be passed to `pulumi stack ls` to see *all* stacks you
  have access to, not just stacks tied to the current project.
2018-04-18 04:54:02 -07:00
Matt Ellis
c0b2c4f17f Introduce backend.StackReference
Long term, a stack name alone will not be sufficent to address a
stack. Introduce a new `backend.StackReference` interface that allows
each backend to give an opaque stack reference that can be used across
operations.
2018-04-18 04:54:02 -07:00
Matt Ellis
bac02f1df1 Remove the need to pulumi init for the local backend
This change removes the need to `pulumi init` when targeting the local
backend. A fair amount of the change lays the foundation that the next
set of changes to stop having `pulumi init` be used for cloud stacks
as well.

Previously, `pulumi init` logically did two things:

1. It created the bookkeeping directory for local stacks, this was
stored in `<repository-root>/.pulumi`, where `<repository-root>` was
the path to what we belived the "root" of your project was. In the
case of git repositories, this was the directory that contained your
`.git` folder.

2. It recorded repository information in
`<repository-root>/.pulumi/repository.json`. This was used by the
cloud backend when computing what project to interact with on
Pulumi.com

The new identity model will remove the need for (2), since we only
need an owner and stack name to fully qualify a stack on
pulumi.com, so it's easy enough to stop creating a folder just for
that.

However, for the local backend, we need to continue to retain some
information about stacks (e.g. checkpoints, history, etc). In
addition, we need to store our workspace settings (which today just
contains the selected stack) somehere.

For state stored by the local backend, we change the URL scheme from
`local://` to `local://<optional-root-path>`. When
`<optional-root-path>` is unset, it defaults to `$HOME`. We create our
`.pulumi` folder in that directory. This is important because stack
names now must be unique within the backend, but we have some tests
using local stacks which use fixed stack names, so each integration
test really wants its own "view" of the world.

For the workspace settings, we introduce a new `workspaces` directory
in `~/.pulumi`. In this folder we write the workspace settings file
for each project. The file name is the name of the project, combined
with the SHA1 of the path of the project file on disk, to ensure that
multiple pulumi programs with the same project name have different
workspace settings.

This does mean that moving a project's location on disk will cause the
CLI to "forget" what the selected stack was, which is unfortunate, but
not the end of the world. If this ends up being a big pain point, we
can certianly try to play games in the future (for example, if we saw
a .git folder in a parent folder, we could store data in there).

With respect to compatibility, we don't attempt to migrate older files
to their newer locations. For long lived stacks managed using the
local backend, we can provide information on where to move things
to. For all stacks (regardless of backend) we'll require the user to
`pulumi stack select` their stack again, but that seems like the
correct trade-off vs writing complicated upgrade code.
2018-04-18 04:53:49 -07:00
Matt Ellis
e83aa175ff Remove configuration upgrade code
Upcoming work to remove the need for `pulumi init` makes testing the
upgrade code much harder than it did in the past (since workspace data
is moving to a different location on the file system, as well as some
other changes).

Instead of trying to maintain the code and test, let's just remove
it. Folks who haven't migrated (LM and the PPC deployment in the
service) should use the 0.11.3 or earlier CLI to migrate their
projects (simply by logging in and running a pulumi command) or move
things forward by hand.
2018-04-18 03:29:19 -07:00
Pat Gavlin
a4d6cba664 Add a Version field to UntypedDeployment.
This field indicates the schema of the serialized deployment. This field
behaves identically to the `Version` field of
`PatchUpdateCheckpointRequest`.

This is part of pulumi/pulumi-service#1046
2018-04-17 16:23:20 -07:00
Joe Duffy
fe45eb875e
Don't prompt for stack removal (#1198)
We already had logic to skip prompting a user to create a stack,
when a stack-specific command was run but none was found, but we
only heeded it in one of two cases.  This fixes the other case.
2018-04-14 10:01:39 -07:00
CyrusNajmabadi
f2b9bd4b13
Remove the explicit 'pulumi preview' command. (#1170)
Old command still exists, but tells you to run "pulumi update --preview".
2018-04-13 22:26:01 -07:00
Matt Ellis
aa98e54239 Add pointer to docs.pulumi.com in the help text
Fixes #989
2018-04-12 13:22:06 -07:00
Joe Duffy
9ba6584bbf
Add a --verbose flag to plugin install (#1160)
To help us diagnose various issues -- and also just as a nice status
reporting thing -- we'll add a --verbose flag to the pulumi plugin
install command.  This will get used in the package scripts.
2018-04-11 17:51:14 -07:00
Chris Smith
ab2385437a
Validate stack properties like names, runtime, etc. (#1146)
* Validate stack properties like names, runtime, etc.

* Fix build error
2018-04-11 10:08:32 -07:00
CyrusNajmabadi
a759f2e085
Switch to a resource-progress oriented view for pulumi preview/update/destroy (#1116) 2018-04-10 12:03:11 -07:00
Matt Ellis
8d4b227fee Early out in project file upgrade code
We have some code that deals with upgrading legacy projects (which had
workspace level configuration) to the new format where configuration
information was stored SxS with the application.

This code requires us to get a list of stacks from the backend (which
for hosted stacks means hitting api.pulumi.com) as part of the upgrade
process, so we knew all the stacks the user's project has. This is a
somewhat slow operation (which we will make faster regardless) but we
can structure things such that we don't need to do this often.

In the common case, we don't need to actually do upgrading at
all (new projects won't need it and once a project is upgraded that
project won't need it either) so update the code first to check if we
would need to do any work and if so, do the expensive operation of
getting stacks from a backend.

This should help with the slight pauses we've seen on the command line
since the work to default to folks logging in has landed.
2018-04-09 17:07:14 -07:00
Joe Duffy
8421fd9e52
Show correct plugin sizes (#1137)
The code that calculated plugin sizes was incorrect; it would show the
total size consumed by all plugins, for each plugin, which is clearly
busted.  We should compute each plugin's size from its own directory.
2018-04-09 12:51:32 -07:00
joeduffy
c7e9a2228b Remove -e shorthand for --exact
This clashes with the existing --emoji/-e flag for the pulumi-wide
flags.  I doubt --exact is going to be very common, so rather than
trying to invent a shorthand for it, we can just support the long form.
2018-04-08 10:58:26 -07:00
Joe Duffy
f2ae3a7afc
Permit plugin versions to float (#1122)
This change lets plugin versions to float in two ways:

1) If a `pulumi plugin install` detects a newer version is available
   already, there's no need to download and install the older version.

2) If the engine attempts to load a plugin at a particular version,
   if a newer version is available, it will be accepted without error.

As part of this, we permit $PATH to have the final say when determining
which version to accept.  That is, it can always override the choice.

Note that I highly suspect, in the limit, that we'll want to stop doing
this for major version incompatibilities. For now, since we don't
envision any such version changes imminently, this will suffice.
2018-04-05 16:37:50 -07:00
Matt Ellis
d3240fdc64 Require pulumi login before commands that need a backend
This change does three major things:

1. Removes the ability to be logged into multiple clouds at the same
time. Previously, we supported being logged into multiple clouds at
the same time and the CLI would fan out requests and join responses
when needed. In general, this was only useful for Pulumi employees
that wanted run against multiple copies of the service (say production
and staging) but overall was very confusing (for example in the old
world a stack with the same identity could appear twice (since it was
in two backends) which the CLI didn't handle very well).

2. Stops treating the "local" backend as a special thing, from the
point of view of the CLI. Previouly we'd always connect to the local
backend and merge that data with whatever was in clouds we were
connected to. We had gestures like `--local` in `pulumi stack init`
that meant "use the local mode". Instead, to use the local mode now
you run `pulumi login --cloud-url local://` and then you are logged in
the local backend. Since you can only ever be logged into a single
backend, we can remove the `--local` and `--remote` flags from `pulumi
stack init`, it just now requires you to be logged in and creates a
stack in whatever back end you were logged into. When logging into the
local backend, you are not prompted for an access key.

3. Prompt for login in places where you have to log in, if you are not
already logged in.
2018-04-05 10:19:41 -07:00
Luke Hoban
5ede33e03d
Run tests against managed stacks backend instead of FnF (#1092)
Tests now target managed stacks instead of local stacks.

The existing logged in user and target backend API are used unless PULUMI_ACCES_TOKEN is defined, in which case tests are run under that access token and against the PULUMI_API backend.

For developer machines, we will now need to be logged in to Pulumi to run tests, and whichever default API backend is logged in (the one listed as current in ~/.pulumi/credentials.json) will be used. If you need to override these, provide PULUMI_ACCESS_TOKEN and possibly PULUMI_API.

For Travis, we currently target the staging service using the Pulumi Bot user.

We have decided to run tests in the pulumi organization. This can be overridden for local testing (or in Travis in the future) by defining PULUMI_API_OWNER_ORGANIZATION and using an access token with access to that organization.

Part of pulumi/home#195.
2018-04-02 21:34:54 -07:00
Chris Smith
aecf0d8d19
Fix typeo in error message
Adding space in "Use--plaintext to avoid this warning".
2018-03-31 13:48:46 -07:00
Pat Gavlin
b7e55108c9 Fix stack import.
At some point--likely when we were tweaking API types--`stack import`
stopped working correctly. These changes straighten things out again.
2018-03-30 15:51:40 -07:00
joeduffy
8b5874dab5 General prep work for refresh
This change includes a bunch of refactorings I made in prep for
doing refresh (first, the command, see pulumi/pulumi#1081):

* The primary change is to change the way the engine's core update
  functionality works with respect to deploy.Source.  This is the
  way we can plug in new sources of resource information during
  planning (and, soon, diffing).  The way I intend to model refresh
  is by having a new kind of source, deploy.RefreshSource, which
  will let us do virtually everything about an update/diff the same
  way with refreshes, which avoid otherwise duplicative effort.

  This includes changing the planOptions (nee deployOptions) to
  take a new SourceFunc callback, which is responsible for creating
  a source specific to the kind of plan being requested.

  Preview, Update, and Destroy now are primarily differentiated by
  the kind of deploy.Source that they return, rather than sprinkling
  things like `if Destroying` throughout.  This tidies up some logic
  and, more importantly, gives us precisely the refresh hook we need.

* Originally, we used the deploy.NullSource for Destroy operations.
  This simply returns nothing, which is how Destroy works.  For some
  reason, we were no longer doing this, and instead had some
  `if Destroying` cases sprinkled throughout the deploy.EvalSource.
  I think this is a vestige of some old way we did configuration, at
  least judging by a comment, which is apparently no longer relevant.

* Move diff and diff-printing logic within the engine into its own
  pkg/engine/diff.go file, to prepare for upcoming work.

* I keep noticing benign diffs anytime I regenerate protobufs.  I
  suspect this is because we're also on different versions.  I changed
  generate.sh to also dump the version into grpc_version.txt.  At
  least we can understand where the diffs are coming from, decide
  whether to take them (i.e., a newer version), and ensure that as
  a team we are monotonically increasing, and not going backwards.

* I also tidied up some tiny things I noticed while in there, like
  comments, incorrect types, lint suppressions, and so on.
2018-03-28 07:45:23 -07:00
Pat Gavlin
e884c63104 Extract a Pulumi API client.
These changes refactor direct interactions with the Pulumi API out of
the cloud backend and into a subpackage, `pkg/backend/cloud/client`.
This package exposes a slightly higher-level API that takes care of
calculating paths, performing HTTP calls, and occasionally wrapping
multiple physical calls into a single logical call (notably the creation
of an update and the upload of its program).

This is primarily intended as preparation for some of the changes
suggested in the feedback for #1067.
2018-03-21 16:45:54 -07:00
Pat Gavlin
a23b10a9bf
Update the copyright end date to 2018. (#1068)
Just what it says on the tin.
2018-03-21 12:43:21 -07:00
Joe Duffy
abbac13655
Assume --remote when --cloud-url is set (#1060)
If you use --cloud-url, as in

    $ pulumi stack init foo --cloud-url https://x.io

we would silently fall back to logic that creates local stacks.

I realize all of this will get better with the new stack identity
model, however in the meantime, let's infer that the user wanted
--remote when --cloud-url is non-"".
2018-03-18 12:44:00 -07:00
Matt Ellis
642ee91065 Advise about --plaintext in config warning message
When run without a `--plaintext` or `--secret` argument, `pulumi
config` warns that the value is stored unecrypted and that you can
pass `--secret` to encrypt it. Now, we also mention that `--plaintext`
can be pased explicity on the command line to avoid the warning.

Fixes #752
2018-03-16 12:07:03 -07:00
Justin Van Patten
c7985ed296
Support offline template descriptions (#1044)
Note: This is a minor issue that I didn't get to for M11 that isn't
required for M11 and would be fine merging for post-M11.

When you specify a template name explicitly (e.g.
`pulumi new typescript`), we'll try to download the template tarball
without first downloading the JSON list of available templates. The JSON
includes a description used when replacing the `${DESCRIPTION}` string
in template files. Since we didn't download the JSON, we won't have a
description, so we fallback to a default value (`"A Pulumi project."`).
This also happens when specifying `--offline` to use an existing
template under `~/.pulumi/templates`; we won't have a description for
the template, so we fallback to a default description. The fallback
value happens to be the same as the description for each of our current
templates, so noone will currently notice an issue.

For M11, I included initial support for a template manifest file where
the description (and any future metadata) could be stored, but didn't go
as far as actually reading the file.

This change makes it so the CLI actually reads the description from the
manifest file (if it exists), otherwise falling back to the default
value as is done currently. Some minor related cleanup is included in
this change.
2018-03-13 16:09:25 -07:00
Joe Duffy
fc3908b853
Fix two DST bugs in logs test (#1042)
This fixes two different DST bugs in the logs test, one of which
led to the failure in pulumi/pulumi#1041:

1) The assertion was using ANSIC formatting, which does not contain
   a time-zone, and because we are now in DST, the resulting time
   didn't parse and format in a round-trippable way.  Instead, let's
   use RFC3339, and let's be explicit about the timezone.

2) The parseSince("1m30s") would in theory fail if the DST changed
   at just the right moment, since the reference time could end up
   different from the reference time that the test is using.  Let's
   expose the reference time to the caller, so that they may decide
   how to deal with these situations, and let's use UTC here.
2018-03-12 15:49:52 -07:00
Justin Van Patten
aa940b40dc
Cleanup pulumi new flag descriptions (#1036)
- Use `templates` instead of `releases`
- Fix typo
2018-03-12 11:22:32 -07:00
Justin Van Patten
8906731315
Adds a pulumi new command to scaffold a project (#1008)
This adds a `pulumi new` command which makes it easy to quickly
automatically create the handful of needed files to get started building
an empty Pulumi project.

Usage:

```
$ pulumi new typescript
```

Or you can leave off the template name, and it will ask you to choose
one:

```
$ pulumi new
Please choose a template:
> javascript
  python
  typescript
```
2018-03-09 15:27:55 -08:00
Matt Ellis
db079b1b0a Emit richer events for resource steps
The engine now emits events with richer metadata during the
ResourceOutputs and ResourcePre callbacks. The CLI can then use this
information to decide if it should display the event or not and how
much of the event to display.

Options dealing with what to display and how to display it have moved
into the CLI and the engine now emits all information for each event.
2018-03-09 13:11:42 -08:00
Matt Ellis
4e2f94df95 Remove UpdateOptions.ShowConfig
The engine now unconditionally emits a new type of event, a
PreludeEvent, which contains the configuration for a stack as well as
an indication if the stack is being previewed or updated. The
responsibility for interpreting the --show-config flag on the command
line is now handled by the CLI, which uses this to decide if it should
print the configuration or not, and then writes the "Previewing
changes" or "Deploying chanages" header.
2018-03-09 11:13:06 -08:00
Matt Ellis
731463c282 Have MakeKey fail if namespace contains a colon
This helper method is only really used for testing, but we should not
allow it to create a Key who's namespace has a colon (as ParseKey
would not build something like this).
2018-03-08 11:52:48 -08:00
Matt Ellis
81a273c7bb Change represention of config.Key
config.Key has become a pair of namespace and name. Because the whole
world has not changed yet, there continues to be a way to convert
between a tokens.ModuleMember and config.Key, however now sometime the
conversion from tokens.ModuleMember can fail (when the module member
is not of the form `<package>:config:<name>`).
2018-03-08 10:52:25 -08:00
Matt Ellis
7c39620e9a Introduce config.Key
Right now, config.Key is a type alias for tokens.ModuleMember. I did a
pass over the codebase such that we use config.Key everywhere it
looked like the value did not leak to some external process (e.g a
resource provider or a langhost).

Doing this makes it a little clearer (hopefully) where code is
depending on a module member structure (e.g. <package>:config:<value>)
instead of just an opaque type.
2018-03-08 10:52:25 -08:00
CyrusNajmabadi
c544accfa6
Only attempt to serialize the properties of an object that are actually used. (#1000) 2018-03-07 21:10:12 -08:00
pat@pulumi.com
45a4a41e0d Configure resource providers upon load.
As it stands, we only configure those providers for which configuration
is present. This can lead to surprising failure modes if those providers
are then used to create resources. These changes ensure that all
resource providers that are not configured during plan initialization
are configured upon first load.

Fixes #758.
2018-03-06 16:38:53 -08:00
Sean Gillespie
93cc2cd5cb
Fix some unecessary branches to appease golint (#999) 2018-03-05 16:15:09 -08:00
Joe Duffy
caceeea290
Reintroduce untyped deployments (#996)
By using untyped deployment structures via `json.RawMessage`, we can
support round-tripping between old CLI clients and newer servers, without
dropping possibly-important information on the floor.  I hadn't realized
this design goal with the original system, and after talking to @pgavlin,
I better realized the intent and that we want to preserve this.
2018-03-03 12:12:54 -08:00