Commit graph

1369 commits

Author SHA1 Message Date
Matt Ellis 8a6ffd08b2 Use crypto/rand when generating a stack suffix
math/rand uses a fixed seed, meaning that across runs the Kth call to
`rand.Int63()` will always return the same value.

Given that we want to provide a unique suffix across multiple
concurrent runs, this isn't the behavior we want.

I saw an instance fail in CI where all three legs ran the test
concurrently and they raced on creating the test stack, since they all
generated the same name.
2018-06-05 11:06:01 -07:00
Matt Ellis 4f28f3d44d Allow overriding config location
Now that the "config" member of Pulumi.yaml has been deprecated for a
while, we'll change it's meaning. When set, the value is treated as a
path and joined with the path to Pulumi.yaml, and per stack
configuration is stored in that folder

Fixes #1031
2018-06-05 09:26:48 -07:00
Chris Smith ebfc3458d5
Export GetCommandResults from Environment (#1436) 2018-06-05 08:34:06 -07:00
Matt Ellis 4ac1a2f355 Pull tracing and reporting information from the environment
This lets us set these values globaly, in our Travis and AppVeyor
configurations instead of forcing every test to opt-in. It also means
that by default, local builds will not report any of this data (and
will not need access to these endpoints).
2018-06-04 14:28:52 -07:00
Matthew Riley d8358f7be3
Merge pull request #1453 from pulumi/retry-post
Send request body on retries
2018-06-04 11:32:19 -07:00
Sean Gillespie 1a2d4d9ff1
Fix an issue where the Version field of an UntypeDeployment was lost (#1450)
The Version field was inadvertently dropped when sending an import
request to the service. Now that we are requiring that the Version field
be set in deployments, this was causing errors.
2018-06-03 14:30:51 -07:00
Matthew Riley 71e9d8c2af Use bytes.Reader instead of bytes.Buffer
Calling `bytes.NewBuffer` gives up ownership of the backing memory and creates
a _mutable_ buffer. We don't need that, so let's use `bytes.NewReader` instead.
2018-06-03 01:33:50 -07:00
Matthew Riley e7498f9468 Send request body on retries
Not sending request body on retry is bad enough, but we keep the original
`Content-Length` and the server ends up expecting a body we never send.
This results in `400 Bad Request` errors from the ALB.
2018-06-03 01:33:50 -07:00
Matthew Riley 261fa86586
Merge pull request #1358 from pulumi/retry-bugs
Fix two bugs in retries
2018-06-01 17:11:23 -07:00
Chris Smith 485bb35180
Relax stack name requirements (#1381)
* Relax stack name requirements

* Add error if stack name too long

* Max tag length is 256 chars
2018-05-29 13:52:11 -07:00
Pat Gavlin da9bd9dbff
Merge pull request #1430 from pulumi/AlwaysPrompt
Always prompt for confirmation after a preview.
2018-05-29 09:47:00 -07:00
Sean Gillespie 924c49d7e0
Fail fast when attempting to load a too-new or too-old deployment (#1382)
* Error when loading a deployment that is not a version that the CLI understands

* Add a test for 'pulumi stack import' on a badly-versioned deployment

* Move current deployment version to 'apitype'

* Rebase against master

* CR: emit CLI-friendly error message at the two points outside of the engine calling 'DeserializeDeployment'
2018-05-25 13:29:59 -07:00
Sean Gillespie fe22ade275
Differentiate between warnings and errors when sending an event (#1424)
* Differentiate between warnings and errors when sending an event

* CR: always write severity field if the event is a diagnostic
2018-05-25 13:29:33 -07:00
Sean Gillespie 58ca6f5f19
Fix a confusing error message arising from multiple API calls failing in the same way (#1432) 2018-05-24 15:36:35 -07:00
Pat Gavlin 31739c8840 Always prompt for confirmation after a preview.
An empty preview is not a guarantee that a subsequent update will cause
no changes. As such, we should always offer the user the option to
perform an update even if the preview proposes no changes.

Note that this indirectly fixes #1410 by masking the effects of #1429.
2018-05-24 12:22:20 -07:00
Matt Ellis 9a8f8881c0 Show manifest information for stacks
This change supports displaying manifest information for a stack and
changes the way we handle Snapshots in our backend.

Previously, every call to GetStack would synthesize a Snapshot by
taking the set of resources returned from the
`/api/stacks/<owner>/<name>` endpoint, combined with an empty
manfiest (since the service was not returning the manifest).

This wasn't great for two reasons:

1. We didn't have manifest information, so we couldn't display any of
   its information (most important the last updated time).

2. This strategy required that the service return all the resources
   for a stack anytime GetStack was called. While the CLI did not
   often need this detailed information the fact that we forced the
   Service to produce it (which in the case of stack managed PPC would
   require the service to talk to yet another service) creates a bunch
   of work that we end up ignoring.

I've refactored the code such that `backend.Stack`'s `Snapshot()` method
now lazily requests the information from the service such that we can
construct a `Snapshot()` on demand and only pay the cost when we
actually need it.

I think making more of this stuff lazy is the long term direction we
want to follow.

Unfortunately, right now, it means in cases where we do need this data
we end up fetching it twice. The service does it once when we call
GetStack and then we do it again when we actually need to get at the
Snapshot.  However, once we land this change, we can update the
service to no longer return resources on the apistack.Stack type. The
CLI no longer needs this property.  We'll likely want to continue in a
direction where `apistack.Stack` can be created quickly by the
service (without expensive database queries or fetching remote
resources) and just add additional endpoints that let us get at the
specific information we want in the specific cases when we want it
instead of forcing us to return a bunch of data that we often ignore.

Fixes pulumi/pulumi-service#371
2018-05-23 16:43:34 -07:00
Sean Gillespie 1a51507206
Delete Before Create (#1365)
* Delete Before Create

This commit implements the full semantics of delete before create. If a
resource is replaced and requires deletion before creation, the engine
will use the dependency graph saved in the snapshot to delete all
resources that depend on the resource being replaced prior to the
deletion of the resource to be replaced.

* Rebase against master

* CR: Simplify the control flow in makeRegisterResourceSteps

* Run Check on new inputs when re-creating a resource

* Fix an issue where the planner emitted benign but incorrect deletes of DBR-deleted resources

* CR: produce the list of dependent resources in dependency order and iterate over the list in reverse

* CR: deps->dependents, fix an issue with DependingOn where duplicate nodes could be added to the dependent set

* CR: Fix an issue where we were considering old defaults and new inputs
inappropriately when re-creating a deleted resource

* CR: save 'iter.deletes[urn]' as a local, iterate starting at cursorIndex + 1 for dependency graph
2018-05-23 14:43:17 -07:00
Pat Gavlin 38d3b83494
Allow tests to perform changes in empty updates. (#1411)
This may be a useful knob to have.
2018-05-23 09:17:10 -07:00
Pat Gavlin f3201baeb5
Fix {computed,output}<> display in preview. (#1409)
At some point the summary and preview parameters to
`engine.GetResourcePropertiesDetails` were flipped s.t. we stopped
rendering computed<> and output<> values properly during previews.
2018-05-22 17:20:03 -07:00
Pat Gavlin 37a3317623
Put local backend support under PULUMI_DEBUG_COMMANDS. (#1408)
Just what it says on the tin.

Fixes #1398.
2018-05-22 17:02:45 -07:00
joeduffy 5967259795 Add license headers 2018-05-22 15:02:47 -07:00
Matt Ellis 0732b05c5d Remove pulumi init
`pulumi init` was part of our old identity model with the service and
is no longer used. We can now delete this code.

Fixes #1241
2018-05-22 13:37:08 -07:00
Pat Gavlin b5efe40f04
Fold "global" messages under the stack resource. (#1406)
We currently display at most two rows for messages that are not
associated with a URN: one for those that arrive before the stack
resource is first observed and one for those that arrive after the stack
resource is first observed. The former is labeled "global"; the latter
is the row associated with the stack resource. These changes remove the
duplication by reassociating the row used for events with no URN that
arrive before the stack resource with the stack resource once it has
been observed.

Fixes #1385.
2018-05-22 13:30:52 -07:00
Pat Gavlin c08ede339f Only display preview/no-op resource rows once on non-TTYs.
Our current strategy for the progress display on non-TTYs causes us to
display multiple identical rows for each resource when the row is a
preview or a no-op. This behavior is not particularly useful, and
generally just makes the display noisier than it needs to be.

These changes avoid displaying resource rows without meaningful output
by supressing the display of resource output events that are delivered
during a preview or that correspond to a no-op update.

Fixes #1384.
2018-05-22 11:35:07 -07:00
joeduffy 614a2cdeb3 Simplify previews, initialize plugin events
This changes two things:

1) Eliminates the fact that we had two kinds of previews in our engine.

2) Always initialize the plugin.Events, to ensure that all plugin loads
   are persisted no matter the update type (update, refresh, destroy),
   and skip initializing it when dryRun == true, since we won't save them.
2018-05-18 14:58:06 -07:00
joeduffy 4e9b228089 Don't pass PluginEvents for refresh
The PluginEvents will now try to register loaded plugins which,
during a refresh preview, will result in attempting to save mutations
when a token is missing.  This change mirrors the changes made to
destroy which avoid it panicing similarly, by simply leaving
PluginEvents unset.  Also adds a bit of tracing that was helpful to
me as I debugged through the underlying issues.

Fixes #1377.
2018-05-18 13:54:23 -07:00
Sean Gillespie 7b7870cdaa
Remove unused stack name from deploy.Snapshot (#1386) 2018-05-18 11:15:35 -07:00
Matthew Riley 4398295954 Use select to notice timeouts
Before we were reading a boolean variable that was set by another goroutine.
Without any sync primitives, I think Go is within its rights to optimize
that away.

We now require that `ctx` is non-`nil`. This was already true for all callers
and consistent with other Go functions that take contexts.
2018-05-17 13:51:00 -07:00
Matthew Riley 77d9cd809a Avoid leaking connections when retry fails 2018-05-17 13:51:00 -07:00
Joe Duffy a1d806321e
Enable refresh for managed stacks (#1380) 2018-05-17 11:44:39 -07:00
Sean Gillespie 68911900fd
Graceful shutdown (#1320)
* Graceful RPC shutdown: CLI side

* Handle unavailable resource monitor in language hosts

* Fix a comment

* Don't commit package-lock.json

* fix mangled pylint pragma

* Rebase against master and fix Gopkg.lock

* Code review feedback

* Fix a race between closing the callerEventsOpt channel and terminating a goroutine that writes to it

* glog -> logging
2018-05-16 15:37:34 -07:00
Pat Gavlin 009096239c
Merge pull request #1378 from pulumi/SortPropertyKeys
Sort property keys in the progress display.
2018-05-16 11:28:11 -07:00
Pat Gavlin 3cc8758b85 Sort property keys in the progress display.
This prevents the "updated keys" list from being arbitrarily reordered
upon refresh.
2018-05-16 11:10:52 -07:00
Pat Gavlin 64d90b263d
Merge pull request #1375 from pulumi/ResourceChangesFromBackend
Validate empty previews and updates in tests.
2018-05-16 10:44:48 -07:00
Pat Gavlin 05ef4fecb9 Validate empty previews and updates in tests.
Fixes #1154.
2018-05-15 18:03:30 -07:00
Pat Gavlin e3020e820b Expose change summaries from the backend.
This is a smallish refactoring that exposes the resource change
summaries reported by the engine from the relevant backend methods.
2018-05-15 17:44:35 -07:00
CyrusNajmabadi ad086a0dbd
Add test for filters. Also, filter anything that goes through our sinks. (#1373) 2018-05-15 16:09:16 -07:00
CyrusNajmabadi 72e00810c4
Filter the logs we emit to glog so that we don't leak out secrets. (#1371) 2018-05-15 15:28:00 -07:00
Matt Ellis 387fffd740 Allow retry of REST calls during a deployment
During a deployment, we end up making a bunch of PATCH/PUT/POST style
REST requests. For these calls, it should be safe to retry the
operation if there was a hickup during our REST call, so mark them as
retryable.
2018-05-15 13:08:03 -07:00
Pat Gavlin 016ae4acba
Add pre/post-update hooks to the test framework. (#1369)
These changes add support for pre- and post-`pulumi` callbacks to the
integration test framework. These callbacks will be invoked immediately
before and after (respectively) running a `pulumi` command.
2018-05-15 09:48:56 -07:00
Joe Duffy 369c619ab9
Skip loading language plugins when not needed (#1367)
In pulumi/pulumi#1356, we observed that we can fail during a destroy
because we attempt to load the language plugin, which now eagerly looks
for the @pulumi/pulumi package.

This is also blocking ingestion of the latest engine bits into the PPC.

It turns out that for destroy (and refresh), we have no need for the
language plugin.  So, let's skip loading it when appropriate.
2018-05-14 20:32:53 -07:00
Pat Gavlin 782a869765
Add support for passing tracing headers. (#1360)
These changes add support for adding a tracing header to API requests
made to the Pulumi service. Setting the `PULUMI_TRACING_HEADER`
environment variable or enabling debug commands and passing the
`--tracing-header` will change the value sent in this header. Setting
this value to `1` will request that the service enable distributed
tracing for all requests made by a particular CLI invocation.
2018-05-14 13:44:44 -07:00
Matt Ellis 6845f9ed20 Add pulumi config refresh to fetch most recent configuration
The newly added `pulumi config refresh` updates your local copy of the
Pulumi.<stack-name>.yaml file to have the same configuration as the
most recent deployment in the cloud.

This can be used in a varirty of ways. One place we plan to use it is
in automation to clean up "leaked" stacks we have in CI. With the
changes you'll now be able to do the following:

```
$ cd $(mktemp -d)
$ echo -e "name: who-cares\nruntime: nodejs" > Pulumi.yaml
$ pulumi stack select <leaked-stack-name>
$ pulumi config refresh -f
$ pulumi destroy --force
```

Having a simpler gesture for the above is something we'll want to do
long term (we should be able to support `pulumi destory <stack-name>`
from a completely empty folder, today you need a Pulumi.yaml file
present, even if the contents don't matter).

But this gets us a little closer to where we want to be and introduces
a helpful primitive in the system.

Contributes to #814
2018-05-14 10:28:42 -07:00
Joe Duffy 457c34ff50
Add a PULUMI_DEV flag, and suppress warnings (#1361)
This change suppresses the warning

    warning: resource plugin aws is expected to have version >=0.11.3,
        but has 0.11.1-dev-1523506162-g06ec765; the wrong version may
        be on your path, or this may be a bug in the plugin

when the PULUMI_DEV envvar is set to a truthy value.

This warning keeps popping up in demos since I'm always using dev builds
and I'd like a way to shut it off, even though this can legitimately
point out a problem.  Eventually I'll switch to official buildsa but,
until then, it seems worth having a simple way to suppress.
2018-05-11 20:59:01 -07:00
CyrusNajmabadi 3e787779e5
Fix issue where replacements were not shown in the progress view. (#1357) 2018-05-11 16:48:05 -07:00
CyrusNajmabadi e94e358ddf
Fix small logic bug when printing out messages. (#1348) 2018-05-09 14:25:03 -07:00
Pat Gavlin 97e38bddc8 Enable distributed tracing.
These changes add support for injecting client tracing spans into HTTP
requests to the Pulumi API. The server can then rematerialize these span
references and attach its own spans for distributed tracing.
2018-05-09 11:43:09 -07:00
Matt Ellis c9db62060b Fix preview against a PPC
This will unblock `pulumi preview` against a PPC, which regressed
recently (c5b702e0ff was the likely
cause).
2018-05-08 21:43:17 -07:00
Pat Gavlin 0c04770f5e
Make tokenSource.Close() synchronous. (#1345)
Without these changes, it is possible for a token renewal to race with
CLI shutdown, which appears to be the cause of #1344.

Fixes #1344.
2018-05-08 15:28:42 -07:00
Pat Gavlin bb5b7da650 Revert "Revert the changes from #1261." 2018-05-08 11:46:15 -07:00