Commit graph

2518 commits

Author SHA1 Message Date
Sean Gillespie 28806ac9f3
Fix the protect_resources test (#1228)
Step 3 should also assert that the protected resource that we failed to
delete is still in the checkpoint.
2018-04-18 12:26:46 -07:00
Joe Duffy 3b8d0e6d96
Merge pull request #1148 from pulumi/1081_refresh_cmd
Implement a refresh command
2018-04-18 12:12:33 -07:00
joeduffy bac58d7922 Respond to CR feedback
Incorporate feedback from @swgillespie and @pgavlin.
2018-04-18 11:46:37 -07:00
joeduffy b77403b4bb Implement a refresh command
This change implements a `pulumi refresh` command.  It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.

It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events.  This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state.  This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion).  The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.

Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint.  That
will be coming soon ...
2018-04-18 10:57:16 -07:00
Pat Gavlin c4234527db
Merge pull request #1225 from pulumi/VersionedDeployment
Add a `Version` field to `UntypedDeployment`.
2018-04-17 17:28:03 -07:00
Pat Gavlin a4d6cba664 Add a Version field to UntypedDeployment.
This field indicates the schema of the serialized deployment. This field
behaves identically to the `Version` field of
`PatchUpdateCheckpointRequest`.

This is part of pulumi/pulumi-service#1046
2018-04-17 16:23:20 -07:00
Pat Gavlin a1626aea36
Add a version field to PatchUpdateCheckpointRequest. (#1222)
And make the deployment an opaque JSON message. The verison field
indicates the schema of the deployment. A missing version field will
behave as if the version was set to `1`. A version of `1` indicates that
the serialized deployment has the `DeploymentV1` schema.

This is part of pulumi/pulumi-service#1046.
2018-04-17 15:58:20 -07:00
Pat Gavlin fb55f3aab6
Add version suffixes to more API types. (#1215)
This covers most of the transitive closure of the types that appear in a
checkpoint. A breaking change to any of these types implies a bump in
the checkpoint version number.

This is part of pulumi/pulumi-service#1046.
2018-04-17 10:38:47 -07:00
CyrusNajmabadi 50dafdae6a
Simplify the diff for primitive values. (#1212) 2018-04-17 01:51:53 -07:00
CyrusNajmabadi 4a5e72fb16
Progress tweaks (#1219) 2018-04-16 23:41:00 -07:00
Sean Gillespie 55711e4ca3
Revert "Lift snapshot management out of the engine and serialize writes to snapshot (#1069)" (#1216)
This reverts commit 2c479c172d.
2018-04-16 23:04:56 -07:00
CyrusNajmabadi 0bd3036115
Small progress tweaks. (#1218) 2018-04-16 19:46:57 -07:00
Joe Duffy 2a9cf66ed0
Merge pull request #1209 from pulumi/1203_dont_use_instanceof
Don't use `instanceof` for RTTI
2018-04-16 18:41:34 -07:00
joeduffy 4e9ff55def Change login prompt text and enable no-echo
This makes two minor tweaks to the login prompt:

1. Change the text so that it hyperlinks in most terminals, including
   iTerm, in a way that doesn't include excess characters.

2. Disable echoing of the token.
2018-04-16 17:43:37 -07:00
Matt Ellis a55d3f9d7e Make diff tests pass when running against a local service
In cases where we are running against a local service, the CLI does
not print a Permalink line when updating a stack, because we can not
determine what the URL for the link would be. This breaks the diff
tests which need to clean the CLI output and compare them to known
values, since the output now has one less line than expected.

Update the test's cleaning logic to handle this case.
2018-04-16 16:38:39 -07:00
CyrusNajmabadi 4dbb56d142
Just the minimal change to unbreak the client requesting a preview from the service. (#1214) 2018-04-16 15:34:59 -07:00
joeduffy d5608f2fee Use x is T return types for isX functions
This change adopts `x is T` style of RTTI inquiry, which fits much
more nicely with TypeScript's typechecking flow.

Thanks to @lukehoban for teaching me a new trick today! :-)
2018-04-16 15:03:23 -07:00
CyrusNajmabadi b4b11991a2
Remove unused param. (#1211) 2018-04-16 14:42:36 -07:00
joeduffy 6f4423895c Don't use instanceof for RTTI
This change moves us away from using JavaScript RTTI, by way of
`instanceof`, for built-in Pulumi types.  If we use `instanceof`,
then the same logical type loaded from separate copies of the
SDK package -- as will happen in SxS scenarios -- are considered
different.  This isn't actually what we want.  The solution is
simple: implement our own quasi-RTTI solution, using __pulumi*
properties and manual as* and is* functions.  Note that we could
have skipped the as* and is* functions, but I found that they led
to slightly easier to read code.

There is one strange thing in here, which I spoke to
@CyrusNajmabadi about: SerializedOutput<T>, because it implements
Output<T> as an _interface_, did not previously masquerade as an
actual Output<T>.  In other words, `instanceof` would have returned
false, and indeed a few important properties (like promise) are
missing.  This change preserves that behavior, although I'll admit
that this is slightly odd.  I suspect we'll want to revisit this as
part of https://github.com/pulumi/pulumi/issues/1074.

Fixes https://github.com/pulumi/pulumi/issues/1203.
2018-04-16 14:08:10 -07:00
Joe Duffy 10f4f2c7c4
Revert "Temporarily work around pulumi/pulumi#1147 (#1161)" (#1207)
This reverts commit 9c243720a6.
2018-04-16 12:29:52 -07:00
Joe Duffy d3ef06fd15
Fix argless plugin install (#1204)
A previous change altered the calculation of plugin sizes broke the
ability to install plugins that weren't backed by a directory.  This
fixes that by making getPluginSize act correctly for any kind of path.
2018-04-16 09:26:46 -07:00
CyrusNajmabadi 541b8b3f4e
Switch to a new grid-view for the progress display. (#1201) 2018-04-15 12:47:53 -07:00
Sean Gillespie 246aaba00e
Only publish packages on the linux v8.11 node leg (#1205) 2018-04-14 22:25:35 -07:00
Joe Duffy 0d79d2106c
Ensure pip 10.0.0 is installed (#1199)
This change installs pip 10.0.0 in all three places it might come up:

1) Machine-wide packages.

2) User-local packages.

3) In the isolated virtualenv.
2018-04-14 16:43:48 -07:00
Joe Duffy f8afe0ed41
Merge pull request #1197 from pulumi/ellismg/fix-1196
Fix plugin loading
2018-04-14 14:02:36 -07:00
Sean Gillespie 1f5be5f8cd
Bring the magic of Pulumi to Node v8.11.1, v9.11.1, v6.10.3 (#1167)
* Introduce a simple repetition operator to match expected error messages against actual ones

* Convert required and optional objects to use a Map (node v9 compat), improve the error formatting for failed tests

* Test node v6, v8, and v9 in CI

* Get rid of PULUMI_API env in .travis.yml, it's set from the Travis console now
2018-04-14 11:50:01 -07:00
Joe Duffy fe45eb875e
Don't prompt for stack removal (#1198)
We already had logic to skip prompting a user to create a stack,
when a stack-specific command was run but none was found, but we
only heeded it in one of two cases.  This fixes the other case.
2018-04-14 10:01:39 -07:00
CyrusNajmabadi f2b9bd4b13
Remove the explicit 'pulumi preview' command. (#1170)
Old command still exists, but tells you to run "pulumi update --preview".
2018-04-13 22:26:01 -07:00
Chris Smith 3470acec02
Add continuation token (for reals) (#1099) 2018-04-13 19:10:34 -07:00
Matt Ellis 23be6e16ab Fix plugin loading
Due to an interaction between pointers and go's `range` operator, we
would end up always returning the last plugin in a user's plugin
cache, instead of the right value.

We now save the current plugin into a local, so if we end up taking
the address the right thing will happen.

Fixes #1196
2018-04-13 18:42:36 -07:00
CyrusNajmabadi 8b4f3a43ec
Don't synthesize a new error to display when we've already emitted a diagnostic error. (#1193) 2018-04-13 16:25:24 -07:00
CyrusNajmabadi 7b96b8cdcf
Produce a single message for the text we receive when running, not a message per line of output. (#1191) 2018-04-13 15:44:35 -07:00
Matt Ellis 682c1c0b45 Fix windows publishing 2018-04-13 14:56:52 -07:00
Matt Ellis ca2fbca13f Continue to add native modules to NODE_PATH
While we no longer use the native runtime module, older versions of
@pulumi/pulumi still require it. Let's continue to have the launcher
put the native module location on the `$PATH`. And we'll include them
in the SDK for a while longer.

Fixes #1177
2018-04-13 14:26:32 -07:00
Joe Duffy 150f57168a
Fix SxS config (#1175)
We weren't properly lazily loading everywhere we need to.
2018-04-13 11:26:01 -07:00
Matt Ellis f4ca4959dd Fix logging error in GetPluginPath
If version was nil, we would assert when trying to log a message. We
were hitting this on windows when trying to destroy a stack.

Fixes #1168
2018-04-12 15:53:23 -07:00
Matt Ellis aa98e54239 Add pointer to docs.pulumi.com in the help text
Fixes #989
2018-04-12 13:22:06 -07:00
Matt Ellis 12860cb81f Fix typo in Windows script
`git` is spelled `git` not `get`
2018-04-12 11:42:42 -07:00
Matt Ellis 5eb199bf02 Fix SDK Publishing after build
- Install `jq` on OSX
- Clone pulumi/sdk into the right location
2018-04-12 11:23:36 -07:00
Joe Duffy 9c243720a6
Temporarily work around pulumi/pulumi#1147 (#1161)
This reverts back to our old diff behavior, temporarily, while we
work on a fix to pulumi/pulumi#1147 and validate that it works broadly.
2018-04-12 10:59:25 -07:00
CyrusNajmabadi 8370a0a38e
Improve how we print out failures when doing progress updates. (#1159)
Preemptively merging in.  Let me know if there are any changes you want me to make.
2018-04-12 10:56:39 -07:00
Matt Ellis 608af9349d Publish SDK per build
After the move to stop including packages in the SDK, we no longer
published an SDK per build. This corrects this. Since the only things
in the SDK today are the language plugins and the CLI itself, we can
publish an SDK per build from pulumi/pulumi.

This change re-uses the existing infrastructure we have in
pulumi/sdk.

Fixes #1076
2018-04-12 10:11:56 -07:00
Sean Gillespie 2c479c172d
Lift snapshot management out of the engine and serialize writes to snapshot (#1069)
* Lift snapshot management out of the engine

This PR is a prerequisite for parallelism by addressing a major problem
that the engine has to deal with when performing parallel resource
construction: parallel mutation of the global snapshot. This PR adds
a `SnapshotManager` type that is responsible for maintaining and
persisting the current resource snapshot. It serializes all reads and
writes to the global snapshot and persists the snapshot to persistent
storage upon every write.

As a side-effect of this, the core engine no longer needs to know about
snapshot management at all; all snapshot operations can be handled as
callbacks on deployment events. This will greatly simplify the
parallelization of the core engine.

Worth noting is that the core engine will still need to be able to read
the current snapshot, since it is interested in the dependency graphs
contained within. The full implications of that are out of scope of this
PR.

Remove dead code, Steps no longer need a reference to the plan iterator that created them

Fixing various issues that arise when bringing up pulumi-aws

Line length broke the build

Code review: remove dead field, fix yaml name error

Rebase against master, provide implementation of StackPersister for cloud backend

Code review feedback: comments on MutationStatus, style in snapshot.go

Code review feedback: move SnapshotManager to pkg/backend, change engine to use an interface SnapshotManager

Code review feedback: use a channel for synchronization

Add a comment and a new test

* Maintain two checkpoints, an immutable base and a mutable delta, and
periodically merge the two to produce snapshots

* Add a lot of tests - covers all of the non-error paths of BeginMutation and End

* Fix a test resource provider

* Add a few tests, fix a few issues

* Rebase against master, fixed merge
2018-04-12 09:55:34 -07:00
Joe Duffy 9ba6584bbf
Add a --verbose flag to plugin install (#1160)
To help us diagnose various issues -- and also just as a nice status
reporting thing -- we'll add a --verbose flag to the pulumi plugin
install command.  This will get used in the package scripts.
2018-04-11 17:51:14 -07:00
Matt Ellis 50843a98c1 Retry some HTTP operations
We've seen failures in CI where DNS lookups fail which cause our
operations against the service to fail, as well as other sorts of
timeouts.

Add a set of helper methods in a new httputil package that helps us do
retries on these operations, and then update our client library to use
them when we are doing GET requests. We also provide a way for non GET
requests to be retried, and use this when updating a lease (since it
is safe to retry multiple requests in this case).
2018-04-11 14:58:25 -07:00
Luke Hoban 880fc2d9f9
Ensure that we create legal stack names for integration tests. (#1152) 2018-04-11 12:17:31 -07:00
Chris Smith ab2385437a
Validate stack properties like names, runtime, etc. (#1146)
* Validate stack properties like names, runtime, etc.

* Fix build error
2018-04-11 10:08:32 -07:00
Joe Duffy 479a2e6ad5
Add an ID property to ReadResponse (#1145)
The RPC provider interface needs a way to convey back to the engine
that a resource being read no longer exists.  To do this, we'll return
the ID property that was read back.  If it is empty, it means the
resource is gone.  If it is non-empty, we expect it to match the input.
2018-04-10 12:58:50 -07:00
CyrusNajmabadi a759f2e085
Switch to a resource-progress oriented view for pulumi preview/update/destroy (#1116) 2018-04-10 12:03:11 -07:00
Matthew Riley 085e8fb412
Merge pull request #1143 from pulumi/use-aws-creds
Don't cache AWS sessions with different credentials
2018-04-10 10:40:00 -07:00