Commit graph

2411 commits

Author SHA1 Message Date
Pat Gavlin 5c0b62e1aa
Serialize resource registration after inputs resolve. (#882)
As it stands, we serialize more than is correct when registering
resources: in addition to serializing the RegisterResource RPC, we also
wait for input properties to resolve in the same context. Unfortunately,
this means that we can create cycles in the promise graph when a
resource A is constructed in an earlier turn than some resource B and
one of B's output properties is an input to resource A. These changes
fix this issue by allowing input properties to resolve *before*
serializing the RegisterResource RPC.

Some integration tests had taken a dependency on the ordering of resources in
either the output of the `pulumi` command or the checkpoint file. The
only test that took a dependency on command output was updated s.t. its
resources have exactly one legal topographical sort (and therefore their
ordering is deterministic). The other tests were updated s.t. their
validation did not depend on resource ordering.
2018-02-05 16:29:20 -08:00
CyrusNajmabadi 275670692b
Introduce Output<T> and update Resource construction code to properly handle it. (#834)
This PR adds a new formalisms at the Resource layer.  First all inputs to a Resource are typed as ```Input<T>```.  This is either a T, ```Promise<T>``
2018-02-05 14:44:23 -08:00
Chris Smith c408068075
Add User-Agent header to CLI requests (#881)
Today we don't send any version information with API requests to the service, so we cannot make breaking changes between versions of the backend API while preserving backwards compatibility.

This PR adds a `User-Agent` header with REST requests that sends a CLI version number of "1". If the service were to make a breaking change, it could use this header to determine which response handler to use. (e.g. return a different response for "" or "1" and another for "2".) Obviously we want to avoid being in this situation, but in the event that we need to make a breaking change, we'll need this value.

We send the Pulumi version as well, though the SDK will probably rev much more quickly than the backend API client version.

Fixes #848
2018-02-05 09:38:00 -08:00
Matthew Riley 1ff84c1991
Merge pull request #874 from pulumi/keep-on-failure
Keep test files on assertion failure
2018-02-02 22:52:37 -08:00
Matthew Riley 606c45ffea Keep test files on assertion failure
We were pretty careful to keep the test directory around if the test ever
exited early due to a panic or error return. But if the test ran to
completion and failed -- for example, if ExtraRuntimeValidation caused the
test to fail -- we would end up deleting the test directory.

Fixes #868
2018-02-01 23:35:49 -08:00
Matt Ellis 818246a708 Allow control of uploaded archive root in Pulumi.yaml
Previously, when uploading a projectm to the service, we would only
upload the folder rooted by the Pulumi.yaml for that project. This
worked well, but it meant that customers needed to structure their
code in a way such that Pulumi.yaml was always as the root of their
project, and if they wanted to share common files between two projects
there was no good solution for doing this.

This change introduces an optional piece of metadata, named context,
that can be added to Pulumi.yaml, which allows controlling the root
folder used for computing the root folder to archive from.  When it is
set, it is combined with the location of the Pulumi.yaml file for the
project we are uploading and that folder is uses as the root of what
we upload to the service.

Fixes: #574
2018-01-31 16:22:58 -08:00
Matt Ellis 39dbdc98e9 Clean up colorization logic
The existing logic would flow colorization information into the
engine, so depending on the settings in the CLI, the engine may or may
not have emitted colorized events. This coupling is not great and we
want to start moving to a world where the presentation happens
exclusively at the CLI level.

With this change, the engine will always produce strings that have the
colorization formatting directives (i.e. the directives that
reconquest/loreley understands) and the CLI will apply
colorization (which could mean either running loreley to turn the
directives into ANSI escape codes, or drop them or retain them, for
debuging purposes).

Fixes #742
2018-01-31 15:46:14 -08:00
Joe Duffy 8ea0133d0e
Copy the package.json with its semver expanded (#864)
Our scripts currently copy the package.json that does *not* have
the expanded semver, so its version is simply "${VERSION}", and NPM
is very much not happy with that.  We can just stop copying the
package.json explicitly since it's inside of the bin/ directory.
2018-01-31 10:34:21 -08:00
Matt Ellis b1496f3051 Remove Document and Location
These types are no longer used as pulumi no longer reads and evaluates
source code.

Contributes to #441
2018-01-30 16:42:39 -08:00
Sean Gillespie ff987c65e5
Updates to the readme: fix up some documentation regarding the Makefile (#861) 2018-01-30 16:31:02 -08:00
Matt Ellis 7ac921f938 Add ignore to linter 2018-01-30 14:46:44 -08:00
Matt Ellis 4422700f0f Run yarn upgrade and commit all resulting lockfiles
This also adds lock files for some of our tests which we previously
did not commit.
2018-01-30 14:46:44 -08:00
Matt Ellis e00f782301 Remove yarn.lock from .gitignore 2018-01-30 14:46:44 -08:00
Chris Smith 63dfd1e302
Add Manifest to GetStackResponse (#854)
Add Manifest to GetStackResponse.
2018-01-30 14:12:11 -08:00
Luke Hoban 0e4fcc16e3
Pass target Cloud and PPC to integation test Reporter (#859)
Ensure that we capture this information in our test reporting so that we can filter on it in queries and reports.
2018-01-30 13:10:32 -08:00
Matthew Riley 27ab056cc7
Merge pull request #858 from pulumi/multiline-log
Keep embedded newlines in Lambda log messages
2018-01-30 13:00:04 -08:00
Matthew Riley b0905d99aa Fix long line 2018-01-30 10:52:42 -08:00
Matthew Riley c74db59231 Keep embedded newlines in Lambda log messages
Set the `s` flag in the regexp so `.` matches `\n`.

Add an explicit `\n` to the regexp to account for the newline inserted by Lambda.
2018-01-29 23:33:59 -08:00
Matt Ellis 07fd36c369 Remove deployOptions.Create
This was entirely unused.

Fixes #813
2018-01-29 16:23:55 -08:00
CyrusNajmabadi 6135a41643
Restore previous marshalling logic for Ids. (#852) 2018-01-29 13:29:45 -08:00
Matt Ellis 70f9dc0a97 Disable yarn cache in Travis 2018-01-29 11:49:42 -08:00
Matt Ellis 803ba93bf5 Pass --network-concurrency 1 to yarn
We've been hitting issues in CI that look like yarnpkg/yarn#4563 and
our hope is that this addresses the issue
2018-01-29 11:49:42 -08:00
joeduffy e7b1fc1aac Remove incorrect punctuation and newline in help text 2018-01-27 09:40:05 -08:00
Pat Gavlin bd4cd2fda9
Merge pull request #845 from pulumi/SetDryRun
Always set UpdateOptions.DryRun for previews.
2018-01-26 17:52:33 -08:00
Pat Gavlin 842a16cee4 Always set UpdateOptions.DryRun for previews.
If we do not do this, the engine will assume that it should elide
unknown input/output properties when interacting with the language
and resource providers and we will produce unexpected results.
2018-01-26 16:05:42 -08:00
Matt Ellis 0ebfa0570c Build tags in addition to master
Previously, we used safelisting to ensure we only build the master
branch on push jobs. Because Travis treated branches and tags as the
same thing when it came to safelisting, it meant that pushes of tags
did not cause builds.

Our versioning workflow requires that we rebuild on tag pushes,
because the tag plays into the version number we embed in our programs
and packages.

Instead, we move to the new Conditional Builds feature of Travis to
also allow us to build tags.

Part of pulumi/sdk#11
2018-01-26 14:45:52 -08:00
CyrusNajmabadi 6f8a74f09f
wrap long comments. (#842) 2018-01-25 18:50:58 -08:00
CyrusNajmabadi afca512bee
Simplify how we do async/await in closure synchronization. (#841) 2018-01-25 17:53:37 -08:00
CyrusNajmabadi 1df66df250
Further simplification of resource RPC requests. (#840) 2018-01-25 15:26:39 -08:00
Chris Smith 001d187c90
Add not-started UpdateResult
Add a "not-started" `UpdateResult` type, for supporting updates tracked by the PPC that have not been applied yet.
2018-01-25 14:23:49 -08:00
CyrusNajmabadi f35f991140
Extract out serialization functionality from transfer function. (#835) 2018-01-25 13:34:21 -08:00
Chris Smith 4c217fd358
Add "pulumi history" command (#826)
This PR adds a new `pulumi history` command, which prints the update history for a stack.

The local backend stores the update history in a JSON file on disk, next to the checkpoint file. The cloud backend simply provides the update metadata, and expects to receive all the data from a (NYI) `/history` REST endpoint.

`pkg/backend/updates.go` defines the data that is being persisted. The way the data is wired through the system is adding a new `backend.UpdateMetadata` parameter to a Stack/Backend's `Update` and `Destroy` methods.

I use `tests/integration/stack_outputs/` as the simple app for the related tests, hence the addition to the `.gitignore` and fixing the name in the `Pulumi.yaml`.

Fixes #636.
2018-01-24 18:22:41 -08:00
Matt Ellis ce05cce77f Provide a rudimentary progress spinner
Previously, the `pulumi` tool did not show any indication of progress
when doing a deployment. Combined with the fact that we do not create
resources in parallel it meant that sometime `pulumi` would appear to
hang, when really it was just waiting on some resource to be created
in AWS. In addition, some AWS resources take a long time to create and
CI systems like travis will kill the job if there is no output. This
causes us (and our customers) to have to do crazy dances where we
launch shell scripts that write a dot to the console every once in a
while so we don't get killed. While we plan to overhaul the output
logic (see #617), we take a first step towards interactivity by simply
having a nice little spinner (in the interactive case) and when run
non interactive have `pulumi` print a message that it is still
working.

Fixes #794
2018-01-22 14:21:08 -08:00
Joe Duffy e50af35d8d
Merge pull request #829 from pulumi/FixPreview
Fix `pulumi preview` by setting `UpdateOptions.DryRun`.
2018-01-22 12:28:35 -08:00
Pat Gavlin 172a192e52 Add a test for derived inputs. 2018-01-22 11:47:14 -08:00
Pat Gavlin 600b267318 Fix pulumi preview by setting UpdateOptions.DryRun.
The engine will use this flag to decide whether or not to provide
undefined input property values to resource providers. This is
important because an input property can be defined if it is sourced from
another resource's output property that is not known to be stable (i.e.
that property is not known to be consistent between preview and apply).
Failing to provide these undefined values can then cause input
validation to fail.
2018-01-22 11:20:00 -08:00
Matthew Riley 4bb86116d0
Merge pull request #821 from pulumi/recover-in-test
Avoid resource leaks by recovering from panics in integration tests
2018-01-22 10:08:15 -08:00
Joe Duffy d6c798ee52
Add a commentChars parameter to EmitHeaderWarning (#827)
Now that we're using this to generate multiple languages, this
needs to be parameterized.  Splitting out into its own change.
2018-01-22 09:32:37 -08:00
Chris Smith 6f6fca7592
Export resource change counts from engine (#823)
This PR exports the aggregate resource changes for update and destroy operations. We'll use this information in #636 when summarizing previous updates.

I initially started with a new struct that had fields like `Created`, `Deleted`, `Unchanged`, etc. But it became cumbersome with the seven different type of resource operations we perform. So instead went with the more flexible `map[deploy.StepOp]int`.
2018-01-20 19:15:19 -08:00
Chris Smith a4f087460f
Fix false-positives in login verification (#825)
Surprisingly `pulumi login -c https://google.com` would succeed. This was because we were too lax in our way of validating credentials. We take the provided cloud URL and call the "GetCurrentUserHandler" method. But we were only checking that it returned a successful response, not that it was actually valid JSON.

So in the "https://google.com" case, Google returned HTML describing a 404 error, but since the sever response was 200, the Pulumi CLI assumed things were on the up and up.

We now parse the response as JSON, and confirm the response has a `name` property that is non-nil. This heuristic covers the majority of false-positive cases, but without us needing to move all of the service's API shape for users, which includes organizations, which includes Clouds, etc. into `pulumi`.

Fixes https://github.com/pulumi/pulumi-service/issues/457. As an added bonus, we now return a much more useful error message.
2018-01-20 19:11:38 -08:00
Chris Smith f21c462e1b Report no changes for dry-runs 2018-01-20 12:15:28 -08:00
Chris Smith 7a5ac5ce06 Export resource change counts from engine 2018-01-20 12:07:03 -08:00
Matthew Riley 580bbcb3ca Avoid resource leaks by recovering from panics in integration tests 2018-01-20 11:48:42 -08:00
Matt Ellis b56e90ab2a Ensure resources are always parented to a stack
It was possiblef for the finally for a stack to complete before all
other resources had been created. In this case, we would put these new
resources at top level, instead of having them as children of the
stack resource.

Since we do not use the langhost across stacks, we can simply set the
stack resource at top level and never remove it.

Fixes #818
2018-01-19 16:54:50 -08:00
Matt Ellis 0d47915beb Provide a better error message for stack init when not logged in
`pulumi stack init` defaults to trying to create a stack in the Pulumi
Cloud. If you are not logged in, it prints an error telling you to log
in.

With this change, the error message also points out that you can pass
`--local` to `pulumi stack init` to create the stack locally.
2018-01-19 09:45:54 -08:00
Matt Ellis 41db5d881d Change --preview's short flag back to -n
In #806 we unintentionally changed the short flag for preview from
`-n` to `-d`. This restores the old flag.
2018-01-19 09:45:54 -08:00
Matt Ellis cc04cd6581 Use per stack key for local stacks instead of per project
In the Pulumi Cloud, there is no guarantee that two stacks will share
the same encryption key. This means that encrypted config can not be
shared across stacks (in the Pulumi.yaml) file. To mimic this behavior
in the local experience, we now use a unique key per stack.

When upgrading an existing project, for any stack with existing
secrets, we copy the existing key into this stack. Future stacks will
get thier own encryption key. This strikes a balance between
expediency of implementation, the end user UX and not having to make a
breaking change.

As part of this change, I have introduced a CHANGELOG.md file in the
root of the repository and added a small note about the change to it.

Fixes #769
2018-01-19 00:50:59 -08:00
Matt Ellis 22938f07be Pretty-print some JSON files we persist to disk
We did not pretty print either the workspace settings file or the
repository settings file, but pretty print other files like the
credentials file and checkpoints. Now we do.

Fixes #540
2018-01-18 20:35:53 -08:00
Chris Smith 3a3d0698ae
Surface update options to the service (#806)
This PR surfaces the configuration options available to updates, previews, and destroys to the Pulumi Service. As part of this I refactored the options to unify them into a single `engine.UpdateOptions`, since they were all overlapping to various degrees.

With this PR we are adding several new flags to commands, e.g. `--summary` was not available on `pulumi destroy`.

There are also a few minor breaking changes.

- `pulumi destroy --preview` is now `pulumi destroy --dry-run` (to match the actual name of the field).
- The default behavior for "--color" is now `Always`. Previously it was `Always` or `Never` based on the value of a `--debug` flag. (You can specify `--color always` or `--color never` to get the exact behavior.)

Fixes #515, and cleans up the code making some other features slightly easier to add.
2018-01-18 11:10:15 -08:00
Matt Ellis c506549a25 Remove MustFprintf in favor of explicitly dropping errors
In travis, we've seen cases where writes to our standard streams
results in an error like: `/dev/stderr: resource temporarily
unavailable` which causes the tests to panic.

Now, in a perfect world, writes to /dev/stderr would not fail in this
way, but we do not live in a perfect world. Other processes on the
machine may make stderr/stdout non-blocking. We've are now seeing this
failure in Travis more often and it is masking real Pulumi failures
we want to debug.
2018-01-16 18:33:44 -08:00