Commit graph

1087 commits

Author SHA1 Message Date
Matt Ellis 818246a708 Allow control of uploaded archive root in Pulumi.yaml
Previously, when uploading a projectm to the service, we would only
upload the folder rooted by the Pulumi.yaml for that project. This
worked well, but it meant that customers needed to structure their
code in a way such that Pulumi.yaml was always as the root of their
project, and if they wanted to share common files between two projects
there was no good solution for doing this.

This change introduces an optional piece of metadata, named context,
that can be added to Pulumi.yaml, which allows controlling the root
folder used for computing the root folder to archive from.  When it is
set, it is combined with the location of the Pulumi.yaml file for the
project we are uploading and that folder is uses as the root of what
we upload to the service.

Fixes: #574
2018-01-31 16:22:58 -08:00
Matt Ellis 39dbdc98e9 Clean up colorization logic
The existing logic would flow colorization information into the
engine, so depending on the settings in the CLI, the engine may or may
not have emitted colorized events. This coupling is not great and we
want to start moving to a world where the presentation happens
exclusively at the CLI level.

With this change, the engine will always produce strings that have the
colorization formatting directives (i.e. the directives that
reconquest/loreley understands) and the CLI will apply
colorization (which could mean either running loreley to turn the
directives into ANSI escape codes, or drop them or retain them, for
debuging purposes).

Fixes #742
2018-01-31 15:46:14 -08:00
Matt Ellis b1496f3051 Remove Document and Location
These types are no longer used as pulumi no longer reads and evaluates
source code.

Contributes to #441
2018-01-30 16:42:39 -08:00
Chris Smith 63dfd1e302
Add Manifest to GetStackResponse (#854)
Add Manifest to GetStackResponse.
2018-01-30 14:12:11 -08:00
Luke Hoban 0e4fcc16e3
Pass target Cloud and PPC to integation test Reporter (#859)
Ensure that we capture this information in our test reporting so that we can filter on it in queries and reports.
2018-01-30 13:10:32 -08:00
Matthew Riley b0905d99aa Fix long line 2018-01-30 10:52:42 -08:00
Matthew Riley c74db59231 Keep embedded newlines in Lambda log messages
Set the `s` flag in the regexp so `.` matches `\n`.

Add an explicit `\n` to the regexp to account for the newline inserted by Lambda.
2018-01-29 23:33:59 -08:00
Matt Ellis 07fd36c369 Remove deployOptions.Create
This was entirely unused.

Fixes #813
2018-01-29 16:23:55 -08:00
Matt Ellis 803ba93bf5 Pass --network-concurrency 1 to yarn
We've been hitting issues in CI that look like yarnpkg/yarn#4563 and
our hope is that this addresses the issue
2018-01-29 11:49:42 -08:00
Pat Gavlin 842a16cee4 Always set UpdateOptions.DryRun for previews.
If we do not do this, the engine will assume that it should elide
unknown input/output properties when interacting with the language
and resource providers and we will produce unexpected results.
2018-01-26 16:05:42 -08:00
Chris Smith 001d187c90
Add not-started UpdateResult
Add a "not-started" `UpdateResult` type, for supporting updates tracked by the PPC that have not been applied yet.
2018-01-25 14:23:49 -08:00
Chris Smith 4c217fd358
Add "pulumi history" command (#826)
This PR adds a new `pulumi history` command, which prints the update history for a stack.

The local backend stores the update history in a JSON file on disk, next to the checkpoint file. The cloud backend simply provides the update metadata, and expects to receive all the data from a (NYI) `/history` REST endpoint.

`pkg/backend/updates.go` defines the data that is being persisted. The way the data is wired through the system is adding a new `backend.UpdateMetadata` parameter to a Stack/Backend's `Update` and `Destroy` methods.

I use `tests/integration/stack_outputs/` as the simple app for the related tests, hence the addition to the `.gitignore` and fixing the name in the `Pulumi.yaml`.

Fixes #636.
2018-01-24 18:22:41 -08:00
Matt Ellis ce05cce77f Provide a rudimentary progress spinner
Previously, the `pulumi` tool did not show any indication of progress
when doing a deployment. Combined with the fact that we do not create
resources in parallel it meant that sometime `pulumi` would appear to
hang, when really it was just waiting on some resource to be created
in AWS. In addition, some AWS resources take a long time to create and
CI systems like travis will kill the job if there is no output. This
causes us (and our customers) to have to do crazy dances where we
launch shell scripts that write a dot to the console every once in a
while so we don't get killed. While we plan to overhaul the output
logic (see #617), we take a first step towards interactivity by simply
having a nice little spinner (in the interactive case) and when run
non interactive have `pulumi` print a message that it is still
working.

Fixes #794
2018-01-22 14:21:08 -08:00
Matthew Riley 4bb86116d0
Merge pull request #821 from pulumi/recover-in-test
Avoid resource leaks by recovering from panics in integration tests
2018-01-22 10:08:15 -08:00
Joe Duffy d6c798ee52
Add a commentChars parameter to EmitHeaderWarning (#827)
Now that we're using this to generate multiple languages, this
needs to be parameterized.  Splitting out into its own change.
2018-01-22 09:32:37 -08:00
Chris Smith 6f6fca7592
Export resource change counts from engine (#823)
This PR exports the aggregate resource changes for update and destroy operations. We'll use this information in #636 when summarizing previous updates.

I initially started with a new struct that had fields like `Created`, `Deleted`, `Unchanged`, etc. But it became cumbersome with the seven different type of resource operations we perform. So instead went with the more flexible `map[deploy.StepOp]int`.
2018-01-20 19:15:19 -08:00
Chris Smith a4f087460f
Fix false-positives in login verification (#825)
Surprisingly `pulumi login -c https://google.com` would succeed. This was because we were too lax in our way of validating credentials. We take the provided cloud URL and call the "GetCurrentUserHandler" method. But we were only checking that it returned a successful response, not that it was actually valid JSON.

So in the "https://google.com" case, Google returned HTML describing a 404 error, but since the sever response was 200, the Pulumi CLI assumed things were on the up and up.

We now parse the response as JSON, and confirm the response has a `name` property that is non-nil. This heuristic covers the majority of false-positive cases, but without us needing to move all of the service's API shape for users, which includes organizations, which includes Clouds, etc. into `pulumi`.

Fixes https://github.com/pulumi/pulumi-service/issues/457. As an added bonus, we now return a much more useful error message.
2018-01-20 19:11:38 -08:00
Chris Smith f21c462e1b Report no changes for dry-runs 2018-01-20 12:15:28 -08:00
Chris Smith 7a5ac5ce06 Export resource change counts from engine 2018-01-20 12:07:03 -08:00
Matthew Riley 580bbcb3ca Avoid resource leaks by recovering from panics in integration tests 2018-01-20 11:48:42 -08:00
Matt Ellis cc04cd6581 Use per stack key for local stacks instead of per project
In the Pulumi Cloud, there is no guarantee that two stacks will share
the same encryption key. This means that encrypted config can not be
shared across stacks (in the Pulumi.yaml) file. To mimic this behavior
in the local experience, we now use a unique key per stack.

When upgrading an existing project, for any stack with existing
secrets, we copy the existing key into this stack. Future stacks will
get thier own encryption key. This strikes a balance between
expediency of implementation, the end user UX and not having to make a
breaking change.

As part of this change, I have introduced a CHANGELOG.md file in the
root of the repository and added a small note about the change to it.

Fixes #769
2018-01-19 00:50:59 -08:00
Matt Ellis 22938f07be Pretty-print some JSON files we persist to disk
We did not pretty print either the workspace settings file or the
repository settings file, but pretty print other files like the
credentials file and checkpoints. Now we do.

Fixes #540
2018-01-18 20:35:53 -08:00
Chris Smith 3a3d0698ae
Surface update options to the service (#806)
This PR surfaces the configuration options available to updates, previews, and destroys to the Pulumi Service. As part of this I refactored the options to unify them into a single `engine.UpdateOptions`, since they were all overlapping to various degrees.

With this PR we are adding several new flags to commands, e.g. `--summary` was not available on `pulumi destroy`.

There are also a few minor breaking changes.

- `pulumi destroy --preview` is now `pulumi destroy --dry-run` (to match the actual name of the field).
- The default behavior for "--color" is now `Always`. Previously it was `Always` or `Never` based on the value of a `--debug` flag. (You can specify `--color always` or `--color never` to get the exact behavior.)

Fixes #515, and cleans up the code making some other features slightly easier to add.
2018-01-18 11:10:15 -08:00
Matt Ellis c506549a25 Remove MustFprintf in favor of explicitly dropping errors
In travis, we've seen cases where writes to our standard streams
results in an error like: `/dev/stderr: resource temporarily
unavailable` which causes the tests to panic.

Now, in a perfect world, writes to /dev/stderr would not fail in this
way, but we do not live in a perfect world. Other processes on the
machine may make stderr/stdout non-blocking. We've are now seeing this
failure in Travis more often and it is masking real Pulumi failures
we want to debug.
2018-01-16 18:33:44 -08:00
Joe Duffy 34984ba1cb
Delete the old IDL compiler (#801)
It is moving to a new location: https://github.com/pulumi/pidlc
2018-01-13 15:11:52 -08:00
Joe Duffy 0d107c742a
Restructure test framework to ease multiple languages (#799)
This change restructures the test framework code a bit, to make it
easier to introduce additional languages.  Our knowledge of Yarn and
Node.js project structure, for instance, was previously baked in to
the test logic, in a way that was hard to make, for instance, Yarn
optional.  (In Python, of course, it will not be used.)  To better
support this, I've moved some state onto a new programTester struct
that we can use to lazily find binaries required during the testing
(such as Yarn, Pip, and so on).  I'm committing this separately so
that I can minimize merge conflicts in the Python work.
2018-01-12 17:10:53 -08:00
Matthew Riley 3620cdb243
Merge pull request #792 from pulumi/fail-test
Keep test temp directories around on failure
2018-01-12 12:04:58 -08:00
khyperia 1a0cf9ff1b
Respond to feedback 2018-01-11 15:20:20 -08:00
khyperia 2598090cb7
Formatting 2018-01-11 13:46:34 -08:00
khyperia 08f3c623e6
Fix some style issues 2018-01-11 12:05:08 -08:00
khyperia 0f73b8c395
Move types to pulumi-service 2018-01-11 11:51:31 -08:00
khyperia d4de351b10
Correct some field types 2018-01-11 10:57:25 -08:00
khyperia 916ba6cd39
Add UpdateProgramRequestUntyped 2018-01-11 10:42:43 -08:00
khyperia f455689bd4
Add apitypes from pulumi-ppc 2018-01-10 15:43:44 -08:00
khyperia 3c644d243f
Create apitype package 2018-01-10 15:04:55 -08:00
Matthew Riley 047a6c3c56 Make "delete temp dir" condition explicit
Previously we checked if the test had failed -- but the call to
`assert.NoError` that actually fails the test sometimes hasn't happened yet.
2018-01-09 23:35:26 -08:00
Matthew Riley d688910360 Remove named parameters from CopyTestToTemporaryDirectory
Fix two references to the now-unnamed `dir` that should have been to
other variables.

Check a real condition before the deferred call to RemoveAll instead of
checking the error return.
2018-01-09 23:35:26 -08:00
Pat Gavlin 208b7296be
Merge pull request #791 from pulumi/AllowUnknownsPreview
Supply unknown properties to providers during preview.
2018-01-09 19:15:17 -08:00
Pat Gavlin 88a22515df Supply unknown properties to providers during preview.
My previous change to stop supplying unknown properties to providers
broke `pulumi preview` in the case of unknown inputs. This change
restores the previous behavior for previews only; the new unknown-free
behavior remains for applies.

Fixes #790.
2018-01-09 18:41:47 -08:00
Matthew Riley 3c070d3ad5 Restore support for non-Additive edits
None were used in this repository, but `pulumi-aws` used them.
2018-01-09 17:02:30 -08:00
Matthew Riley 9a869eb2d7 Print output of failed commands 2018-01-09 17:02:30 -08:00
Matthew Riley d9fa8237be Use length of correct string for prefixer 2018-01-09 16:48:04 -08:00
pat@pulumi.com f3cb37ef95 Do not expose unknowns to resource providers.
Before these changes, we were inconsistent in our treatment of unknown
property values across the resource provider RPC interface. `Check` and
`Diff` were retaining unknown properties in inputs and outputs;
`Create`, `Update`, and `Delete` were not. This interacted badly with
recent changes to `Check` to return all provider inputs--i.e. not just
defaults--from that method: if an unknown input was provided, it would
be present in the returned inputs, which would eventually confuse the
differ by giving the appearance of changes where none were present.

These changes remove unknowns from the provider interface entirely:
unknown property values are never passed to a provider, and a provider
must never return an unknown property value.

This is the primary piece of the fix for pulumi/pulumi-terraform#93.
2018-01-09 12:21:47 -08:00
pat@pulumi.com 5f28d52e00 Remove the Engine type entirely.
It no longer carries any state. All of its methods are now package-level
functions.
2018-01-08 14:20:51 -08:00
pat@pulumi.com c56e716c31 Refactor the engine's entrypoints.
These changes refactor the engine's entrypoints--Deploy, Destroy, and
Preview--to be update-centric rather than stack-centric. Each of these
methods now takes a value of a new type, Update, that abstracts away the
vagaries of fetching and maintaining the update's state. This
refactoring also reinforces Pulumi.yaml as a CLI concept rather than an
engine concept; the CLI is now the only reader/writer of this format.

These changes will smooth the way for a few refactorings on the service
side that will aid in update isolation.
2018-01-08 14:15:16 -08:00
Matthew Riley 9e3976513c AssertNoError instead of Assert(err == nil)
This may not be exhaustive, but I replaced all instances I could find.
2018-01-08 13:46:21 -08:00
Matthew Riley ff10b65f3a Add environment variable to keep data for failed tests
Useful to keep command output and maybe-still-relevant checkpoints around.
2018-01-08 13:46:21 -08:00
Matthew Riley 594ccf755a Write command output to log file 2018-01-08 13:46:21 -08:00
Matthew Riley b0eed85871 Remove non-additive edits
Now we know we only create one temporary directory per test
2018-01-08 13:46:21 -08:00
Matthew Riley afb734a712 Unexport a few private utility functions
Makes it bit easier to see the module entrypoints.
2018-01-08 13:46:21 -08:00