Commit graph

1623 commits

Author SHA1 Message Date
Chris Smith 5619fbce49
Add EngineEvents perf test (#2315)
* Add EngineEvents stress test

* Address PR feedback

* Specify value to config bag

* Don't test run in parallel
2019-01-03 14:18:19 -08:00
Justin Van Patten ea3b681a54
Link directly to /account/tokens (#2328)
We're changing /account to redirect to /account/profile instead of
/account/tokens as the user profile settings are a more natural place
to land when going into account settings.

This commit changes the CLI to link
directly to /account/tokens, avoiding having to click on
"Access Tokens" to go to the tokens page to get an access token when
coming from the URL outputted by the CLI.
2018-12-28 18:06:03 -05:00
Chris Smith c189a49172
Wait until all EEs are recorded after CancelEvent (#2325)
When returning immediately from the loop, we are closing the `done` channel early. This signals that we have finished processing every engine event, however that isn't true. Since some events may still be in-flight in `recordEngineEvent`. (This could potentially lead to a race condition in the `diag.Sink` passed to the API client used to record the call.)
2018-12-24 10:23:32 -08:00
Chris Smith 3045236dec
Record engine events concurrently (#2314)
* Record engine events concurrently

* Drop errors recording engine events
2018-12-20 15:34:30 -08:00
CyrusNajmabadi c3c51e6ea7
Don't record reads as changes during updates unless something actually changed. (#2308) 2018-12-19 13:19:56 -08:00
CyrusNajmabadi f7e6368218
During preview, don't record reads if they didn't actually result in any property changes. (#2305) 2018-12-18 16:33:13 -08:00
Pat Gavlin 24f89e1121
Close plugin context on plan creation failure (#2304)
This ensures that the gRPC server is properly shut down. This fixes an
issue in which a resource plugin that is still configuring could report
log messages to the plugin host, which would in turn attempt to send
diagnostic packets over a closed channel, causing a panic.

Fixes #2170.
2018-12-18 13:25:52 -08:00
Pat Gavlin ded3882afd
Fix synchronization with the event renderer (#2290)
The event rendering goroutine in the remote backend was not properly
synchronizing with the goroutine that created it, and could continue
executing after its creator finished. I believe that this is the root
cause of #1850.
2018-12-13 19:58:26 -08:00
Matt Ellis ad78f3ef59 Improve error message when Pulumi.yaml can't be found
Fixes #2234
2018-12-06 14:04:01 -08:00
Chris Smith 84ea2f0df7
Don't emit pre-rendered update logs (#2268)
* Don't emit pre-rendered update logs

* Retry posting engine events if needed

* Update comment
2018-12-04 14:46:32 -08:00
Matt Ellis f3fbc1d9ee Do not disable echo when stdin is not a terminal
When reading values like access keys or secrets from the terminal, we
would use the `terminal.ReadPassword` function to ensure characters
the user typed were not echo'd back to the console, as a convience.

When standard input was not connected to a tty (which would happen in
some cases like in docker when -t was not passed or in CI), this would
fail with an error about an bad ioctl. Update our logic such that
when standard in is not connected to a terminal, we just read input
normally.

While I was in the area, I unified the code for Windows and *NIX for
these functions.

Fixes #2017
2018-12-03 16:40:51 -08:00
Chris Smith 71375fe47e
Record engine events by default (#2256) 2018-12-03 08:25:58 -08:00
CyrusNajmabadi ca8169e344
Use 'output<...>' as our terminology for 'computed' properties. (#2267) 2018-12-02 19:44:50 -08:00
CyrusNajmabadi d93e930856
Provide an actual 'table' printing routine so that we can appropriate choose columns widths dynamically. (#2266) 2018-12-02 00:22:07 -08:00
Pat Gavlin 9c5526e7dd
Add a --config-file option for stack ops (#2258)
This option allows the user to override the file used to fetch and store
configuration information for a stack. It is available for the config,
destroy, logs, preview, refresh, and up commands.

Note that this option is not persistent: if it is not specified, the
stack's default configuration will be used. If an alternate config file
is used exclusively for a stack, it must be specified to all commands
that interact with that stack.

This option can be used to share plaintext configuration across multiple
stacks. It cannot be used to share secret configuration, as secrets are
associated with a particular stack and cannot be decryptex by other
stacks.
2018-11-30 15:11:05 -08:00
Chris Smith 55bea65276
Fix error when canceling update (#2254)
* Fix error when canceling update

* Update error message
2018-11-29 12:54:05 -08:00
CyrusNajmabadi af9671a4dd
Add test helper function. (#2251) 2018-11-28 11:46:10 -08:00
Chris Smith ba31c6c69a
Incorporate feedback on Engine Events API (#2235)
* Add Sequence number to EngineEvents apitype

* Fix typo

* Send Timestamp with EngineEvents
2018-11-27 07:34:44 -08:00
Matt Ellis e3f8726d0a Do not pass --verbose to yarn install
When the install fails, we end up printing the entire contents of
yarn's stdout to stdout. This output can often be quite long and will
cause Travis to fail in some cases.

The regular error output should be sufficent for us to diagnose any
issues we'll face.
2018-11-25 22:02:28 -08:00
Matt Ellis 872c7661e3 Provide a way to override packages during a test run
Add a new property to ProgramTestOptions, `Overrides` that allows a
test to request a different version of a package is used instead of
what would be listed in the package.json file.

This will be used by our nightly automation to run everything "at head"
2018-11-25 22:02:28 -08:00
Matt Ellis e0918723ae Allow build metadata in semver for plugins
Semver allows you to attach "build metadata" to a version by appending
the version with `+` and then metadata. In #2216 we started to take
advantage of this as the place to put the git commit information,
instead of including it as part of the "version". This is more in line
with what Semver expects to be done, because git commit information
isn't orderable.

Because of this, we started to publish plugins with versions like
`v0.16.5-dev.1542649729+g07d8224`. However, our logic for discovering
plugins in the cache did an initial filtering based on folder names in
the cache and the regex did not allow a + in the "version" field.

This meant that from the point of view of the cache, the plugin was
not present. This would lead to very confusing behavior where
something like `pulumi plugin install resource azure
v0.16.5-dev.1542649729+g07d8224` would download the plugin, but
`pulumi plugin ls` would not see it and attempting to do an update
with it would fail with an error saying the plugin was not installed.

This change relaxes the regular expression to allow it to match these
sorts of paths. We still use the `semver` library to ensure that the
version we've extracted from the directory name is a valid semver.
2018-11-21 17:10:43 -08:00
Pat Gavlin ab36b1116f
Handle unconfigured plugins in Diff. (#2238)
After #2088, we began calling `Diff` on providers that are not configured
due to unknown configuration values. This hit an assertion intended to
detect exactly this scenario, which was previously unexpected.

These changes adjust `Diff` to indicate that a Diff is unavailable and
return an error message that describes why. The step generator then
interprets the diff as indicating a normal update and issues the error
message to the diagnostic stream.

Fixes #2223.
2018-11-21 16:53:29 -08:00
Matt Ellis 72d52c6e1f Don't fail on configuration keys like a:config:b:c
Configuration keys are simple namespace/name pairs, delimited by
":". For compatability, we also allow
"<namespace>:config:<name>", but we always record the "nice" name in
`Pulumi.<stack-name>.yaml`.

While `pulumi config` and friends would block setting a key like
`a🅱️c` (where the "name" has a colon in it), it would allow
`a:config:b:c`. However, this would be recorded as `a🅱️c` in
`Pulumi.<stack-name>.yaml`, which meant we'd error when parsing the
configuration file later.

To work around this, disallow ":" in the "name" part of a
configuration key.  With this change the following all work:

```
keyName
my-project:keyName
my-project:config:keyName
```

However, both

`my-project:keyName:subKey`
`my-project:config:keyName:subKey`

are now disallowed.

I considered allowing colons in subkeys, but I think it adds more
confusion (due to the interaction with how we allow you elide the
project name in the default case) than is worthwhile at this point.

Fixes #2171
2018-11-20 14:14:37 -08:00
Matt Ellis c878916901 Look for language plugins next to pulumi when probing
When launching plugins today, `pulumi` looks in two places:

1. It looks to see if the plugin in on the $PATH and if so, uses
   it. This makes it easy to force a specific version of a resource
   provider to be used and is what happens at development time (since
   resource providers make their way onto $PATH via GOBIN).

2. If the above fails, it looks in the "plugin cache" in
   `~/.pulumi/plugins`. This is the location that `pulumi plugin
   install` places plugins.

Unlike resource provider plugins, we don't yet deliver language
plugins via `pulumi plugin install` so the language provider plugins
must be on the `$PATH` to be found. This is okay, because when we ship
the SDK, we include the executables next to `pulumi` itself.

However, if a user chooses to not put `pulumi` on their $PATH, or they
do but it is a symlink to the real `pulumi` binary installed
somewhere, we'd fail to find the language plugins, since they would
not be on the `$PATH`

To address this, when probing for language plugins, also consider
binaries next to the currently running `pulumi` process.

Fixes #1956
2018-11-20 12:02:36 -08:00
CyrusNajmabadi a7d2f10eaf
Allow tests to pass additional flags when doing a preview. (#2232) 2018-11-20 02:05:24 -08:00
Matt Ellis 35215d6a07 Write .yarnrc with both test frameworks
While the lifecycle tests wrote a `.yarnrc` file to ensure that copies
of `yarn` did not race with one another, the more barebones testing
framework did not.

This should address some of the yarn issues we've been seeing in CI
recently
2018-11-19 17:12:18 -08:00
Pat Gavlin 676adf62b8
Use an explicit address when dialing plugins (#2224)
This is necessary in order for gRPC's proxy support to properly respect
NO_PROXY.

Fixes #2134.
2018-11-19 13:47:39 -08:00
Matt Ellis 6e95bdda9c Merge branch 'release/0.16' into ellismg/merge-release 2018-11-16 20:22:13 -08:00
Matt Ellis 2b3c7f61c5 Rework get-version scripts
Under our old versioning system, when we started a new point release,
we'd tag the HEAD commit of master with a tag like `v0.16.6-dev` and
our scripts would use this to generate a new version number. This
required a great deal of gymnastics when producing a release and
caused us to litter these -dev tags everywhere.

To improve this, we change version number generation to the following
strategy:

1. If the commit we are building has a tag applied to it, use that tag
as the version (appending the dirty bit metadata to the version, if
needed).

2. If the commit we are bulding does not have a tag applied to it,
take the version from the next reachable tag, increment the patch
version and then append the `-dev` pre-release tag. As part of this,
we also make a slight tweek to our semver generation such that instead
of `-dev<TIMESTAMP>` we use `-dev.<TIMESTAMP>` which is more in line
with what semver recommends.
2018-11-16 20:11:04 -08:00
Matt Ellis faff805f1e Fix an issue where plugin install would fail on windows
The issue is related to this code:

https://github.com/pulumi/pulumi/blob/v0.16.4/pkg/workspace/plugins.go#L155-L195

Note that we use `defer` to ensure we close our handle to the file we
are unpacking when we encounter a file in the tarball.  However, the
defers don't run until the containing function ends, so when we go to
do the rename, or process still has a bunch of open file handles, which
prevents the directory from being renamed because it is "in use".

By doing all of the work in an anonymous function, we ensure that the
defer statements run before we go to rename the directory

Fixes #2217
2018-11-16 20:07:24 -08:00
Matt Ellis c63a99ac2d Ensure the plugins root folder exists
On clean installs, where `~/.pulumi/plugins` doesn't exist, we failed
to create the temporary directory.
2018-11-16 18:04:25 -08:00
Matt Ellis 79204ad1b2 Do not fail plugin install when the destination folder exists
This is code that should have been part of #2211 but was accidently
dropped during a rebase when responding to CR feedback.

When two installs for the same plugin are racing, the second one will
see the destination directory already exists and fail. We can safely
ignore this error.
2018-11-15 17:15:27 -08:00
Matt Ellis 76e20281f4 Fix a race when pulumi plugin install ran concurrently 2018-11-15 15:50:08 -08:00
Pat Gavlin bc08574136
Add an API for importing stack outputs (#2180)
These changes add a new resource to the Pulumi SDK,
`pulumi.StackReference`, that represents a reference to another stack.
This resource has an output property, `outputs`, that contains the
complete set of outputs for the referenced stack. The Pulumi account
performing the deployment that creates a `StackReference`  must have
access to the referenced stack or the call will fail.

This resource is implemented by a builtin provider managed by the engine.
This provider will be used for any custom resources and invokes inside
the `pulumi:pulumi` module. Currently this provider supports only the
`pulumi:pulumi:StackReference` resource.

Fixes #109.
2018-11-14 13:33:35 -08:00
Matt Ellis b5a0e039fb Download plugins to temp folder before installing
Fixes #1861
2018-11-12 14:09:26 -08:00
Matt Ellis bbd2172541
Merge pull request #2191 from pulumi/chrsmith/fix-minor-issues
Fix issue with converting keys
2018-11-12 10:33:17 -08:00
Joe Duffy 1e9f5b96f2
Merge pull request #2164 from pulumi/ellismg/update-tools
Update to Go 1.11 and golangci-lint
2018-11-10 17:32:14 -08:00
Chris Smith 6b862e8afe Fix issue with converting keys 2018-11-09 14:01:15 -08:00
Chris Smith c63c43b3a9 Add missing annotation 2018-11-09 11:11:05 -08:00
Chris Smith 8b76fd53d1
Record engine events (#2168)
* apitypes for surfacing engine events

* Add more comments to runEngineAction

* Persist engine events in the service

* Use cmdutil.IsTruthy
2018-11-09 10:01:29 -08:00
Matt Ellis 992b048dbf Adopt golangci-lint and address issues
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
2018-11-08 14:11:47 -08:00
Matt Ellis c398e8ba80 Don't write a lock file when installing test packages
The lockfile is not super interesting when running the lifecycle
tests, since the test project is not a long lived artifact. There was
no real harm in writing it, exepct that in some cases `pipenv` gets
very confused when you have dependencies on pre-release
versions (which is the case right now as we are brining up Python 3).

The packages are still installed correctly, even when we don't write a
lock file.
2018-11-08 12:09:10 -08:00
Sean Gillespie 4ae39b8bec
Ask for the newest Python 3 instead of 3.6 specifically (#2165)
* Ask for the newest Python 3 instead of 3.6 specifically

* Remove PythonVersion from test opts
2018-11-06 10:38:46 -08:00
Sean Gillespie 36c88aab37
Fix Python support in integration test framework (#2158)
* Fix Python support in integration test framework

Update the integratino test framework to use pipenv to bootstrap new
virtual environments for tests and use those virtual environments to run
pulumi update and pulumi preview.

Fixes pulumi/pulumi#2138

* Install packages via 'Dependencies' field

* Remove code for installing packages from Dependencies
2018-11-05 13:52:37 -08:00
CyrusNajmabadi 4e2a6aee0f
Ensure we show the properties that changed when doing a multi-stage replace. (#2153) 2018-11-05 12:37:53 -08:00
CyrusNajmabadi 1ea63761d9
Fix log retrieval for lambdas. (#2160) 2018-11-05 12:27:11 -08:00
Chris Smith 0a76923c64
Add CircleCI support (#2143) 2018-11-01 11:20:31 -07:00
Joe Duffy 9aedb234af
Tidy up some data structures (#2135)
In preparation for some workspace restructuring, I decided to scratch a
few itches of my own in the code:

* Change project's RuntimeInfo field to just Runtime, to match the
  serialized name in JSON/YAML.

* Eliminate the no-longer-used Context and NoDefaultIgnores fields on
  project, and all of the associated legacy PPC-related code.

* Eliminate the no-longer-used IgnoreFile constant.

* Remove a bunch of "// nolint: lll" annotations, and simply format
  the structures with comments on dedicated lines, to avoid overly
  lengthy lines and lint suppressions.

* Mark Dependencies and InitErrors as `omitempty` in the JSON
  serialization directives for CheckpointV2 files. This was done for
  the YAML directives, but (presumably accidentally) omitted for JSON.
2018-11-01 08:28:11 -07:00
Pat Gavlin b748935753
Do not assert on duplicate resources. (#2127)
Just what it says on the tin.
2018-10-31 10:33:00 -07:00
CyrusNajmabadi 76d08f8590
Simplify summary text. (#2136) 2018-10-30 21:57:38 -07:00