Commit graph

1394 commits

Author SHA1 Message Date
CyrusNajmabadi 0b0927d257
Fix printing out outputs in a pulumi program. (#1531) 2018-06-18 16:03:26 -07:00
Pat Gavlin 0c8dedec22
Issue an error when attempting to read an invalid Asset. (#1528)
As an alternative we could consider simply returning an empty `Blob`.
I think it's better to fail loudly, though, as this situation can result
from unexpected/invalid archives (which we need to do a better job of
validating).

Fixes #1493.
2018-06-18 11:08:17 -07:00
Matt Ellis a47babd194 Run export/import rounttrip as part of our lifecycle tests
This allows us to delete the one off export/import test, which is nice
because it failed to run when PULUMI_ACCESS_TOKEN was not set in the
environment (due to an interaction between `pulumi login` and the
bare-bones integration test framework)
2018-06-14 16:01:28 -07:00
Matt Ellis 2669f65b39 Don't dirty work tree when running tests
Because we run our golang integration tests "in tree" (due to
the need to be under $GOPATH and have a vendor folder around), the
"command-output" folder was getting left behind, dirtying the worktree
after building.

This change does two things:

1. On a succecssful run, remove the folder.
2. Ignore the folder via .gitignore (this way if a test fails and you
do `git add .` you don't end up commiting this folder).
2018-06-14 15:58:37 -07:00
joeduffy 7bce92085d Add a colors.Highlight function
This change makes it a little easier to do the style of highlighting
we are doing now with the login prompt, cleaning up some of the
padding calculations that were otherwise complicated due to ANSI
escape sequences.
2018-06-12 08:14:41 -07:00
joeduffy f4436416e8 Tidy up the login message and colors
This change makes our login prompt a little "friendlier", especially
important since this will be the first thing a new user sees.

The new message is:

    $ pulumi new
    We need your Pulumi account to identify you.
    Enter your access token from https://app.pulumi.com/account
        or hit <ENTER> to log in using your browser            :
2018-06-12 07:52:22 -07:00
Matthew Riley bdb3e98aa0 Use an even more reasonable modtime in ZIP entries
The ZIP format started with MS-DOS dates, which start in 1980. Other dates
have been layered on, but the ZIP file handler used by Azure websites still
relies on the MS-DOS dates.

Using the Unix epoch here (1970) results in ZIP entries that (e.g.)
OSX `unzip` sees as 12-31-1969 (timezones) but Azure websites sees as
01/01/2098.
2018-06-11 19:36:04 -07:00
Pat Gavlin dc0c604ae7
Enable fork builds. (#1495)
If we're building in CI and do not have an access token, skip any
lifecycle tests.
2018-06-11 16:01:04 -07:00
Pat Gavlin f06b50d428
Merge pull request #1494 from pulumi/LinterAppeasement
Appease linters.
2018-06-11 14:52:42 -07:00
Joe Duffy b69ad03625
Merge pull request #1491 from pulumi/go_testing
Fix a few things in the Go test harness
2018-06-11 14:44:07 -07:00
Pat Gavlin e6849a283f Appease linters.
- Fix a couple self-assignment issues in the Go language support
- Disable `megacheck` for `fh.SetModTime`, which we use for go1.9
  compat.
2018-06-11 14:32:27 -07:00
joeduffy c81a5152f0 Skip copying to a temp dir for Go projects
Due to the way GOPATH and vendoring works, copying Go tests out to a
random temp directory simply will not work.  This is largely a holdover
to the days of when .pulumi/ directories would pollute the directory,
but is also done for languages like Node.js whose preparatory and build
steps also pollute the working directory (with bin/ and node_modules/
directories).  Go does not have this problem, so we can safely skip.
2018-06-11 14:24:29 -07:00
joeduffy 6599240861 Default to $HOME/go for GOPATH during tests 2018-06-11 13:08:46 -07:00
Matt Ellis bb6b492d55 Do not include git information in PyPI version.
Previously, we would include information about what git commit a build
came from in the "local" portion of the PEP-440 version. This was a
problem because PyPI does not allow packages to be upload to the
registry if they contain local parts.

So, for now, we'll just never put in the git commit information in the
generate version. We'll continue to add a dirty tag in the local
part. This will be prevent us from publishing dirty builds to PyPI,
but that's in line with what we want.
2018-06-11 13:01:49 -06:00
Matt Ellis 9d4e24c165 Allow tests to be run by users outside of the Pulumi organization
Today we defaulted our tests to create stacks in the `pulumi`
organization. We did this because our tests run with `pulumi-bot` and
we'd rather create the stacks in our shared organization, so any
Pulumi developer can see them.

Of course, as we prepare to have folks outside of the Pulumi
organization write and run tests, this has now become a bad default.

Remove the ability to explicitly set an owner in
ProgramTestOptions (since that would more or less only lead to pain
going forward) and default to just creating the stacks in whatever
account is currently logged in. In CI, we'll set a new environment
variable "PULUMI_TEST_OWNER" which controls the owner of the stacks,
which we'll set to `pulumi`.

Impact to day to day developers is during test runs locally you'll see
stacks in your list of stacks. If any of the tests fail to clean up,
you'll see these lingering stacks (but you can go clean them up).
2018-06-11 11:03:17 -06:00
joeduffy b28f643164 Add integration test support for Go
This adds integration test support framework for Go.

It also adds a test case for the basic empty Pulumi Go program.
2018-06-10 09:17:19 -07:00
joeduffy 74a896bb7a Add strongly typed outputs
This change adds some convenience functions and types, to make strongly
typed outputs more pleasant to interact with.  It also includes tests
for output generally, in addition to these new functions and types.
2018-06-08 10:36:10 -07:00
joeduffy 7d8995991b Support Pulumi programs written in Go
This adds rudimentary support for Pulumi programs written in Go.  It
is not complete yet but the basic resource registration works.

Note that, stylistically speaking, Go is a bit different from our other
languages.  This made it a bit easier to build this initial prototype,
since what we want is actually a rather thin veneer atop our existing
RPC interfaces.  The lack of generics, however, adds some friction and
is something I'm continuing to hammer on; this will most likely lead to
little specialized types (e.g. StringOutput) once the dust settles.

There are two primary components:

1) A new language host, `pulumi-language-go`, which is responsible for
   communicating with the engine through the usual gRPC interfaces.
   Because Go programs are pre-compiled, it very simply loads a binary
   with the same name as the project.

2) A client SDK library that Pulumi programs bind against.  This exports
   the core resource types -- including assets -- properties -- including
   output properties -- and configuration.

Most remaining TODOs are marked as such in the code, and this will not
be merged until they have been addressed, and some better tests written.
2018-06-08 10:36:10 -07:00
Matt Ellis 0fcfbf39c3 Support browser based logins to the CLI
During login, if no access token is provided, use our browser
based login.
2018-06-08 11:32:25 -06:00
Pat Gavlin 8e12e739c2
Merge pull request #1469 from pulumi/ErrorIDs
Allow duplicate error IDs.
2018-06-08 10:15:29 -07:00
Matthew Riley 9916cd5e6b
Merge pull request #1475 from pulumi/zip-header
Use a reasonable value for Modified in ZIP headers
2018-06-07 14:40:10 -07:00
Matthew Riley 2fe284a139 Use a reasonable value for Modified in ZIP headers
Azure websites can't extract the archive without this.

We can't use the `Modified` field of `FileHeader` because it was added in
Go 1.10.
2018-06-07 14:21:03 -07:00
Sean Gillespie b5e4d87687
Improve the error message when data source invocations fail (#1472) 2018-06-07 11:21:38 -07:00
Pat Gavlin 5ea895d9c7 Remove an unused import 2018-06-06 14:28:49 -07:00
Pat Gavlin 4ee73a0042 Allow duplicate error IDs.
There is no longer a reason to ensure that these IDs are distinct.

Fixes #1464.
2018-06-06 14:10:48 -07:00
Matt Ellis 8a6ffd08b2 Use crypto/rand when generating a stack suffix
math/rand uses a fixed seed, meaning that across runs the Kth call to
`rand.Int63()` will always return the same value.

Given that we want to provide a unique suffix across multiple
concurrent runs, this isn't the behavior we want.

I saw an instance fail in CI where all three legs ran the test
concurrently and they raced on creating the test stack, since they all
generated the same name.
2018-06-05 11:06:01 -07:00
Matt Ellis 4f28f3d44d Allow overriding config location
Now that the "config" member of Pulumi.yaml has been deprecated for a
while, we'll change it's meaning. When set, the value is treated as a
path and joined with the path to Pulumi.yaml, and per stack
configuration is stored in that folder

Fixes #1031
2018-06-05 09:26:48 -07:00
Chris Smith ebfc3458d5
Export GetCommandResults from Environment (#1436) 2018-06-05 08:34:06 -07:00
Matt Ellis 4ac1a2f355 Pull tracing and reporting information from the environment
This lets us set these values globaly, in our Travis and AppVeyor
configurations instead of forcing every test to opt-in. It also means
that by default, local builds will not report any of this data (and
will not need access to these endpoints).
2018-06-04 14:28:52 -07:00
Matthew Riley d8358f7be3
Merge pull request #1453 from pulumi/retry-post
Send request body on retries
2018-06-04 11:32:19 -07:00
Sean Gillespie 1a2d4d9ff1
Fix an issue where the Version field of an UntypeDeployment was lost (#1450)
The Version field was inadvertently dropped when sending an import
request to the service. Now that we are requiring that the Version field
be set in deployments, this was causing errors.
2018-06-03 14:30:51 -07:00
Matthew Riley 71e9d8c2af Use bytes.Reader instead of bytes.Buffer
Calling `bytes.NewBuffer` gives up ownership of the backing memory and creates
a _mutable_ buffer. We don't need that, so let's use `bytes.NewReader` instead.
2018-06-03 01:33:50 -07:00
Matthew Riley e7498f9468 Send request body on retries
Not sending request body on retry is bad enough, but we keep the original
`Content-Length` and the server ends up expecting a body we never send.
This results in `400 Bad Request` errors from the ALB.
2018-06-03 01:33:50 -07:00
Matthew Riley 261fa86586
Merge pull request #1358 from pulumi/retry-bugs
Fix two bugs in retries
2018-06-01 17:11:23 -07:00
Chris Smith 485bb35180
Relax stack name requirements (#1381)
* Relax stack name requirements

* Add error if stack name too long

* Max tag length is 256 chars
2018-05-29 13:52:11 -07:00
Pat Gavlin da9bd9dbff
Merge pull request #1430 from pulumi/AlwaysPrompt
Always prompt for confirmation after a preview.
2018-05-29 09:47:00 -07:00
Sean Gillespie 924c49d7e0
Fail fast when attempting to load a too-new or too-old deployment (#1382)
* Error when loading a deployment that is not a version that the CLI understands

* Add a test for 'pulumi stack import' on a badly-versioned deployment

* Move current deployment version to 'apitype'

* Rebase against master

* CR: emit CLI-friendly error message at the two points outside of the engine calling 'DeserializeDeployment'
2018-05-25 13:29:59 -07:00
Sean Gillespie fe22ade275
Differentiate between warnings and errors when sending an event (#1424)
* Differentiate between warnings and errors when sending an event

* CR: always write severity field if the event is a diagnostic
2018-05-25 13:29:33 -07:00
Sean Gillespie 58ca6f5f19
Fix a confusing error message arising from multiple API calls failing in the same way (#1432) 2018-05-24 15:36:35 -07:00
Pat Gavlin 31739c8840 Always prompt for confirmation after a preview.
An empty preview is not a guarantee that a subsequent update will cause
no changes. As such, we should always offer the user the option to
perform an update even if the preview proposes no changes.

Note that this indirectly fixes #1410 by masking the effects of #1429.
2018-05-24 12:22:20 -07:00
Matt Ellis 9a8f8881c0 Show manifest information for stacks
This change supports displaying manifest information for a stack and
changes the way we handle Snapshots in our backend.

Previously, every call to GetStack would synthesize a Snapshot by
taking the set of resources returned from the
`/api/stacks/<owner>/<name>` endpoint, combined with an empty
manfiest (since the service was not returning the manifest).

This wasn't great for two reasons:

1. We didn't have manifest information, so we couldn't display any of
   its information (most important the last updated time).

2. This strategy required that the service return all the resources
   for a stack anytime GetStack was called. While the CLI did not
   often need this detailed information the fact that we forced the
   Service to produce it (which in the case of stack managed PPC would
   require the service to talk to yet another service) creates a bunch
   of work that we end up ignoring.

I've refactored the code such that `backend.Stack`'s `Snapshot()` method
now lazily requests the information from the service such that we can
construct a `Snapshot()` on demand and only pay the cost when we
actually need it.

I think making more of this stuff lazy is the long term direction we
want to follow.

Unfortunately, right now, it means in cases where we do need this data
we end up fetching it twice. The service does it once when we call
GetStack and then we do it again when we actually need to get at the
Snapshot.  However, once we land this change, we can update the
service to no longer return resources on the apistack.Stack type. The
CLI no longer needs this property.  We'll likely want to continue in a
direction where `apistack.Stack` can be created quickly by the
service (without expensive database queries or fetching remote
resources) and just add additional endpoints that let us get at the
specific information we want in the specific cases when we want it
instead of forcing us to return a bunch of data that we often ignore.

Fixes pulumi/pulumi-service#371
2018-05-23 16:43:34 -07:00
Sean Gillespie 1a51507206
Delete Before Create (#1365)
* Delete Before Create

This commit implements the full semantics of delete before create. If a
resource is replaced and requires deletion before creation, the engine
will use the dependency graph saved in the snapshot to delete all
resources that depend on the resource being replaced prior to the
deletion of the resource to be replaced.

* Rebase against master

* CR: Simplify the control flow in makeRegisterResourceSteps

* Run Check on new inputs when re-creating a resource

* Fix an issue where the planner emitted benign but incorrect deletes of DBR-deleted resources

* CR: produce the list of dependent resources in dependency order and iterate over the list in reverse

* CR: deps->dependents, fix an issue with DependingOn where duplicate nodes could be added to the dependent set

* CR: Fix an issue where we were considering old defaults and new inputs
inappropriately when re-creating a deleted resource

* CR: save 'iter.deletes[urn]' as a local, iterate starting at cursorIndex + 1 for dependency graph
2018-05-23 14:43:17 -07:00
Pat Gavlin 38d3b83494
Allow tests to perform changes in empty updates. (#1411)
This may be a useful knob to have.
2018-05-23 09:17:10 -07:00
Pat Gavlin f3201baeb5
Fix {computed,output}<> display in preview. (#1409)
At some point the summary and preview parameters to
`engine.GetResourcePropertiesDetails` were flipped s.t. we stopped
rendering computed<> and output<> values properly during previews.
2018-05-22 17:20:03 -07:00
Pat Gavlin 37a3317623
Put local backend support under PULUMI_DEBUG_COMMANDS. (#1408)
Just what it says on the tin.

Fixes #1398.
2018-05-22 17:02:45 -07:00
joeduffy 5967259795 Add license headers 2018-05-22 15:02:47 -07:00
Matt Ellis 0732b05c5d Remove pulumi init
`pulumi init` was part of our old identity model with the service and
is no longer used. We can now delete this code.

Fixes #1241
2018-05-22 13:37:08 -07:00
Pat Gavlin b5efe40f04
Fold "global" messages under the stack resource. (#1406)
We currently display at most two rows for messages that are not
associated with a URN: one for those that arrive before the stack
resource is first observed and one for those that arrive after the stack
resource is first observed. The former is labeled "global"; the latter
is the row associated with the stack resource. These changes remove the
duplication by reassociating the row used for events with no URN that
arrive before the stack resource with the stack resource once it has
been observed.

Fixes #1385.
2018-05-22 13:30:52 -07:00
Pat Gavlin c08ede339f Only display preview/no-op resource rows once on non-TTYs.
Our current strategy for the progress display on non-TTYs causes us to
display multiple identical rows for each resource when the row is a
preview or a no-op. This behavior is not particularly useful, and
generally just makes the display noisier than it needs to be.

These changes avoid displaying resource rows without meaningful output
by supressing the display of resource output events that are delivered
during a preview or that correspond to a no-op update.

Fixes #1384.
2018-05-22 11:35:07 -07:00
joeduffy 614a2cdeb3 Simplify previews, initialize plugin events
This changes two things:

1) Eliminates the fact that we had two kinds of previews in our engine.

2) Always initialize the plugin.Events, to ensure that all plugin loads
   are persisted no matter the update type (update, refresh, destroy),
   and skip initializing it when dryRun == true, since we won't save them.
2018-05-18 14:58:06 -07:00