As it stands, we serialize more than is correct when registering
resources: in addition to serializing the RegisterResource RPC, we also
wait for input properties to resolve in the same context. Unfortunately,
this means that we can create cycles in the promise graph when a
resource A is constructed in an earlier turn than some resource B and
one of B's output properties is an input to resource A. These changes
fix this issue by allowing input properties to resolve *before*
serializing the RegisterResource RPC.
Some integration tests had taken a dependency on the ordering of resources in
either the output of the `pulumi` command or the checkpoint file. The
only test that took a dependency on command output was updated s.t. its
resources have exactly one legal topographical sort (and therefore their
ordering is deterministic). The other tests were updated s.t. their
validation did not depend on resource ordering.
This PR adds a new `pulumi history` command, which prints the update history for a stack.
The local backend stores the update history in a JSON file on disk, next to the checkpoint file. The cloud backend simply provides the update metadata, and expects to receive all the data from a (NYI) `/history` REST endpoint.
`pkg/backend/updates.go` defines the data that is being persisted. The way the data is wired through the system is adding a new `backend.UpdateMetadata` parameter to a Stack/Backend's `Update` and `Destroy` methods.
I use `tests/integration/stack_outputs/` as the simple app for the related tests, hence the addition to the `.gitignore` and fixing the name in the `Pulumi.yaml`.
Fixes#636.
These changes add the ability to export a stack's latest deployment or
import a new deployment to a stack via the Pulumi CLI. These
capabilities are exposed by two new verbs under `stack`:
- export, which writes the current stack's latest deployment to stdout
- import, which reads a new deployment from stdin and applies it to the
current stack.
In the local case, this simply involves reading/writing the stack's
latest checkpoint file. In the cloud case, this involves hitting two new
endpoints on the service to perform the export or import.
This PR updates the `pkg/testing/integration` package to support running integration tests against the Pulumi Service if desired. This is done through adding new options to `ProgramTestOptions`. (Generally adding support for providing values to flags that were previously inaccessible.)
I added an integration test to confirm that it all works if the PULUMI_API environment variable is set. These tests aren't run in Travis, only manually. Since we cannot reliably run tests from `master` against the service because of the delay in rolling out updates to the Pulumi SDK, etc.
This change implements resource protection, as per pulumi/pulumi#689.
The overall idea is that a resource can be marked as "protect: true",
which will prevent deletion of that resource for any reason whatsoever
(straight deletion, replacement, etc). This is expressed in the
program. To "unprotect" a resource, one must perform an update setting
"protect: false", and then afterwards, they can delete the resource.
For example:
let res = new MyResource("precious", { .. }, { protect: true });
Afterwards, the resource will display in the CLI with a lock icon, and
any attempts to remove it will fail in the usual ways (in planning or,
worst case, during an actual update).
This was done by adding a new ResourceOptions bag parameter to the
base Resource types. This is unfortunately a breaking change, but now
is the right time to take this one. We had been adding new settings
one by one -- like parent and dependsOn -- and this new approach will
set us up to add any number of additional settings down the road,
without needing to worry about breaking anything ever again.
This is related to protected stacks, as described in
pulumi/pulumi-service#399. Most likely this will serve as a foundational
building block that enables the coarser grained policy management.
This will allow us to remove a lot of current boilerplate in individual tests, and move it into the test harness.
Note that this will require updating users of the integration test framework. By moving to a property bag of inputs, we should avoid needing future breaking changes to this API though.
As articulated in #714, the way config defaults to workspace-local
configuration is a bit error prone, especially now with the cloud
workflow being the default. This change implements several improvements:
* First, --save defaults to true, so that configuration changes will
persist into your project file. If you want the old local workspace
behavior, you can specify --save=false.
* Second, the order in which we applied configuration was a little
strange, because workspace settings overwrote project settings.
The order is changed now so that we take most specific over least
specific configuration. Per-stack is considered more specific
than global and project settings are considered more specific
than workspace.
* We now warn anytime workspace local configuration is used. This
is a developer scenario and can have subtle effects. It is simply
not safe to use in a team environment. In fact, I lost an arm
this morning due to workspace config... and that's why you always
issue warnings for unsafe things.
* Take an options pointer so values can change as a test runs.
* Don't pass redundant information.
* Extract initialization routine.
* Fix caller.
* Check return value.
* Extract destruction logic.
* Move preview and update into their own function.
* Inline null check.
This change adds a `pulumi stack output` command. When passed no
arguments, it prints all stack output properties, in exactly the
same format as `pulumi stack` does (just without all the other stuff).
More importantly, if you pass a specific output property, a la
`pulumi stack output clusterARN`, just that property will be printed,
in a scriptable-friendly manner. This will help us automate wiring
multiple layers of stacks together during deployments.
This fixespulumi/pulumi#659.
* Revert "Make sure we properly update dir so that pulumi-destroy works."
This reverts commit 56bfc57998.
* Revert "Edits needs to continuously pass along the new directory. (#668)"
This reverts commit 8bd1822722.
* Revert "Refactor test code to make it simpler to validate code in the middle. (#662)"
This reverts commit ed65360157.
This fixes two closure bugs.
First, we had special cased `__awaiter` from days of yore, when we had
special cased its capture. I also think we were confused at some point
and instead of fixing the fact that we captured `this` for non-arrow
functions, which `__awaiter` would trigger, we doubled down on this
incorrect hack. This means we missed a real bonafide `this` capture.
Second, we had a global cache of captured variable objects. So, if a
free variable resolved to the same JavaScript object, it always resolved
to the first serialization of that object. This is clearly wrong if
the object had been mutated in the meantime. The cache is required to
reach a fixed point during mutually recursive captures, but we should
only be using it for the duration of a single closure serialization
call. That's precisely what this commit does.
Also add a fix for this case.
This fixespulumi/pulumi#663.
The two-phase output properties change broke the ability to recover
from a failed replacement that yields pending deletes in the checkpoint.
The issue here is simply that we should remember pending registrations
only for logical operations that *also* have a "new" state (create or
update). This commit fixes this, and also adds a new step test with
fault injection to probe many interesting combinations of steps.
At some point, we fixed a bug in the way state is managed for "same"
steps, which meant that we wouldn't see newly added output properties.
This had the effect that, if you had a stack already stood up, and
updated it to have output properties, we would miss them. (Stacks
stood up from scratch would still have them.) This fixes that problem,
in addition to two other things: 1) we need to sort output property
names to ensure a deterministic ordering, and 2) we need to also
unconditionally apply the outputs RPC coming in, to ensure that the
resulting resource always has the correct outputs (so that for example
deleting prior output properties actually deletes them).
Also add some testing for this area to make sure we don't break again.
Fixespulumi/pulumi#631.
This improves the overall cloud CLI experience workflow.
Now whether a stack is local or cloud is inherent to the stack
itself. If you interact with a cloud stack, we transparently talk
to the cloud; if you interact with a local stack, we just do the
right thing, and perform all operations locally. Aside from sometimes
seeing a cloud emoji pop-up ☁️, the experience is quite similar.
For example, to initialize a new cloud stack, simply:
$ pulumi login
Logging into Pulumi Cloud: https://pulumi.com/
Enter Pulumi access token: <enter your token>
$ pulumi stack init my-cloud-stack
Note that you may log into a specific cloud if you'd like. For
now, this is just for our own testing purposes, but someday when we
support custom clouds (e.g., Enterprise), you can just say:
$ pulumi login --cloud-url https://corp.acme.my-ppc.net:9873
The cloud is now the default. If you instead prefer a "fire and
forget" style of stack, you can skip the login and pass `--local`:
$ pulumi stack init my-faf-stack --local
If you are logged in and run `pulumi`, we tell you as much:
$ pulumi
Usage:
pulumi [command]
// as before...
Currently logged into the Pulumi Cloud ☁️https://pulumi.com/
And if you list your stacks, we tell you which one is local or not:
$ pulumi stack ls
NAME LAST UPDATE RESOURCE COUNT CLOUD URL
my-cloud-stack 2017-12-01 ... 3 https://pulumi.com/
my-faf-stack n/a 0 n/a
And `pulumi stack` by itself prints information like your cloud org,
PPC name, and so on, in addition to the usuals.
I shall write up more details and make sure to document these changes.
This change also fairly significantly refactors the layout of cloud
versus local logic, so that the cmd/ package is resonsible for CLI
things, and the new pkg/backend/ package is responsible for the
backends. The following is the overall resulting package architecture:
* The backend.Backend interface can be implemented to substitute
a new backend. This has operations to get and list stacks,
perform updates, and so on.
* The backend.Stack struct is a wrapper around a stack that has
or is being manipulated by a Backend. It resembles our existing
Stack notions in the engine, but carries additional metadata
about its source. Notably, it offers functions that allow
operations like updating and deleting on the Backend from which
it came.
* There is very little else in the pkg/backend/ package.
* A new package, pkg/backend/local/, encapsulates all local state
management for "fire and forget" scenarios. It simply implements
the above logic and contains anything specific to the local
experience.
* A peer package, pkg/backend/cloud/, encapsulates all logic
required for the cloud experience. This includes its subpackage
apitype/ which contains JSON schema descriptions required for
REST calls against the cloud backend. It also contains handy
functions to list which clouds we have authenticated with.
* A subpackage here, pkg/backend/state/, is not a provider at all.
Instead, it contains all of the state management functions that
are currently shared between local and cloud backends. This
includes configuration logic -- including encryption -- as well
as logic pertaining to which stacks are known to the workspace.
This addresses pulumi/pulumi#629 and pulumi/pulumi#494.
This PR just wires the `Package.Main` field to the Pulumi Service (and in subsequent PRs, the `pulumi-service` and `pulumi-ppc` repos).
@joeduffy , should we just upload the entire `package.Package` type with the `UpdateProgramRequest` type? I'm not sure we want to treat that type as part of part of our public API surface area. But on the other hand, we'll need to mirror relevant fields in N places if we don't.
This change fixes getProject to return the project name, as
originally intended. (One line was missing.)
It also adds an integration test for this.
Fixespulumi/pulumi#580.
Because the Pulumi.yaml file demarcates the boundary used when
uploading a program to the Pulumi.com service at the moment, we
have trouble when a Pulumi program uses "up and over" references.
For instance, our customer wants to build a Dockerfile located
in some relative path, such as `../../elsewhere/`.
To support this, we will allow the Pulumi.yaml file to live
somewhere other than the main Pulumi entrypoint. For example,
it can live at the root of the repo, while the Pulumi program
lives in, say, `infra/`:
Pulumi.yaml:
name: as-before
main: infra/
This fixespulumi/pulumi#575. Further work can be done here to
provide even more flexibility; see pulumi/pulumi#574.
PR #501 regressed this test due to a change in its error message. We
should consider updating the rest of the `stack` commands to use a
similar message (rather than the "file not found" they currently emit).
This PR adds integration tests for exercising `pulumi init` and the `pulumi stack *` commands. The only functional change is merging in https://github.com/pulumi/pulumi/pull/492 , which I found while writing the tests and (of course 😁 ) wrote a regression for.
To do this I introduce a new test driver called `PulumiProgram`. This is different from the one found in the `testing/integration`package in that it doesn't try to prescribe a workflow. It really just deals in executing commands, and confirming strings are in the output.
While it doesn't hurt to have more tests for `pulumi`, my motivation here was so that I could reuse these to ensure I keep the same behavior for my pending PR that implements Cloud-enabled variants of some of these commands.
