Commit graph

1889 commits

Author SHA1 Message Date
Pat Gavlin 2543966110 Shorten lines. 2017-10-16 23:06:53 -07:00
Pat Gavlin e4ae5bcd03 Update error reporting and add a couple comments. 2017-10-16 23:06:53 -07:00
pat@pulumi.com e66d45ed30 Add a newline. 2017-10-16 23:06:53 -07:00
Pat Gavlin 308045b274 Move serializeProvider out of dynamic.Resource. 2017-10-16 23:06:53 -07:00
Pat Gavlin 6c14dcd559 s/[Cc]allbacks/[Pprovider/g 2017-10-16 23:06:53 -07:00
Pat Gavlin 1b4ed6cce3 PR feedback 2017-10-16 23:06:53 -07:00
Pat Gavlin afd7c400ad Remove the testing provider.
This provider has been obviated by dynamic resources.
2017-10-16 23:06:53 -07:00
pat@pulumi.com 9453f86c2e Implement dynamic resources.
A dynamic resource is a resource whose provider is implemented alongside
the resource itself. This provider may close over and use orther
resources in the implementation of its CRUD operations. The provider
itself must be stateless, as each CRUD operation for a particular
dynamic resource type may use an independent instance of the provider.
Changes to the definition of a resource's provider result in replacement
of the resource itself (rather than a simple update), as this allows the
old provider definition to delete the old resource and the new provider
definition to create an appropriate replacement.
2017-10-16 23:06:53 -07:00
Pat Gavlin 546612a354 Drain std{out,err} when a plugin fails to load.
If a plugin fails to load after we've set up the goroutines that copy
from its std{out,err} streams, then those goroutines can end up writing
to a closed event channel. This change ensures that we properly drain
those streams in this case.
2017-10-16 21:38:11 -07:00
Matt Ellis 908d081e88 Construct version based on git information
We use `git describe --tags` to construct a version number based on
the current version tag.

The properties VERSION (when using make) and Version (when using
MSBuild) can be explicitly set to use a fixed value instead.

Fixes #13
2017-10-16 18:35:41 -07:00
Matt Ellis 15a0692ac8 Support using short names for config keys
Previously, you had to fully qualify configuration values (e.g
example:config:message). As a convience, let's support adding
configuration values where the key is not a fully qualified module
member. In this case, we'll treat the key as if
`<program-name>:config:` had been prepended to it.

In addition, when we print config, shorten keys of the form
`<program-name>:config:<key-name>` to `<key-name>`.

I've updated one integration test to use the new syntax and left the
other as is to ensure both continue to work.
2017-10-16 16:10:24 -07:00
Matt Ellis 8fda24a776 Write a .yarnrc file when running tests to enable mutex
In #411 we started to run tests in parallel again. To support that, we
added a .yarnrc file in the root of our repository to pass --mutex
network to all yarn invocations, because tests may run yarn commands
concurrently with one another and yarn is not safe to run
concurrently.

However, when we run the integration tests, we actually copy them into
a folder under `/tmp` and so yarn's logic to walk up the directory to
tree to find a `.yarnrc` will not see this `.yarnrc` and we're back
where we started.

Have the testing package explicitly write this file, which should
prevent this issue from happening in the future.
2017-10-16 16:08:50 -07:00
Matt Ellis 996ac8872d Merge pull request #420 from pulumi/rename-env-to-stack
Use `Stack` over `Environment` to describe a deployment target
2017-10-16 14:24:11 -07:00
Pat Gavlin ea98499548 Merge pull request #418 from pulumi/EventChannel
Ensure a plugin's std{out,err} streams are drained in Close().
2017-10-16 14:09:29 -07:00
pat@pulumi.com bdbb1b59d4 Ensure a plugin's std{out,err} streams are drained in Close().
Not doing so can cause panics, as the goroutines we use to copy these
streams can end up writing to a closed channel.
2017-10-16 13:44:37 -07:00
Matt Ellis 22c9e0471c Use Stack over Environment to describe a deployment target
Previously we used the word "Environment" as the term for a deployment
target, but since then we've started to use the term Stack. Adopt this
across the CLI.

From a user's point of view, there are a few changes:

1. The `env` verb has been renamed to `stack`
2. The `-e` and `--env` options to commands which operate on an
environment now take `-s` or `--stack` instead.
3. Becase of (2), the commands that used `-s` to display a summary now
only support passing the full option name (`--summary`).

On the local file system, we still store checkpoint data in the `env`
sub-folder under `.pulumi` (so we can reuse existing checkpoint files
that were written to the old folder)
2017-10-16 13:04:20 -07:00
Matt Ellis 6fc250447d Merge pull request #411 from pulumi/move-build-into-make
Restore TESTPARALLELISM to 10
2017-10-16 11:56:47 -07:00
Matt Ellis be1e698f0b Restore TESTPARALLELISM to 10
Now that we no longer recursively call yarn via yarn run and scripts,
we can use a `--mutex network` directive in a .yarnrc file at the root
of the repository to prevent multiple copies of yarn running
concurrently.
2017-10-16 10:47:37 -07:00
Matt Ellis fc14329cfc Stop using yarn scripts for building
We have Makefiles, so we might as well use them, instead of spliting
build logic across two systems.
2017-10-16 10:47:37 -07:00
Joe Duffy 3169dae6e0 Merge pull request #415 from pulumi/340_components
Implement components
2017-10-15 05:54:43 -07:00
joeduffy 301739c6b5 Add auto-parenting
This changes a few things about "components":

* Rename what was previously ExternalResource to CustomResource,
  and all of the related fields and parameters that this implies.
  This just seems like a much nicer and expected name for what
  these represent.  I realize I am stealing a name we had thought
  about using elsewhere, but this seems like an appropriate use.

* Introduce ComponentResource, to make initializing resources
  that merely aggregate other resources easier to do correctly.

* Add a withParent and parentScope concept to Resource, to make
  allocating children less error-prone.  Now there's no need to
  explicitly adopt children as they are allocated; instead, any
  children allocated as part of the withParent callback will
  auto-parent to the resource provided.  This is used by
  ComponentResource's initialization function to make initialization
  easier, including the distinction between inputs and outputs.
2017-10-15 04:38:26 -07:00
joeduffy fbfca58a3f Implement components
This change implements core support for "components" in the Pulumi
Fabric.  This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.

In a nutshell, resources no longer imply external providers.  It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.

For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.

To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.

All resources now have the ability to adopt children.  This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).

Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output.  For instance:

    + aws:serverless:Function: (create)
      [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
      => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
      => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
      => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda

The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 18:30:59 -07:00
joeduffy d41c016edb Use install.sh 2017-10-14 07:24:20 -07:00
Matthew Riley d0472f0ac4 Merge pull request #410 from pulumi/yarn-cache
Fix Yarn errors during build
2017-10-13 15:34:56 -07:00
Matthew Riley a4fba1c5ee Limit test parallelism to avoid Yarn cache corruption
See e.g. https://travis-ci.com/pulumi/pulumi/builds/56843128
2017-10-13 15:18:16 -07:00
Matthew Riley 4daa7ca342 Update to latest version of Yarn 2017-10-13 14:44:08 -07:00
Pat Gavlin 9a685fcf0f Merge pull request #407 from pulumi/ReceiveOnlyEvents
Narrow event channel types.
2017-10-13 12:42:16 -07:00
Pat Gavlin aaf46c12f4 Narrow destroy events as well. 2017-10-13 10:12:09 -07:00
pat@pulumi.com 73bb5851c8 Accept a receive-only channel in displayEvents. 2017-10-12 14:20:01 -07:00
pat@pulumi.com 107f667b87 Accept a send-only event channel in Deploy and Preview.
Just what it says on the tin.
2017-10-12 14:16:44 -07:00
joeduffy 7aff81db2c Tidy up the examples
* Remove the bitrotted and useless basic/abc/ test.

* No need for the basic/ subdirectory.  Move minimal to the top.

* Update TypeScript to 2.5.3.

* Check in lockfiles to ensure repeatability in Travis tests.
2017-10-12 06:05:14 -07:00
joeduffy 65184ec6bd Enable PULUMI_CONFIG envvars, use them
This change adds environment variable fallbacks for configuration
variables, such that you can either set them explicitly, as a specific
variable PULUMI_CONFIG_<K>, or an entire JSON serialized bag via
PULUMI_CONFIG.

This is convenient when simply invoking programs at the command line,
via node, e.g.

    PULUMI_CONFIG_AWS_CONFIG_REGION=us-west-2 node bin/index.js

Our language host also now uses this to communicate config when invoking
a Run RPC, rather than at the command line.  This fixes pulumi/pulumi#336.
2017-10-11 18:41:52 -07:00
joeduffy ce87899792 Add Maybe<T> as a shortcut for T | undefined
This arose during a conversation with @CyrusNajmabadi, where he
suggested it would be useful in user code to have a "name" for these,
since they show up so frequently during resource property consumption.
2017-10-11 17:35:44 -07:00
Pat Gavlin ee410bfe1e Add a mock resource provider for testing purposes. (#401)
This resource provider accepts a single configuration parameter, `testing:provider:module`, that is the path to a Javascript module that implements CRUD operations for a set of resource types. This allows e.g. a test case to provide its own implementation of these operations that may succeed or fail in interesting ways.

Fixes #338.
2017-10-11 15:27:34 -07:00
Matt Ellis e7c3aaba69 Merge pull request #395 from pulumi/pulumi-service-interface
More engine refactoring
2017-10-11 13:44:36 -07:00
Matt Ellis 12cabcb187 Remove unneeded validation logic
Our checkpoint file is now in a shape such that go's built in
marshalling and unmarshalling is sufficent, so we can remove the code
we had on our decode path to do extra validation
2017-10-11 13:24:27 -07:00
Matt Ellis 2676e8bad1 Split apart EnvironmentProvider interface 2017-10-11 13:23:44 -07:00
Matt Ellis 9660da4d14 Run Travis on both macOS and Linux
Unlike go binaries (where we can cross compile) the node module that
we publish needs to be built on the platform we publish for. Update
our `.travis.yml` file to also build on macOS and fix the publishing
script so we don't don't cross publish Darwin from Linux. Once we have
CI working for Windows, we'll remove the loop over PUBLISH_GOOS and
each build will publish just the artifacts for the host OS.
2017-10-11 11:40:35 -07:00
CyrusNajmabadi b713990b5e Enable 'use const' linter rule. (#405)
* Enable 'use const' linter rule.
2017-10-10 14:50:55 -07:00
joeduffy 21812ef4b5 Pretty-format build status table 2017-10-10 13:41:45 -07:00
joeduffy 66b4c2df66 Add a Windows build badge 2017-10-10 13:40:11 -07:00
Matt Ellis 6f4537010c Fix typo 2017-10-10 12:16:41 -07:00
joeduffy d52c29b763 Make "up" a short-alias for "update" 2017-10-10 08:52:04 -07:00
CyrusNajmabadi 556345c68e Include environment when computing the hash of closure. (#403)
* Include environment when computing the hash of closure.
2017-10-09 21:46:24 -07:00
Matt Ellis 377eb61e32 Always emit debug events into the stream 2017-10-09 18:27:05 -07:00
Matt Ellis 7587bcd7ec Have engine emit "events" instead of writing to streams
Previously, the engine would write to io.Writer's to display output.
When hosted in `pulumi` these writers were tied to os.Stdout and
os.Stderr, but other applications hosting the engine could send them
other places (e.g. a log to be sent to an another application later).

While much better than just using the ambient streams, this was still
not the best. It would be ideal if the engine could just emit strongly
typed events and whatever is hosting the engine could care about
displaying them.

As a first step down that road, we move to a model where operations on
the engine now take a `chan engine.Event` and during the course of the
operation, events are written to this channel. It is the
responsibility of the caller of the method to read from the channel
until it is closed (singifying that the operation is complete).

The events we do emit are still intermingle presentation with data,
which is unfortunate, but can be improved over time. Most of the
events today are just colorized in the client and printed to stdout or
stderr without much thought.
2017-10-09 18:24:56 -07:00
Matt Ellis 5fd0ada303 Remove Checkpoint return value from GetEnvironment 2017-10-09 18:21:55 -07:00
Matt Ellis e7e4e75af3 Don't examine the Checkpoint in the CLI
The checkpoint is an implementation detail of the storage of an
environment. Instead of interacting with it, make sure that all the
data we need from it either hangs off the Snapshot or Target
objects (which you can get from a Checkpoint) and then start consuming
that data.
2017-10-09 18:21:55 -07:00
Matt Ellis 6e8185884e Remove GetEnvironmentInfo from Engine 2017-10-09 18:21:55 -07:00
Matt Ellis 7e4a1f515b Remove GetEnvironments from Engine 2017-10-09 18:21:55 -07:00