Commit graph

135 commits

Author SHA1 Message Date
Pat Gavlin afd7c400ad Remove the testing provider.
This provider has been obviated by dynamic resources.
2017-10-16 23:06:53 -07:00
pat@pulumi.com 9453f86c2e Implement dynamic resources.
A dynamic resource is a resource whose provider is implemented alongside
the resource itself. This provider may close over and use orther
resources in the implementation of its CRUD operations. The provider
itself must be stateless, as each CRUD operation for a particular
dynamic resource type may use an independent instance of the provider.
Changes to the definition of a resource's provider result in replacement
of the resource itself (rather than a simple update), as this allows the
old provider definition to delete the old resource and the new provider
definition to create an appropriate replacement.
2017-10-16 23:06:53 -07:00
joeduffy 301739c6b5 Add auto-parenting
This changes a few things about "components":

* Rename what was previously ExternalResource to CustomResource,
  and all of the related fields and parameters that this implies.
  This just seems like a much nicer and expected name for what
  these represent.  I realize I am stealing a name we had thought
  about using elsewhere, but this seems like an appropriate use.

* Introduce ComponentResource, to make initializing resources
  that merely aggregate other resources easier to do correctly.

* Add a withParent and parentScope concept to Resource, to make
  allocating children less error-prone.  Now there's no need to
  explicitly adopt children as they are allocated; instead, any
  children allocated as part of the withParent callback will
  auto-parent to the resource provided.  This is used by
  ComponentResource's initialization function to make initialization
  easier, including the distinction between inputs and outputs.
2017-10-15 04:38:26 -07:00
joeduffy fbfca58a3f Implement components
This change implements core support for "components" in the Pulumi
Fabric.  This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.

In a nutshell, resources no longer imply external providers.  It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.

For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.

To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.

All resources now have the ability to adopt children.  This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).

Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output.  For instance:

    + aws:serverless:Function: (create)
      [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
      => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
      => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
      => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda

The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 18:30:59 -07:00
joeduffy 7aff81db2c Tidy up the examples
* Remove the bitrotted and useless basic/abc/ test.

* No need for the basic/ subdirectory.  Move minimal to the top.

* Update TypeScript to 2.5.3.

* Check in lockfiles to ensure repeatability in Travis tests.
2017-10-12 06:05:14 -07:00
Pat Gavlin ee410bfe1e Add a mock resource provider for testing purposes. (#401)
This resource provider accepts a single configuration parameter, `testing:provider:module`, that is the path to a Javascript module that implements CRUD operations for a set of resource types. This allows e.g. a test case to provide its own implementation of these operations that may succeed or fail in interesting ways.

Fixes #338.
2017-10-11 15:27:34 -07:00
Joe Duffy f6e694c72b Rename pulumi-fabric to pulumi
This includes a few changes:

* The repo name -- and hence the Go modules -- changes from pulumi-fabric to pulumi.

* The Node.js SDK package changes from @pulumi/pulumi-fabric to just pulumi.

* The CLI is renamed from lumi to pulumi.
2017-09-21 19:18:21 -07:00
joeduffy f64b8e25e7 Clean up legacy examples 2017-09-07 07:59:16 -07:00
joeduffy 7c848bfff4 Add config to the basic/minimal test 2017-09-04 11:35:21 -07:00
joeduffy 590e9e539b Rename Lumi.yaml to Pulumi.yaml
And also eliminate lots of accumulated cruft around "packfiles", etc.
in the workspace code.
2017-09-04 11:35:21 -07:00
joeduffy 1df1b6d572 Get integration tests passing
This makes a few tweaks to get the integration tests passing:

* Add `runtime: nodejs` to the minimal example's `Lumi.yaml` file.

* Remove usage of `@lumi/lumirt { printf }` and just use `console.log`.

* Remove calls to `lumijs` in the integration test framework and
  the minimal example's package.json.  Instead, we just run
  `yarn run build`, which itself internally just invokes `tsc`.

* Add package validation logic and eliminate the pkg/compiler/metadata
  library, in favor of the simpler code in pkg/engine.

* Simplify the Node.js langhost plugin CLI, and simply take an
  argument rather than requiring required and optional --flags.

* Use a default path of "." if the program path isn't provided.  This
  is a legal scenario if you've passed a pwd and just want to load
  the package's default module ("./index.js" or whatever main says).

* Add an executable script, lumi-langhost-nodejs, that fires up the
  `bin/cmd/langhost/index.js` file to serve the Node.js language plugin.
2017-09-04 11:35:21 -07:00
joeduffy 2f4d7a9789 Run integration tests
This change runs the examples integration tests for every test
run.  They used to be split out because the AWS tests take so long,
but now those are in their own separate package.  Running the
integration tests here more frequently will prevent breaking the
most basic Lumi CLI commands and capabilities.
2017-08-06 08:26:20 -07:00
joeduffy 35aa6b7559 Rename pulumi/lumi to pulumi/pulumi-fabric
We are renaming Lumi to Pulumi Fabric.  This change simply renames the
pulumi/lumi repo to pulumi/pulumi-fabric, without the CLI tools and other
changes that will follow soon afterwards.
2017-08-02 09:25:22 -07:00
joeduffy d6b6cbf4ff Build/test the minimal Lumi program as a nightly integration test 2017-07-23 12:12:43 -06:00
joeduffy e35bfc35b2 Remove AWS examples
These are moving to their respective repos.
2017-07-21 14:00:30 -07:00
joeduffy 87b7091b35 Remove serverless examples
These have moved to [pulumi/aws-serverless/examples](
https://github.com/pulumi/aws-serverless/examples/).
2017-07-21 14:00:29 -07:00
Luke Hoban 92418cf687 Refactor integration testing framework
Generalizes Lumi program validation so that it can be applied
to integration testing for other packages (such as the
pulumi/lumi-platform package examples).
2017-07-13 12:20:57 -07:00
Luke Hoban 1d7792cd83 Allow additional configuration on aws.serverless.Function
We need to provide higher level abstractions with the
ability to set additional properties on a Lambda Function
even when using the closure serialization support of
aws.serverless.Function.

Note that this is an API breaking change, and may require
updates in any other libraries dependent on this API.
2017-07-09 21:03:31 -07:00
Luke Hoban dc6594f06b Temporary workaround for #276
Disable invocation of `lumi plan` during examples
integration testing, pending resolution of #276 to
support planning in the face of output properties.
2017-06-29 10:49:05 -07:00
Luke Hoban 9ba8567d59 Support for AWS Cloudwatch Logs
Adds support for LogGroup and LogSubscriptionFilter
resources.
2017-06-28 15:42:42 -07:00
joeduffy 2daea4c3d8 Clarify aspects of using the DCO 2017-06-26 14:46:34 -07:00
joeduffy 3c1041af49 Update license headers 2017-06-23 14:53:41 -07:00
Luke Hoban 201a3d1a51 Additional nightly test fixes
Address several issues with running the Beanstalk
example in newer AWS regions with different requirements.

Ensures S3 bucket names adhere to required naming patterns
outside of us-east-1.

Also add InstanceProfile and ServiceRole configuration to the
beanstalk example as required in newer regions.
2017-06-22 16:59:16 -07:00
Luke Hoban 72bb2b2309 Further improvements for nightly tests
Make RestAPI more robust to TooManyRequestsException.

Fix imports in minimal example.

Make printing for examples test more explicit to help with diagnostics during parallel test execution.
2017-06-21 17:33:47 -07:00
Britton Forsyth 233c5a8c52 [#245, #226, #185] Finished linting (#257)
* Added typescript linting to Makefiles
* Added tslint to CI and ran on examples
* Fixed tslint calling in Makefiles
* Deleted unnecessary files
2017-06-21 11:58:22 -07:00
Luke Hoban bcc37d78a8 Nightly test fixes
Fix a couple of issues that have been preventing nightlies
from running cleanly.
2017-06-20 12:29:04 -07:00
Luke Hoban ac4a56bb6d Additional tslint cleanup in examples 2017-06-19 17:09:55 -07:00
Luke Hoban e6509e3814 Make examples tslint clean 2017-06-16 13:44:29 -07:00
Luke Hoban 6840dba051 Move examples test to seperate package
We need to run examples tests only after building and
installing all Lumi commands.
2017-06-16 09:24:31 -07:00
Luke Hoban 639a2d323d Test more examples
Tests all of our commonly used examples.

Also sets test parallelism to 10 by default
since we are I/O bound on API calls to
the resource providers.

Also avoids using larger EC2 examples in
our samples so that we can keep our test
costs lower :-).
2017-06-16 09:24:31 -07:00
Luke Hoban ae03d69645 Wire up APIs to lambdas using output properties
We now have enough output properties implementation
working to change our API gateway examples and API
wrapper to correctly wire the API routes to the ARNs of
lambdas passed in to them.

We both wire up the lambda to the route, but also create
a permission specific to each route to assign to the
corresponding lambda - providing least privelege needed
for the API definition.

Also adds `string#toUpperCase` and fixes NewUniqueHex
to match how we are using it.
2017-06-15 16:01:00 -07:00
Luke Hoban 1ba0bf7f95 Fix the serverless API example
The aws.serverless.API component was previously relying
on the fact that Lumi delayed resource creation until the
program was done executing.  With the changes to execution
for output properties, this no longer works.

For now, we will address this by change API to create the
RestAPI resource at the time of `publish`, after all of the
routes are already defined.
2017-06-15 09:06:40 -07:00
Luke Hoban 282f40d3e3 Merge branch 'master' into bforsyth927-gometalinter 2017-06-13 16:28:12 -07:00
Britton Forsyth 01003ad48b Implemented highlighted edits 2017-06-13 11:01:23 -07:00
Luke Hoban 29fcde459b Add demo script and raw serverless example
Adds an initial cut at a demo script along with
a raw version of the serverless example that
is a better stepping stone between the low-level
AWS infrastructure providers and the high-level
`aws.serverless` APIs.
2017-06-12 15:20:37 -07:00
Luke Hoban 9bd441be05 Support for nested lambdas and node_modules
LumiJS lambdas can now be serialized when they include calls to other LumiJS lambdas.  The chain of lambda dependencies is jointly serialized into the target Lambda.

Also, LumiJS lambdas now include `node_modules` automatically in the AWS Lambda, ensuring the the runtime execution environment more closely matches the deployment time environment.

An early version of the gh-cicd example supporting #134 is added which uses these capabilities, currently including a mocked GitHub resource provider.
2017-06-12 10:15:20 -07:00
joeduffy 97468ef38d Add an invoke.sh script for easier serverless demos 2017-06-10 15:59:53 -07:00
Britton Forsyth 69e4834f63 Merge branch 'master' into gometalinter 2017-06-09 14:34:51 -07:00
Britton Forsyth 13dbcdbafc Finalized tslint edits and deletes 2017-06-09 08:54:05 -07:00
Luke Hoban d77c51ff7f Allow runtime lambda to reference globals.
For lambdas which will execute at runtime,
we want to allow them to reference Node.js
global variables, like `console`.

This change makes Lumijs generated IL
incrementally more dynamic by preferring to
generate `TryLoadDynamic` over `LoadLocation`
for references to global variables (except for
references to imports).

Also introduces `console.log` in LumiJS, though
it is not yet attached to a Lumi global environment.

Fixes #174.
2017-06-08 22:06:41 -07:00
Britton Forsyth 11c74f12d9 Fixed tslint issues 2017-06-08 16:40:12 -07:00
Luke Hoban e838c6ff2d Allow lambdas to capture reference to module scope variables
The scope chain currently does not include module-scope
vairables, which are instead stored on a module object.  For
now, we are capturing this module object along with the
scope chain as part of a Lambda object so that we can use
it when evaluating variable references within a lambda
expression.

Fixes #175.
2017-06-08 15:56:50 -07:00
Luke Hoban 771a30c688 Save build artifacts aftifacts for Go builds
In the places we run `go build`, we should use
`go build -i` to save the `.a` files generated
during the build.  This ensures the artifacts
are availble for other Go tools (linters, IDEs), and
should also improve build speeds.
2017-06-07 17:03:07 -07:00
Luke Hoban 73e4be9f5e Some cleanup on AWS examples 2017-06-07 16:16:55 -07:00
joeduffy c2f2fbd2ff Regenerate some code 2017-06-04 19:37:27 -07:00
Luke Hoban 92a9925201 Merge branch 'master' into apigateway 2017-06-03 14:58:23 -07:00
Luke Hoban 5358080ca6 Output property improvements for AWS Function and Role
The AssumeRolePolicyDocument property returned by the AWS IAM GetRole API returns
a URL-encoded JSON string, so we need to decode this before JSON unmarshalling.

The Code property returned by AWS Lambda GetFunction provides a pre-signed S3 URL,
which changes on each call, and is of a different format to what is provided by the user.
For now, we'll not store this back into the Function object.

Add additional output properties to AWS Lambda Function that are stable values returned
from GetFunction.

Also corrects a gap where some property delete operations were not being correctly reported.
2017-06-01 15:04:37 -07:00
Luke Hoban c117b43ae4 Update serverless API programming model
Updates  the higher level AWS APIGateway programming model
in aws.serverless.API  to use an Express-like imperative API.
2017-06-01 10:54:26 -07:00
joeduffy 4d63e6d672 Add a few more RawResources: trues 2017-06-01 08:39:48 -07:00
Luke Hoban 01af21a1e4 Support for multiple methods on route in aws.serverless.API
Also adds length property to String objects and a toLowerCase method to the String prototype.
2017-05-31 11:45:02 -07:00