Commit graph

1494 commits

Author SHA1 Message Date
Luke Hoban d862112682 Fix Makefile target names
Fix copy/paste issue in some new Makefile targets.
2017-06-16 10:34:34 -07:00
Luke Hoban 33a9452ece Merge pull request #256 from pulumi/examplestest
Add integration testing for examples
2017-06-16 10:17:51 -07:00
joeduffy 5e85e6d543 Improve property validation diagnostics
The change prints the value of a property that fails resource validation.
This makes it much easier to diagnose what's going on.
2017-06-16 10:04:49 -07:00
Luke Hoban e27549e5c2 Add nightly build target
Adds a nightly build target that runs the long running provider
and examples tests.  Enables travis to run this on cron jobs,
which we will configure to trigger nightly.
2017-06-16 09:53:07 -07:00
joeduffy 7d19abc2a3 Print the current environment
This change implements showing a summary of the current environment.
All you need to do is run

    $ lumi env

and the current environment's information will be printed.

This makes it convenient to grab resource information that might be
required, for instance, to correlate with logs (e.g., lambda ARNs).

Eventually, as per pulumi/lumi#184, we want to print details about
all of the resources too.
2017-06-16 09:46:09 -07:00
Luke Hoban 6840dba051 Move examples test to seperate package
We need to run examples tests only after building and
installing all Lumi commands.
2017-06-16 09:24:31 -07:00
Luke Hoban 639a2d323d Test more examples
Tests all of our commonly used examples.

Also sets test parallelism to 10 by default
since we are I/O bound on API calls to
the resource providers.

Also avoids using larger EC2 examples in
our samples so that we can keep our test
costs lower :-).
2017-06-16 09:24:31 -07:00
Luke Hoban 8d8eba5c65 Add integration testing for examples
Adds an integration test that runs the following commands on the
AWS webserver example, failing if any command returns an error
code:
* lumijs
* lumi env init
* lumi config
* lumi plan
* lumi deploy
* lumi destroy
* lumi env rm

Also ensures that plan and deploy failures propagate errors through
to error codes at the CLI.
2017-06-16 09:24:31 -07:00
joeduffy d013501e04 Restructure Rendezvous to have a distinct Let vs. Meet
On the first turn, we want to distinguish between a coroutine
running that owns its turn, and a coroutine that knows it doesn't
own the turn and is simply awaiting its turn.  The old Meet logic
wasn't quite right; instead, we'll have the caller tell us this.
2017-06-15 18:20:12 -07:00
joeduffy 34ea2e8e43 Rewrite some binop code to share more logic
Also add some logging to help diagnose some issues I was running into.
2017-06-15 17:29:51 -07:00
joeduffy e26f934aec Propagate computed values in assignments
This change fixes an issue with the way we deal with computed values in
assignments.  Specifically, the assignment expression should resolve to
the computed value itself, but it must actually perform the assignment!
Previously, we evaluated to the right thing, but skipped the assignment.
2017-06-15 17:04:16 -07:00
Luke Hoban ae03d69645 Wire up APIs to lambdas using output properties
We now have enough output properties implementation
working to change our API gateway examples and API
wrapper to correctly wire the API routes to the ARNs of
lambdas passed in to them.

We both wire up the lambda to the route, but also create
a permission specific to each route to assign to the
corresponding lambda - providing least privelege needed
for the API definition.

Also adds `string#toUpperCase` and fixes NewUniqueHex
to match how we are using it.
2017-06-15 16:01:00 -07:00
joeduffy b7a3b28734 Implement array push and pop
This implements array push and pop as intrinsics.

Also:

* Tighten up some assertions while I'm in here.

* Default initialize pointer slots to Null, if not done explicitly.
2017-06-15 14:51:57 -07:00
joeduffy 085e39230b Test missing and output properties
This tests the conditions that triggered pulumi/lumi#251.
2017-06-15 13:08:06 -07:00
joeduffy 3e4e791fa4 Also overwrite output property slots
This change overwrites output property slots in runtime objects
after performing a CRUD operation, in addition to null or missing
slots, fixing #251.  The problem is that we sometimes have output
property values pre-populated in an object, and sometimes don't,
depending on various things (both are legal).  We should handle both.
2017-06-15 13:08:06 -07:00
Luke Hoban 1a21dd8368 Skip provider tests when AWS credentials are unavailable (#250)
Instead of asserting, we will skip these tests if the environment
does not provide access to credentials to create an AWS context.

This fixes #232.
2017-06-15 10:58:46 -07:00
Luke Hoban 1ba0bf7f95 Fix the serverless API example
The aws.serverless.API component was previously relying
on the fact that Lumi delayed resource creation until the
program was done executing.  With the changes to execution
for output properties, this no longer works.

For now, we will address this by change API to create the
RestAPI resource at the time of `publish`, after all of the
routes are already defined.
2017-06-15 09:06:40 -07:00
joeduffy 9698309f2b Model resource ID and URN as output properties
This change exposes ID and URN properties on resources, as appropriate,
so that they may be read and used in Lumi scripts.
2017-06-14 17:00:13 -07:00
joeduffy 2ac303f703 Fix deployment hang (pulumi/lumi#246)
The recent change to run the interpreter and planner on separate goroutines
created the need to perform rendezvous-style synchronization between them.
Although the case of an invoked function properly tore down the synchronization
by communicating the error, we seldom directly invoke functions for JavaScript
programs because the way module entrypoint code ends up in initializers.
This requires that we propagate errors correctly out of module and class
initializers, in the standard way, so that the unwind makes its way to the top.

This fixes pulumi/lumi#246.
2017-06-14 15:52:36 -07:00
joeduffy 3a899b304e Fix empty body issues
We recently changed the Resource base type to have no constructor,
rather than a manual empty constructor.  This ought to work just fine.
The LumiJS compiler indeed generates a constructor, however, it is
missing a body and when the interpreter tries to invoke it, we crash
with a nil reference panic.  The runtime actually tolerates missing
constructors entirely, although the way LumiJS binds super calls
doesn't tolerate the missing base constructor.  This change simply
generates such constructors in LumiJS with empty bodies.

In addition, I've added an error that will catch the empty body
problem during binding, since technically speaking, all functions
must have bodies.  (Our runtime happens to support the notion of
"abstract", however, so we only fire the error on concrete functions.)
2017-06-14 10:30:46 -07:00
joeduffy 792490e814 Fix the polarity of some asserts 2017-06-14 09:44:58 -07:00
joeduffy 54b19a5a42 Add missing yarn link to Makefile 2017-06-13 18:19:04 -07:00
joeduffy a42b40f1a8 Refactor runtime functionality
This change splits the core Lumi library -- which is meant to be a pure
LumiJS library without any special status -- from the runtime library --
which is really meant to be the underpinnings of "special" functionality
that integrates with the runtime in sophisticated ways.

After this change, LumiRT is at the very bottom, and, despite it using
a subset of LumiJS, it must not trigger any functionality that would
mandate the use of the LumiJS runtime library.  Atop that, the LumiJS
library is layered.  And finally, above that, Lumi depends on LumiJS.
2017-06-13 18:11:59 -07:00
Luke Hoban 3a47f2ca72 Merge remote-tracking branch 'origin/master' into bforsyth927-gometalinter 2017-06-13 18:04:36 -07:00
Luke Hoban 9a0575b518 Allow classes without explicit constrctors
When a class has no constructor, we automatically generate an empty
constructor in the Lumipack.

This allows us to adhere to tslint rule suggesting leaving off empty
constructors with default signatures.
2017-06-13 17:54:45 -07:00
joeduffy 1411f33c0c Print the install location in some Makefiles 2017-06-13 17:14:34 -07:00
Luke Hoban 441f32d155 A few tweaks to lint fixes 2017-06-13 16:47:55 -07:00
Luke Hoban 282f40d3e3 Merge branch 'master' into bforsyth927-gometalinter 2017-06-13 16:28:12 -07:00
Luke Hoban cdd5471cfe Merge branch 'gometalinter' of https://github.com/bforsyth927/lumi into bforsyth927-gometalinter 2017-06-13 16:15:12 -07:00
Luke Hoban e915dd3b42 Upgrade LumiJS Typescript dependency to 2.3.4
Fixes #242.
2017-06-13 15:48:14 -07:00
Britton Forsyth 01003ad48b Implemented highlighted edits 2017-06-13 11:01:23 -07:00
Joe Duffy bfe7db03c9 Merge pull request #236 from pulumi/90-plan_revamp
Implement output properties
2017-06-13 07:27:16 -07:00
joeduffy 01bbc23144 Only mark immediate outputs as outputs
The primary purposes of this change is to mark only immediate ouptuts
on a resource object as "output" and categories the rest as computed.

It also contains a few minor things:

    * Rebase atop the latest in master.

    * Always marshal unknows as their default value.

    * Permit computed as the existing ID property, in addition to null.

    * Tidy up some asserts.
2017-06-13 07:10:13 -07:00
joeduffy 0d836ae0bd Recover from deployment failures 2017-06-13 07:10:13 -07:00
joeduffy 2eb78e1036 Label replacements as simply "replace" 2017-06-13 07:10:13 -07:00
joeduffy 75a2f14d10 Propagate IDs/outs differently based on step kind
This change updates the ID/output propagation logic to properly handle
the case of replacements, in addition to accurately conveying the fact
that an update may change the values of output properties (but not the ID).

Also fixes a formatting issue with the replacement diffing displays.
2017-06-13 07:10:13 -07:00
joeduffy 25c52a04c5 Tidy up some loose ends
This removes some loose ends and reimplements `lumi pack eval`.
2017-06-13 07:10:13 -07:00
joeduffy dedeab3c5e Update test util to use new packages
Also add a license, fix some long lines, and make it gofmt clean.
2017-06-13 07:10:13 -07:00
joeduffy dd9e6b35f4 Introduce an OpSame planning step
This change introduces an OpSame planning step.  The reason we need
this is so that we can apply the necessary output properties, including
the ID, even as we are simply walking the plan (i.e., when we aren't
actually performing a deployment).  This ensures that the object state
evolves as required to let reads of output properties propagate in the
ways necessary to reproduce past executions of the program.
2017-06-13 07:10:13 -07:00
joeduffy 9d73c54994 Run post-construction hooks before freezing
We need to run the post-construction hook *before* freezing an object's
readonly properties, since the hook will actually mutate the object in
the case of a deployment (it stores the output properties).  In a sense,
this hook simply becomes an extension of the object's constructor.
2017-06-13 07:10:13 -07:00
joeduffy d1414af321 Fix a few minor things; clean stuff up
* Assert new things in new places.

* Log more interesting tidbits during evaluation.

* Invoke the OnStart hook before triggering initializers.

* Tolerate nil prev snapshots during deletion calculation.

* Handle and serialize missing resource IDs as output props.

* Return "done" flag from Rendezvous.Meet.
2017-06-13 07:10:13 -07:00
joeduffy d277dd5800 More progress on pulumi/lumi#90
This change refactors a number of aspects of the CLI's treatment of
steps, in line with the new scheme, and a number of other miscellaneous
and minor fixes.  It also regenerates all RPC code impacted by recent renames.
2017-06-13 07:10:13 -07:00
joeduffy d044720045 Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.

The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.

The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine.  It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.

There are two other sources, however.  First, a nullSource, which simply
refuses to create new objects.  This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment.  Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.

Boatloads of code is now changed and updated in the various CLI commands.

This further chugs along towards pulumi/lumi#90.  The end is in sight.
2017-06-13 07:10:13 -07:00
joeduffy 6b2408e086 Rewrite plans and deployments
This change guts the deployment planning and execution process, a
necessary component of pulumi/lumi#90.

The major effect of this change is that resources are actually
connected to the live objects, instead of being snapshots taken at
inopportune moments in time.
2017-06-13 07:10:13 -07:00
joeduffy c53ddeb678 Overhaul resources, planning, and environments
This change, part of pulumi/lumi#90, overhauls quite a bit of the
core resource, planning, environments, and related areas.

The biggest amount of movement comes from the splitting of pkg/resource
into multiple sub-packages.  This results in:

- pkg/resource: just the core resource data structures.

- pkg/resource/deployment: all planning and deployment logic.

- pkg/resource/environment: all environment, configuration, and
      serialized checkpoint structures and logic.

- pkg/resource/plugin: all dynamically loaded analyzer and
      provider logic, including the actual loading and RPC mechanisms.

This also splits the resource abstraction up.  We now have:

- resource.Resource: a shared interface.

- resource.Object: a resource that is connected to a live object
      that will periodically observe mutations due to ongoing
      evaluation of computations.  Snapshots of its state may be
      taken; however, this is purely a "pre-planning" abstraction.

- resource.State: a snapshot of a resource's state that is frozen.
      In other words, it is no longer connected to a live object.
      This is what will store provider outputs (ID and properties),
      and is what may be serialized into a deployment record.

The branch is in a half-baked state as of this change; more changes
are to come...
2017-06-13 07:10:13 -07:00
Luke Hoban 29fcde459b Add demo script and raw serverless example
Adds an initial cut at a demo script along with
a raw version of the serverless example that
is a better stepping stone between the low-level
AWS infrastructure providers and the high-level
`aws.serverless` APIs.
2017-06-12 15:20:37 -07:00
Luke Hoban 2914505a08 Expose the HTTP URL as an APIGateway Stage output property
Adds two output properties on APIGateway Stage resources.
* `url` is the full URL to the root of the deployed stage.
* `executionARN` is the arn needed to pass to Lambda to
give the stage permission to invoke a Lambda handler.
2017-06-12 10:40:41 -07:00
Luke Hoban 9bd441be05 Support for nested lambdas and node_modules
LumiJS lambdas can now be serialized when they include calls to other LumiJS lambdas.  The chain of lambda dependencies is jointly serialized into the target Lambda.

Also, LumiJS lambdas now include `node_modules` automatically in the AWS Lambda, ensuring the the runtime execution environment more closely matches the deployment time environment.

An early version of the gh-cicd example supporting #134 is added which uses these capabilities, currently including a mocked GitHub resource provider.
2017-06-12 10:15:20 -07:00
joeduffy 19febfcd9a Use a typedef for serverless function handler 2017-06-12 08:38:11 -07:00
joeduffy 97468ef38d Add an invoke.sh script for easier serverless demos 2017-06-10 15:59:53 -07:00