Commit graph

1280 commits

Author SHA1 Message Date
joeduffy 3e4e791fa4 Also overwrite output property slots
This change overwrites output property slots in runtime objects
after performing a CRUD operation, in addition to null or missing
slots, fixing #251.  The problem is that we sometimes have output
property values pre-populated in an object, and sometimes don't,
depending on various things (both are legal).  We should handle both.
2017-06-15 13:08:06 -07:00
Luke Hoban 1a21dd8368 Skip provider tests when AWS credentials are unavailable (#250)
Instead of asserting, we will skip these tests if the environment
does not provide access to credentials to create an AWS context.

This fixes #232.
2017-06-15 10:58:46 -07:00
Luke Hoban 1ba0bf7f95 Fix the serverless API example
The aws.serverless.API component was previously relying
on the fact that Lumi delayed resource creation until the
program was done executing.  With the changes to execution
for output properties, this no longer works.

For now, we will address this by change API to create the
RestAPI resource at the time of `publish`, after all of the
routes are already defined.
2017-06-15 09:06:40 -07:00
joeduffy 9698309f2b Model resource ID and URN as output properties
This change exposes ID and URN properties on resources, as appropriate,
so that they may be read and used in Lumi scripts.
2017-06-14 17:00:13 -07:00
joeduffy 2ac303f703 Fix deployment hang (pulumi/lumi#246)
The recent change to run the interpreter and planner on separate goroutines
created the need to perform rendezvous-style synchronization between them.
Although the case of an invoked function properly tore down the synchronization
by communicating the error, we seldom directly invoke functions for JavaScript
programs because the way module entrypoint code ends up in initializers.
This requires that we propagate errors correctly out of module and class
initializers, in the standard way, so that the unwind makes its way to the top.

This fixes pulumi/lumi#246.
2017-06-14 15:52:36 -07:00
joeduffy 3a899b304e Fix empty body issues
We recently changed the Resource base type to have no constructor,
rather than a manual empty constructor.  This ought to work just fine.
The LumiJS compiler indeed generates a constructor, however, it is
missing a body and when the interpreter tries to invoke it, we crash
with a nil reference panic.  The runtime actually tolerates missing
constructors entirely, although the way LumiJS binds super calls
doesn't tolerate the missing base constructor.  This change simply
generates such constructors in LumiJS with empty bodies.

In addition, I've added an error that will catch the empty body
problem during binding, since technically speaking, all functions
must have bodies.  (Our runtime happens to support the notion of
"abstract", however, so we only fire the error on concrete functions.)
2017-06-14 10:30:46 -07:00
joeduffy 792490e814 Fix the polarity of some asserts 2017-06-14 09:44:58 -07:00
joeduffy 54b19a5a42 Add missing yarn link to Makefile 2017-06-13 18:19:04 -07:00
joeduffy a42b40f1a8 Refactor runtime functionality
This change splits the core Lumi library -- which is meant to be a pure
LumiJS library without any special status -- from the runtime library --
which is really meant to be the underpinnings of "special" functionality
that integrates with the runtime in sophisticated ways.

After this change, LumiRT is at the very bottom, and, despite it using
a subset of LumiJS, it must not trigger any functionality that would
mandate the use of the LumiJS runtime library.  Atop that, the LumiJS
library is layered.  And finally, above that, Lumi depends on LumiJS.
2017-06-13 18:11:59 -07:00
Luke Hoban 3a47f2ca72 Merge remote-tracking branch 'origin/master' into bforsyth927-gometalinter 2017-06-13 18:04:36 -07:00
Luke Hoban 9a0575b518 Allow classes without explicit constrctors
When a class has no constructor, we automatically generate an empty
constructor in the Lumipack.

This allows us to adhere to tslint rule suggesting leaving off empty
constructors with default signatures.
2017-06-13 17:54:45 -07:00
joeduffy 1411f33c0c Print the install location in some Makefiles 2017-06-13 17:14:34 -07:00
Luke Hoban 441f32d155 A few tweaks to lint fixes 2017-06-13 16:47:55 -07:00
Luke Hoban 282f40d3e3 Merge branch 'master' into bforsyth927-gometalinter 2017-06-13 16:28:12 -07:00
Luke Hoban cdd5471cfe Merge branch 'gometalinter' of https://github.com/bforsyth927/lumi into bforsyth927-gometalinter 2017-06-13 16:15:12 -07:00
Luke Hoban e915dd3b42 Upgrade LumiJS Typescript dependency to 2.3.4
Fixes #242.
2017-06-13 15:48:14 -07:00
Britton Forsyth 01003ad48b Implemented highlighted edits 2017-06-13 11:01:23 -07:00
Joe Duffy bfe7db03c9 Merge pull request #236 from pulumi/90-plan_revamp
Implement output properties
2017-06-13 07:27:16 -07:00
joeduffy 01bbc23144 Only mark immediate outputs as outputs
The primary purposes of this change is to mark only immediate ouptuts
on a resource object as "output" and categories the rest as computed.

It also contains a few minor things:

    * Rebase atop the latest in master.

    * Always marshal unknows as their default value.

    * Permit computed as the existing ID property, in addition to null.

    * Tidy up some asserts.
2017-06-13 07:10:13 -07:00
joeduffy 0d836ae0bd Recover from deployment failures 2017-06-13 07:10:13 -07:00
joeduffy 2eb78e1036 Label replacements as simply "replace" 2017-06-13 07:10:13 -07:00
joeduffy 75a2f14d10 Propagate IDs/outs differently based on step kind
This change updates the ID/output propagation logic to properly handle
the case of replacements, in addition to accurately conveying the fact
that an update may change the values of output properties (but not the ID).

Also fixes a formatting issue with the replacement diffing displays.
2017-06-13 07:10:13 -07:00
joeduffy 25c52a04c5 Tidy up some loose ends
This removes some loose ends and reimplements `lumi pack eval`.
2017-06-13 07:10:13 -07:00
joeduffy dedeab3c5e Update test util to use new packages
Also add a license, fix some long lines, and make it gofmt clean.
2017-06-13 07:10:13 -07:00
joeduffy dd9e6b35f4 Introduce an OpSame planning step
This change introduces an OpSame planning step.  The reason we need
this is so that we can apply the necessary output properties, including
the ID, even as we are simply walking the plan (i.e., when we aren't
actually performing a deployment).  This ensures that the object state
evolves as required to let reads of output properties propagate in the
ways necessary to reproduce past executions of the program.
2017-06-13 07:10:13 -07:00
joeduffy 9d73c54994 Run post-construction hooks before freezing
We need to run the post-construction hook *before* freezing an object's
readonly properties, since the hook will actually mutate the object in
the case of a deployment (it stores the output properties).  In a sense,
this hook simply becomes an extension of the object's constructor.
2017-06-13 07:10:13 -07:00
joeduffy d1414af321 Fix a few minor things; clean stuff up
* Assert new things in new places.

* Log more interesting tidbits during evaluation.

* Invoke the OnStart hook before triggering initializers.

* Tolerate nil prev snapshots during deletion calculation.

* Handle and serialize missing resource IDs as output props.

* Return "done" flag from Rendezvous.Meet.
2017-06-13 07:10:13 -07:00
joeduffy d277dd5800 More progress on pulumi/lumi#90
This change refactors a number of aspects of the CLI's treatment of
steps, in line with the new scheme, and a number of other miscellaneous
and minor fixes.  It also regenerates all RPC code impacted by recent renames.
2017-06-13 07:10:13 -07:00
joeduffy d044720045 Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.

The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.

The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine.  It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.

There are two other sources, however.  First, a nullSource, which simply
refuses to create new objects.  This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment.  Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.

Boatloads of code is now changed and updated in the various CLI commands.

This further chugs along towards pulumi/lumi#90.  The end is in sight.
2017-06-13 07:10:13 -07:00
joeduffy 6b2408e086 Rewrite plans and deployments
This change guts the deployment planning and execution process, a
necessary component of pulumi/lumi#90.

The major effect of this change is that resources are actually
connected to the live objects, instead of being snapshots taken at
inopportune moments in time.
2017-06-13 07:10:13 -07:00
joeduffy c53ddeb678 Overhaul resources, planning, and environments
This change, part of pulumi/lumi#90, overhauls quite a bit of the
core resource, planning, environments, and related areas.

The biggest amount of movement comes from the splitting of pkg/resource
into multiple sub-packages.  This results in:

- pkg/resource: just the core resource data structures.

- pkg/resource/deployment: all planning and deployment logic.

- pkg/resource/environment: all environment, configuration, and
      serialized checkpoint structures and logic.

- pkg/resource/plugin: all dynamically loaded analyzer and
      provider logic, including the actual loading and RPC mechanisms.

This also splits the resource abstraction up.  We now have:

- resource.Resource: a shared interface.

- resource.Object: a resource that is connected to a live object
      that will periodically observe mutations due to ongoing
      evaluation of computations.  Snapshots of its state may be
      taken; however, this is purely a "pre-planning" abstraction.

- resource.State: a snapshot of a resource's state that is frozen.
      In other words, it is no longer connected to a live object.
      This is what will store provider outputs (ID and properties),
      and is what may be serialized into a deployment record.

The branch is in a half-baked state as of this change; more changes
are to come...
2017-06-13 07:10:13 -07:00
Luke Hoban 29fcde459b Add demo script and raw serverless example
Adds an initial cut at a demo script along with
a raw version of the serverless example that
is a better stepping stone between the low-level
AWS infrastructure providers and the high-level
`aws.serverless` APIs.
2017-06-12 15:20:37 -07:00
Luke Hoban 2914505a08 Expose the HTTP URL as an APIGateway Stage output property
Adds two output properties on APIGateway Stage resources.
* `url` is the full URL to the root of the deployed stage.
* `executionARN` is the arn needed to pass to Lambda to
give the stage permission to invoke a Lambda handler.
2017-06-12 10:40:41 -07:00
Luke Hoban 9bd441be05 Support for nested lambdas and node_modules
LumiJS lambdas can now be serialized when they include calls to other LumiJS lambdas.  The chain of lambda dependencies is jointly serialized into the target Lambda.

Also, LumiJS lambdas now include `node_modules` automatically in the AWS Lambda, ensuring the the runtime execution environment more closely matches the deployment time environment.

An early version of the gh-cicd example supporting #134 is added which uses these capabilities, currently including a mocked GitHub resource provider.
2017-06-12 10:15:20 -07:00
joeduffy 19febfcd9a Use a typedef for serverless function handler 2017-06-12 08:38:11 -07:00
joeduffy 97468ef38d Add an invoke.sh script for easier serverless demos 2017-06-10 15:59:53 -07:00
joeduffy 2b7b8c6160 Increase the default AWS timeout
We just hit a CI failure due to a timeout waiting for a lambda's
permissions to become available.  The AWS documents say "it may take
a few seconds", which is remarkably unhelpful.  Our current default
wait time is 30 seconds.  I am bumping it to 1 minute.

I also filed pulumi/lumi#233 for future consideration, since I would
like to think we can be more principled in our approach here...
2017-06-10 12:45:36 -07:00
joeduffy 7fb9a977db Fix a typo: "intance" should be "instance" 2017-06-10 12:15:03 -07:00
Luke Hoban 9bb868191f Add support for template literals in LumiJS
Support for untagged template literals.

Also unblocks a couple of cases where dynamic was not
propogated through the binder correctly.

Fixes #102.
2017-06-09 18:46:09 -07:00
Britton Forsyth 69e4834f63 Merge branch 'master' into gometalinter 2017-06-09 14:34:51 -07:00
Britton Forsyth 8c8f68d4ae Added specified changes 2017-06-09 12:51:31 -07:00
Luke Hoban 07ac77b903 Merge pull request #229 from pulumi/freevars
Improved lambda environment serialization
2017-06-09 11:35:53 -07:00
Luke Hoban 2870e8a0b1 Merge branch 'gometalinter' of https://github.com/bforsyth927/lumi into bforsyth927-gometalinter 2017-06-09 10:30:50 -07:00
Luke Hoban e87d21cb80 Store module object on Functions
We now store the module object instead of the
symbol on function stubs.
2017-06-09 10:23:07 -07:00
Luke Hoban 7171b58459 Address PR feedback on #229 2017-06-09 10:15:03 -07:00
Britton Forsyth 13dbcdbafc Finalized tslint edits and deletes 2017-06-09 08:54:05 -07:00
Luke Hoban 705c0edbfc Fix lumijs tests
Update baselines for Lumijs tests after change to
emit `TryLoadDynamic` for module-scoped
variable references.
2017-06-08 22:22:55 -07:00
Luke Hoban d77c51ff7f Allow runtime lambda to reference globals.
For lambdas which will execute at runtime,
we want to allow them to reference Node.js
global variables, like `console`.

This change makes Lumijs generated IL
incrementally more dynamic by preferring to
generate `TryLoadDynamic` over `LoadLocation`
for references to global variables (except for
references to imports).

Also introduces `console.log` in LumiJS, though
it is not yet attached to a Lumi global environment.

Fixes #174.
2017-06-08 22:06:41 -07:00
Britton Forsyth 11c74f12d9 Fixed tslint issues 2017-06-08 16:40:12 -07:00
Luke Hoban e838c6ff2d Allow lambdas to capture reference to module scope variables
The scope chain currently does not include module-scope
vairables, which are instead stored on a module object.  For
now, we are capturing this module object along with the
scope chain as part of a Lambda object so that we can use
it when evaluating variable references within a lambda
expression.

Fixes #175.
2017-06-08 15:56:50 -07:00