Commit graph

1228 commits

Author SHA1 Message Date
Luke Hoban
e838c6ff2d Allow lambdas to capture reference to module scope variables
The scope chain currently does not include module-scope
vairables, which are instead stored on a module object.  For
now, we are capturing this module object along with the
scope chain as part of a Lambda object so that we can use
it when evaluating variable references within a lambda
expression.

Fixes #175.
2017-06-08 15:56:50 -07:00
Luke Hoban
ae9730bad3 Only capture free variables in lambda environment
Introduces a free variable AST visitor, and uses this to limit
the environment exposed by the `serizlizeClosure` intrinsic
to only those variables that are referenced by the lambda body.

Fixes #177.
2017-06-08 11:55:24 -07:00
joeduffy
2ab2b09474 Introduce object resources
This change slightly refactors the way resources are created and
implemented.  We now have two implementations of the Resource interface:

* `resource` (in resource_value.go), which is a snapshot of a resource's
  state.  All values are resolved and there is no live reference to any
  heap state or objects.  This will be used when serializing and/or
  deserializing snapshots of deployments.

* `objectResource` (in resource_object.go), which is an implementation
  of the Resource interface that wraps an underlying, live runtime object.
  This currently introduces no functional difference, as fetching Inputs()
  amounts to taking a snapshot of the full state.  But this at least
  gives us a leg to stand on in making sure that output properties are
  read at the right times during evaluation.

This is a fundamental part of pulumi/lumi#90.
2017-06-08 09:26:06 -07:00
joeduffy
027fe0766c Rename computed/output's eventual property to element
This rename now ensures the name mirrors that in the type/symbol layer.
2017-06-08 06:48:23 -07:00
joeduffy
dce2fae4c2 Track implicated objects
This change begins to track objects that are implicated in the
creation of computed values.  This ultimately translates into the
resource URNs which are used during dependency analysis and
serialization.  This is part of pulumi/lumi#90.
2017-06-08 06:46:28 -07:00
Luke Hoban
771a30c688 Save build artifacts aftifacts for Go builds
In the places we run `go build`, we should use
`go build -i` to save the `.a` files generated
during the build.  This ensures the artifacts
are availble for other Go tools (linters, IDEs), and
should also improve build speeds.
2017-06-07 17:03:07 -07:00
Luke Hoban
96c25a9614 Generate valid Permission names for aws.serverless.API
The previous checkin exposed some validation
errors that had previously been surpressed,
including an error validating the Permission
statementId's generated by aws.serverless.API.
2017-06-07 16:34:17 -07:00
Luke Hoban
73e4be9f5e Some cleanup on AWS examples 2017-06-07 16:16:55 -07:00
Luke Hoban
0f8bd74a82 Address PR feedback on #224
Adds Check implementation for aws.lambda.Permission
resources using AWS-defined regexps.

Fixes bug in lumidl Check wrapper which was dropping
reported check failures (an regen all rpc files).

Add calls to Get into the AWS provider test framework.
2017-06-07 15:13:56 -07:00
Luke Hoban
977e56e03e Add aws.lambda.Permission resource (#224)
Adds a Permission resource to enable lambda Functions
to be invoked by event sources.

Per #223, we are continuing to discuss whether these
should really be a seperate resource or just an inline
component of a Function resource.  However, until we
support cycle-breaking, we'll need them to be a separate
resource type.

Progress on #222.
2017-06-06 18:42:00 -07:00
joeduffy
fd719d64cd Fix pulumi/lumi#198
This change fixes the serialization of resource properties during
deployment checkpoints.  We erroneously serialized empty arrays and
empty maps as though they were nil; instead, we want to keep them
serialized as non-nil empty collections, since the presence of a
value might be semantically meaningful.  (We still skip nils.)

Also added some test cases.
2017-06-06 16:42:14 -07:00
joeduffy
f33b20e6cf Dereference some names; add some missing licenses 2017-06-06 15:19:32 -07:00
joeduffy
c7dc3036d7 Finish scrubbing TODOs
This is a final pass over our TODOs, and closes pulumi/lumi#212.
2017-06-06 06:05:35 -07:00
joeduffy
ec2b964daa Do an initial pass over TODOs
This scrubs about 80% of our TODOs, as part of pulumi/lumi#212.
The remaining 20% will come shortly.
2017-06-05 18:11:51 -07:00
joeduffy
5698735866 Add the IDEA category for TODOs 2017-06-05 17:51:54 -07:00
joeduffy
db99092334 Implement mapper.Encode "for real"
This change implements `mapper.Encode` "for real" (that is, in a way
that isn't a complete embarrassment).  It uses the obvious reflection
trickery to encode a tagged struct and its values as a JSON-like
in-memory map and collection of keyed values.

During this, I took the opportunity to also clean up a few other things
that had been bugging me.  Namely, the presence of `mapper.Object` was
always error prone, since it isn't a true "typedef" in the sence that
it carries extra RTTI.  Instead of doing that, let's just use the real
`map[string]interface{}` "JSON-map-like" object type.  Even better, we
no longer require resource providers to deal with the mapper
infrastructure.  Instead, the `Check` function can simply return an
array of errors.  It's still best practice to return field-specific errors
to facilitate better diagnostics, but it's no longer required; and I've
added `resource.NewFieldError` to eliminate the need to import mapper.

As of this change, we can also consistently emit RPC structs with `lumi`
tags, rather than `lumi` tags on the way in and `json` on the way out.

This completes pulumi/lumi#183.
2017-06-05 17:49:00 -07:00
Luke Hoban
658d4b3c8a Add array-valued output properties to RestAPI
These properties no longer trigger assertons after
work on #198.
2017-06-05 11:23:42 -07:00
Luke Hoban
09f2968d18 Decouple Deployment from Stage in aws.apigateway
The raw AWS API and CloudFormation projection allow
a stage to be created as part of creating a deployment.
This leads to difficulties tracking the ownership of this
extra stage, since it is neither created and owned
seperately, nor is it discoverable after the fact from the
deployment.

We can keep the API simpler by not projecting this feature
of the AWS API into the Lumi resource.  The stage will have
to be created seperately in Lumi, and it's lifecycle is well
understood as a separate Lumi resource.

Fixes #202.
2017-06-04 21:34:51 -07:00
joeduffy
c2f2fbd2ff Regenerate some code 2017-06-04 19:37:27 -07:00
joeduffy
87004a124e Store both input and output properties distinctly
This changes the resource model to persist input and output properties
distinctly, so that when we diff changes, we only do so on the programmer-
specified input properties.  This eliminates problems when the outputs
differ slightly; e.g., when the provider normalizes inputs, adds its own
values, or fails to produce new values that match the inputs.

This change simultaneously makes progress on pulumi/lumi#90, by beginning
tracking the resource objects implicated in a computed property's value.

I believe this fixes both #189 and #198.
2017-06-04 19:24:48 -07:00
Luke Hoban
5f1e8b7653 Merge pull request #199 from pulumi/apigateway
Adds support for AWS API Gateway RestApi, Deployment and Stage resources. Together, these enable publishing an HTTP endpoint from a Lumi script.

Also creates the aws.serverless sub-package for higher level serverless abstractions similar to the AWS Serverless Application Model (SAM). Currently includes higher-level Function and API abstractions.
2017-06-04 15:06:06 -07:00
Luke Hoban
6710b919ea Cleanup APIGateway ARN parsing 2017-06-04 14:58:50 -07:00
Luke Hoban
318bcf9542 Merge branch 'master' into apigateway 2017-06-04 13:59:39 -07:00
joeduffy
812cd4ea00 Revert "Eliminate the @lumi.out decorator"
This reverts commit 048c35d428.

I have a pending change that fixes this along with a number of
other issues (including pulumi/lumi#198 and pulumi/lumi#187),
however, it's going to take a little longer and I want to unblock.

This fixes pulumi/lumi#200.
2017-06-04 13:05:21 -07:00
Luke Hoban
cfc25552e1 Move APIGateway resources to use ARNs as IDs
Uses the ARN format roughly as described at:
http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-control-access-using-iam-policies-to-create-and-manage-api.html

Deployments do not have a defined ARN format, so we
follow a similar pattern as for Stages.
2017-06-04 12:10:29 -07:00
Luke Hoban
fd68aab8c3 Merge branch 'master' into apigateway 2017-06-04 10:53:36 -07:00
Luke Hoban
26a2f95c48 Support output properties on aws.apigateway APIs 2017-06-04 10:19:04 -07:00
joeduffy
048c35d428 Eliminate the @lumi.out decorator
The @lumi.out decorator ended up not being required after all.
The engine essentially treats all resource properties as potentially
output properties now, given the way we read back the state from
the provider after essential operations.

This is a good thing, because the evaluator currently doesn't
perform dynamic decoration in a way that would be amenable to
a faithful ECMAScript implementation (see pulumi/lumi#128).
2017-06-04 08:12:58 -07:00
Luke Hoban
92a9925201 Merge branch 'master' into apigateway 2017-06-03 14:58:23 -07:00
Luke Hoban
bdcda5717a Merge pull request #194 from pulumi/providertest
Test coverage for more AWS providers
2017-06-03 14:55:01 -07:00
Luke Hoban
9f4d48b8f8 Address PR feedback on #194 2017-06-03 14:47:27 -07:00
Luke Hoban
ee987c9961 Add provider tests for aws.lambda.Function
Expands the resource provider test framework
to include support for provider tests which require
creating, updating and deleteing multiple resources.
2017-06-03 14:35:40 -07:00
Luke Hoban
5d2dffdcc9 Support Tags on aws.ec2.Instance
Also adds test coverage for aws.ec2.Instance resources.
2017-06-03 14:35:40 -07:00
Joe Duffy
72919f7526 Merge pull request #196 from pulumi/no_nonstandard_tools
Eliminate use of nonstandard tools
2017-06-03 11:22:33 -07:00
joeduffy
cfaa7c9310 Eliminate use of nonstandard tools
This change eliminates the use of nonstandard tools in our build:

* `go test` automatically uses `GOMAXPROCS` for its parallelism
  setting.  In modern Go, this is now set to the number of processors.
  So, there is no need to set it explicitly using `nproc`.

* We can avoid `realpath` in the `lumijs` executable by making it
  a Node.js file and using a relative require import of main.

* We can avoid `realpath` in our Makefiles by just using `pwd`.
2017-06-03 11:08:09 -07:00
joeduffy
f552832a7a Alter diag.Message to discourage format mistakes
This change alters diag.Message to not format strings and, instead,
encourages developers to use the Infof, Errorf, and Warningf varargs
functions.  It also tests that arguments are never interepreted as
format strings.
2017-06-02 18:37:28 -07:00
Joe Duffy
52b2e62157 Merge pull request #195 from pulumi/install_all
Also build the Lumi stdlib during `make all`
2017-06-02 15:36:52 -07:00
joeduffy
39db4dca63 Also build the Lumi stdlib during make all 2017-06-02 15:26:39 -07:00
Joe Duffy
8bbe89bd75 Makeify more; add a "full build" target (#193)
* Makeify more; add a "full build" target

This change uses make for more of our tree.  Namely, the AWS provider
and LumiJS compilers each now use make to build and/or install them.
Not only does this bring about some consistency to how we build and
test things, but also made it easy to add a full build target:

    $ make all

This target will build, test, and install the core Go tools, the LumiJS
compiler, and the AWS provider, in that order.

Each can be made in isolation, however, which ensures that the inner
loop for those is fast and so that, when it comes to finishing
pulumi/lumi#147, we can easily split them out and make from the top.
2017-06-02 14:26:34 -07:00
joeduffy
43bcbed23d Tidy up project loading for pack commands
There are a few things that annoyed me about the way our CLI works with
directories when loading packages.  For example, `lumi pack info some/pack/dir/`
never worked correctly.  This is unfortunate when scripting commands.
This change fixes the workspace detection logic to handle these cases.
2017-06-02 12:43:04 -07:00
Joe Duffy
bb9834f4a1 Merge pull request #192 from pulumi/inner_loop
Only run quick tests for `make`; add a `make full` target
2017-06-02 10:51:45 -07:00
joeduffy
769b0a4af0 Only run quick tests for make; add a make full target
The AWS tests take a while to run and it's easy to forget.  Further,
they are going to start taking quite a bit longer very soon.

So, this change introduces a `make full` target, which is what our
CI tests run.  But the ordinary `make` skips the long tests for faster
inner loop verification.  Over time, I'm sure we'll get far more
sophisticated with the split between inner vs. outer loop testing,
especially for performance, stress, and so on.
2017-06-02 10:30:57 -07:00
joeduffy
717609a31b Fail the build on lint warnings
Golint by default doesn't return a non-zero exit code when lint errors
are present.  To address this, we need to pass -set_exit_status.

But it gets more complicated: because we are suppressing some lint
warnings, we cannot use -set_exit_status in the lint_quiet target.
For that, we use test -z to fail if any output passes the filter.

Eventually when we do pulumi/lumi#191, we can remove this hack.
2017-06-02 09:05:28 -07:00
joeduffy
ff37f0b8f9 Fix two lint issues that crept in 2017-06-02 09:05:10 -07:00
joeduffy
e1673dfdb0 Add some basic plan tests 2017-06-02 08:53:40 -07:00
joeduffy
934802d54a Add some basic ARN tests 2017-06-02 08:13:38 -07:00
joeduffy
545906f38f Add a check for egress rules on EC2-Classic groups
If you try to specify egress rules on an EC2-Classic security group,
you get obscure error messages like "The parameter CidrIp is not recognized."
It turns out, the underlying AuthorizeSecurityGroupEgress function
isn't supported on EC2-Classic security groups (those w/out VPCs); see:
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_AuthorizeSecurityGroupEgress.html

This change adds a check up front for this condition so developers get
a nicer error message.
2017-06-02 08:13:02 -07:00
Luke Hoban
6194945c27 Get Elastic Beanstalk sample working again
Changes the Source property on AWS S3 Objects
to be an `in` property.

Adds an AllOptionSettings output property to
store the settings returned from AWS.  This
unblocks keeping Beanstalk workign while
we evaluate options for #189.

Silently skip unsupported AWS ElasticBeanstalk
Environment properties in Get operation.

Fixes ARN resource name pair extraction to skip the
"/" between parts of the name pair.
2017-06-01 23:09:51 -07:00
joeduffy
ce35fc78cf Add some property diff tests 2017-06-01 15:36:22 -07:00
Luke Hoban
5358080ca6 Output property improvements for AWS Function and Role
The AssumeRolePolicyDocument property returned by the AWS IAM GetRole API returns
a URL-encoded JSON string, so we need to decode this before JSON unmarshalling.

The Code property returned by AWS Lambda GetFunction provides a pre-signed S3 URL,
which changes on each call, and is of a different format to what is provided by the user.
For now, we'll not store this back into the Function object.

Add additional output properties to AWS Lambda Function that are stable values returned
from GetFunction.

Also corrects a gap where some property delete operations were not being correctly reported.
2017-06-01 15:04:37 -07:00