Commit graph

1218 commits

Author SHA1 Message Date
joeduffy
fd719d64cd Fix pulumi/lumi#198
This change fixes the serialization of resource properties during
deployment checkpoints.  We erroneously serialized empty arrays and
empty maps as though they were nil; instead, we want to keep them
serialized as non-nil empty collections, since the presence of a
value might be semantically meaningful.  (We still skip nils.)

Also added some test cases.
2017-06-06 16:42:14 -07:00
joeduffy
f33b20e6cf Dereference some names; add some missing licenses 2017-06-06 15:19:32 -07:00
joeduffy
c7dc3036d7 Finish scrubbing TODOs
This is a final pass over our TODOs, and closes pulumi/lumi#212.
2017-06-06 06:05:35 -07:00
joeduffy
ec2b964daa Do an initial pass over TODOs
This scrubs about 80% of our TODOs, as part of pulumi/lumi#212.
The remaining 20% will come shortly.
2017-06-05 18:11:51 -07:00
joeduffy
5698735866 Add the IDEA category for TODOs 2017-06-05 17:51:54 -07:00
joeduffy
db99092334 Implement mapper.Encode "for real"
This change implements `mapper.Encode` "for real" (that is, in a way
that isn't a complete embarrassment).  It uses the obvious reflection
trickery to encode a tagged struct and its values as a JSON-like
in-memory map and collection of keyed values.

During this, I took the opportunity to also clean up a few other things
that had been bugging me.  Namely, the presence of `mapper.Object` was
always error prone, since it isn't a true "typedef" in the sence that
it carries extra RTTI.  Instead of doing that, let's just use the real
`map[string]interface{}` "JSON-map-like" object type.  Even better, we
no longer require resource providers to deal with the mapper
infrastructure.  Instead, the `Check` function can simply return an
array of errors.  It's still best practice to return field-specific errors
to facilitate better diagnostics, but it's no longer required; and I've
added `resource.NewFieldError` to eliminate the need to import mapper.

As of this change, we can also consistently emit RPC structs with `lumi`
tags, rather than `lumi` tags on the way in and `json` on the way out.

This completes pulumi/lumi#183.
2017-06-05 17:49:00 -07:00
Luke Hoban
658d4b3c8a Add array-valued output properties to RestAPI
These properties no longer trigger assertons after
work on #198.
2017-06-05 11:23:42 -07:00
Luke Hoban
09f2968d18 Decouple Deployment from Stage in aws.apigateway
The raw AWS API and CloudFormation projection allow
a stage to be created as part of creating a deployment.
This leads to difficulties tracking the ownership of this
extra stage, since it is neither created and owned
seperately, nor is it discoverable after the fact from the
deployment.

We can keep the API simpler by not projecting this feature
of the AWS API into the Lumi resource.  The stage will have
to be created seperately in Lumi, and it's lifecycle is well
understood as a separate Lumi resource.

Fixes #202.
2017-06-04 21:34:51 -07:00
joeduffy
c2f2fbd2ff Regenerate some code 2017-06-04 19:37:27 -07:00
joeduffy
87004a124e Store both input and output properties distinctly
This changes the resource model to persist input and output properties
distinctly, so that when we diff changes, we only do so on the programmer-
specified input properties.  This eliminates problems when the outputs
differ slightly; e.g., when the provider normalizes inputs, adds its own
values, or fails to produce new values that match the inputs.

This change simultaneously makes progress on pulumi/lumi#90, by beginning
tracking the resource objects implicated in a computed property's value.

I believe this fixes both #189 and #198.
2017-06-04 19:24:48 -07:00
Luke Hoban
5f1e8b7653 Merge pull request #199 from pulumi/apigateway
Adds support for AWS API Gateway RestApi, Deployment and Stage resources. Together, these enable publishing an HTTP endpoint from a Lumi script.

Also creates the aws.serverless sub-package for higher level serverless abstractions similar to the AWS Serverless Application Model (SAM). Currently includes higher-level Function and API abstractions.
2017-06-04 15:06:06 -07:00
Luke Hoban
6710b919ea Cleanup APIGateway ARN parsing 2017-06-04 14:58:50 -07:00
Luke Hoban
318bcf9542 Merge branch 'master' into apigateway 2017-06-04 13:59:39 -07:00
joeduffy
812cd4ea00 Revert "Eliminate the @lumi.out decorator"
This reverts commit 048c35d428.

I have a pending change that fixes this along with a number of
other issues (including pulumi/lumi#198 and pulumi/lumi#187),
however, it's going to take a little longer and I want to unblock.

This fixes pulumi/lumi#200.
2017-06-04 13:05:21 -07:00
Luke Hoban
cfc25552e1 Move APIGateway resources to use ARNs as IDs
Uses the ARN format roughly as described at:
http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-control-access-using-iam-policies-to-create-and-manage-api.html

Deployments do not have a defined ARN format, so we
follow a similar pattern as for Stages.
2017-06-04 12:10:29 -07:00
Luke Hoban
fd68aab8c3 Merge branch 'master' into apigateway 2017-06-04 10:53:36 -07:00
Luke Hoban
26a2f95c48 Support output properties on aws.apigateway APIs 2017-06-04 10:19:04 -07:00
joeduffy
048c35d428 Eliminate the @lumi.out decorator
The @lumi.out decorator ended up not being required after all.
The engine essentially treats all resource properties as potentially
output properties now, given the way we read back the state from
the provider after essential operations.

This is a good thing, because the evaluator currently doesn't
perform dynamic decoration in a way that would be amenable to
a faithful ECMAScript implementation (see pulumi/lumi#128).
2017-06-04 08:12:58 -07:00
Luke Hoban
92a9925201 Merge branch 'master' into apigateway 2017-06-03 14:58:23 -07:00
Luke Hoban
bdcda5717a Merge pull request #194 from pulumi/providertest
Test coverage for more AWS providers
2017-06-03 14:55:01 -07:00
Luke Hoban
9f4d48b8f8 Address PR feedback on #194 2017-06-03 14:47:27 -07:00
Luke Hoban
ee987c9961 Add provider tests for aws.lambda.Function
Expands the resource provider test framework
to include support for provider tests which require
creating, updating and deleteing multiple resources.
2017-06-03 14:35:40 -07:00
Luke Hoban
5d2dffdcc9 Support Tags on aws.ec2.Instance
Also adds test coverage for aws.ec2.Instance resources.
2017-06-03 14:35:40 -07:00
Joe Duffy
72919f7526 Merge pull request #196 from pulumi/no_nonstandard_tools
Eliminate use of nonstandard tools
2017-06-03 11:22:33 -07:00
joeduffy
cfaa7c9310 Eliminate use of nonstandard tools
This change eliminates the use of nonstandard tools in our build:

* `go test` automatically uses `GOMAXPROCS` for its parallelism
  setting.  In modern Go, this is now set to the number of processors.
  So, there is no need to set it explicitly using `nproc`.

* We can avoid `realpath` in the `lumijs` executable by making it
  a Node.js file and using a relative require import of main.

* We can avoid `realpath` in our Makefiles by just using `pwd`.
2017-06-03 11:08:09 -07:00
joeduffy
f552832a7a Alter diag.Message to discourage format mistakes
This change alters diag.Message to not format strings and, instead,
encourages developers to use the Infof, Errorf, and Warningf varargs
functions.  It also tests that arguments are never interepreted as
format strings.
2017-06-02 18:37:28 -07:00
Joe Duffy
52b2e62157 Merge pull request #195 from pulumi/install_all
Also build the Lumi stdlib during `make all`
2017-06-02 15:36:52 -07:00
joeduffy
39db4dca63 Also build the Lumi stdlib during make all 2017-06-02 15:26:39 -07:00
Joe Duffy
8bbe89bd75 Makeify more; add a "full build" target (#193)
* Makeify more; add a "full build" target

This change uses make for more of our tree.  Namely, the AWS provider
and LumiJS compilers each now use make to build and/or install them.
Not only does this bring about some consistency to how we build and
test things, but also made it easy to add a full build target:

    $ make all

This target will build, test, and install the core Go tools, the LumiJS
compiler, and the AWS provider, in that order.

Each can be made in isolation, however, which ensures that the inner
loop for those is fast and so that, when it comes to finishing
pulumi/lumi#147, we can easily split them out and make from the top.
2017-06-02 14:26:34 -07:00
joeduffy
43bcbed23d Tidy up project loading for pack commands
There are a few things that annoyed me about the way our CLI works with
directories when loading packages.  For example, `lumi pack info some/pack/dir/`
never worked correctly.  This is unfortunate when scripting commands.
This change fixes the workspace detection logic to handle these cases.
2017-06-02 12:43:04 -07:00
Joe Duffy
bb9834f4a1 Merge pull request #192 from pulumi/inner_loop
Only run quick tests for `make`; add a `make full` target
2017-06-02 10:51:45 -07:00
joeduffy
769b0a4af0 Only run quick tests for make; add a make full target
The AWS tests take a while to run and it's easy to forget.  Further,
they are going to start taking quite a bit longer very soon.

So, this change introduces a `make full` target, which is what our
CI tests run.  But the ordinary `make` skips the long tests for faster
inner loop verification.  Over time, I'm sure we'll get far more
sophisticated with the split between inner vs. outer loop testing,
especially for performance, stress, and so on.
2017-06-02 10:30:57 -07:00
joeduffy
717609a31b Fail the build on lint warnings
Golint by default doesn't return a non-zero exit code when lint errors
are present.  To address this, we need to pass -set_exit_status.

But it gets more complicated: because we are suppressing some lint
warnings, we cannot use -set_exit_status in the lint_quiet target.
For that, we use test -z to fail if any output passes the filter.

Eventually when we do pulumi/lumi#191, we can remove this hack.
2017-06-02 09:05:28 -07:00
joeduffy
ff37f0b8f9 Fix two lint issues that crept in 2017-06-02 09:05:10 -07:00
joeduffy
e1673dfdb0 Add some basic plan tests 2017-06-02 08:53:40 -07:00
joeduffy
934802d54a Add some basic ARN tests 2017-06-02 08:13:38 -07:00
joeduffy
545906f38f Add a check for egress rules on EC2-Classic groups
If you try to specify egress rules on an EC2-Classic security group,
you get obscure error messages like "The parameter CidrIp is not recognized."
It turns out, the underlying AuthorizeSecurityGroupEgress function
isn't supported on EC2-Classic security groups (those w/out VPCs); see:
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_AuthorizeSecurityGroupEgress.html

This change adds a check up front for this condition so developers get
a nicer error message.
2017-06-02 08:13:02 -07:00
Luke Hoban
6194945c27 Get Elastic Beanstalk sample working again
Changes the Source property on AWS S3 Objects
to be an `in` property.

Adds an AllOptionSettings output property to
store the settings returned from AWS.  This
unblocks keeping Beanstalk workign while
we evaluate options for #189.

Silently skip unsupported AWS ElasticBeanstalk
Environment properties in Get operation.

Fixes ARN resource name pair extraction to skip the
"/" between parts of the name pair.
2017-06-01 23:09:51 -07:00
joeduffy
ce35fc78cf Add some property diff tests 2017-06-01 15:36:22 -07:00
Luke Hoban
5358080ca6 Output property improvements for AWS Function and Role
The AssumeRolePolicyDocument property returned by the AWS IAM GetRole API returns
a URL-encoded JSON string, so we need to decode this before JSON unmarshalling.

The Code property returned by AWS Lambda GetFunction provides a pre-signed S3 URL,
which changes on each call, and is of a different format to what is provided by the user.
For now, we'll not store this back into the Function object.

Add additional output properties to AWS Lambda Function that are stable values returned
from GetFunction.

Also corrects a gap where some property delete operations were not being correctly reported.
2017-06-01 15:04:37 -07:00
joeduffy
e2cb211d93 Enable parallel tests
This change enables parallelism for our tests.

It also introdues a `test_core` Makefile target to just run the
core engine tests, and not the providers, since they take a long time.
This is intended only as part of the inner developer loop.
2017-06-01 14:01:26 -07:00
joeduffy
fff4a9e8af Respond to CR feedback on PR #186
This change responds to some great feedback from @lukehoban on
https://github.com/pulumi/lumi/pull/186.  Namely:

* Relax many assertions in the providers.  A failure here is pretty
  bad because it takes down the entire provider due to fail-fast.
  That said, I'd rather fail in response to a bug than let it go
  silently.  Nevertheless, a few of them were candidates for dynamic
  failures.

* Use `aws.StringValue` and friends in more places.

* Remove a superfluous waitForTableState in DynamoDB.

* Fix some erroneous references to `Read` in some `Get` comments.
2017-06-01 13:25:49 -07:00
joeduffy
b07056ab10 Create a plan plugin host
This is a minor refactoring to introduce a ProviderHost interface
that is associated with the context and can be swapped in and out for
custom plugin behavior.  This is required to write tests that mock
certain aspects, like loading packages from the filesystem.

In theory, this change incurs zero behavioral changes.
2017-06-01 11:41:24 -07:00
Luke Hoban
c117b43ae4 Update serverless API programming model
Updates  the higher level AWS APIGateway programming model
in aws.serverless.API  to use an Express-like imperative API.
2017-06-01 10:54:26 -07:00
joeduffy
0e5ba9655f Pretty print outputs during planning 2017-06-01 10:52:25 -07:00
joeduffy
8de9411697 Mark instance security groups as replaces
Altering an instance's security group list used to imply a replace, but
it must have gotten broken somewhere along the line during the IDL updates.
(We desperately need those test cases...)  For now, we will mark the groups
as `replaces, and when we get around to pulumi/lumi#187, we'll fix this.
2017-06-01 10:39:21 -07:00
Joe Duffy
4ec2745fc1 Merge pull request #186 from pulumi/lumi-90-outprops
Implement 1/3rds of output properties
2017-06-01 10:28:39 -07:00
joeduffy
7b5f9df917 Make updates work in the face of output properties
This change fixes up a few things so that updates correctly deal
with output properties.  This involves a few things:

    1) All outputs stored on the pre snapshot need to get propagated
       to the post snapshot during planning at various points.  This
       ensures that the diffing logic doesn't need to be special cased
       everywhere, including both the Lumi and the provider sides.

    2) Names are changed to "input" properties (using a new `lumi` tag
       option, `in`).  These are properties that providers are expected
       to know nothing about, which we must treat with care during diffs.

    3) We read back properties, via Get, after doing an Update just like
       we do after performing a Create.  This ensures that if an update
       has a cascading impact on other properties, it will be detected.

    4) Inspecting a change, prior to updating, must be done using the
       computed property set instead of the real one.  This is to avoid
       mutating the resource objects ahead of actually applying a plan,
       which would be wrong and misleading.
2017-06-01 10:09:52 -07:00
joeduffy
b5df277815 Fix a few merges when this branch hit master 2017-06-01 08:51:33 -07:00
joeduffy
23493be8af Classify output properties as adds too 2017-06-01 08:39:48 -07:00