Commit graph

1221 commits

Author SHA1 Message Date
joeduffy
164d4db30a Implement CIDLC RPC code generation
This change implements the boilerplate RPC stub generation for CIDLC
resource providers.  This includes a lot of the marshaling goo required
to bridge between the gRPC plugin interfaces and the strongly typed
resource types and supporting marshalable structures.

This completes the major tasks of implementing CIDLC (pulumi/coconut#133),
and I have most of the AWS package running locally on it.  There are
undoubtedly bugs left to shake out, but no planned work items remain.
2017-04-27 07:33:00 -07:00
joeduffy
507a2609a7 Add an initial implementation of CIDLC
This is an initial implementation of the Coconut IDL Compiler (CIDLC).
This is described further in
https://github.com/pulumi/coconut/blob/master/docs/design/idl.md,
and the work is tracked by coconut/pulumi#133.

I've been kicking the tires with this locally enough to checkpoint the
current version.  There are quite a few loose ends not yet implemented,
most of them minor, with the exception of the RPC stub generation which
I need to flesh out more before committing.
2017-04-25 15:05:51 -07:00
joeduffy
8c58950639 Tolerate nils in output property marshaling 2017-04-25 14:04:22 -07:00
joeduffy
d3899981e0 Make AWS instance DNS/IP outputs optional 2017-04-25 14:03:42 -07:00
joeduffy
90b172aacf Fix a few typos 2017-04-24 10:04:18 -07:00
joeduffy
3f866e353f Merge branch 'master' of github.com:pulumi/coconut 2017-04-24 09:52:38 -07:00
joeduffy
7d3192637e Jot down design for resource provider IDL
As part of pulumi/coconut#133, we will be doing an IDL dialect of Go
plus a compiler, to support writing resource providers more easily.

This commit includes a design note for how it works.
2017-04-24 09:50:48 -07:00
Eric Rudder
f8405325db Fixed typo 2017-04-24 09:16:56 -07:00
joeduffy
4671e5f435 Take an initial cut at AWS API Gateway support in Mantle
There are still several loose ends, but this sketches out the
overall shape of the API Gateway usage in Mantle.
2017-04-21 16:47:31 -07:00
joeduffy
0827ca5b58 Implement creation outputs for AWS resources
This change implements the appropriate creation outputs for AWS
resource providers, both on the IDL side and in the providers themselves.
2017-04-21 16:14:59 -07:00
joeduffy
1edced2d4b Add the ability to convert structs to PropertyMaps 2017-04-21 15:27:32 -07:00
joeduffy
d6abea728c Add outputs to the Create provider's return
In order to support output properties (pulumi/coconut#90), we need to
modify the Create gRPC interface for resource providers slightly.  In
addition to returning the ID, we need to also return any properties
computed by the AWS provider itself.  For instance, this includes ARNs
and IDs of various kinds.  This change simply propagates the resources
but we don't actually support reading the outputs just yet.
2017-04-21 14:15:06 -07:00
joeduffy
a2b37d7dda Implement an aws.ARN type
This "strongly types" the aws.ARN type.  In reality, it is simply an
alias for a string at the moment (as this is what it boils down to on
the wire), but this at least gets us some amount of compile-time safety.
2017-04-21 14:03:06 -07:00
joeduffy
9b9fac4254 Make iam.Role's managed policy ARNs a string list 2017-04-21 13:27:34 -07:00
joeduffy
cb10ec7014 Mark lambda.Permission's source args as optional 2017-04-21 13:24:56 -07:00
joeduffy
6a6b82e2ab Add the AWS lambda.Permission resource IDL 2017-04-21 13:23:13 -07:00
joeduffy
689dd06949 Add AWS API Gateway resource IDL definitions
This change includes resource IDL definitions for all of AWS API Gateway's
corresponding CloudFormation resource types.  There are no actual provider
implementations yet.
2017-04-21 10:15:19 -07:00
joeduffy
25fb74803a Add a sample "Hello, World" for Mantle
This is a sample serverless function, written in Node.js, and exposed
over an HTTP API gateway endpoint.

It even works!  (For Kubernetes Fission; the AWS support is dependent
upon projecting the various API gateway resource providers...)
2017-04-20 18:46:21 -07:00
joeduffy
fdd441fd25 Implement the Kubernetes Fission HTTPTrigger provider 2017-04-20 18:40:05 -07:00
joeduffy
aa44b46608 Lower instanceof in CocoJS; implement IsInst in CocoIL 2017-04-20 17:38:15 -07:00
joeduffy
3f25aca7dd Add a http.API abstraction to Mantle
This introduces an http.API abstraction to Mantle and implements it
for Kubernetes Fission.  (The AWS implementation will come shortly,
but requires a fair bit of API Gateway elbow grease first.)

This lets you invoke a function in response to an API event:

    let func = new mantle.func.Function(
        (e, ctx, cb => { ... },
    );
    let api = new mantle.http.API("/hello", "GET", func);
2017-04-20 16:05:19 -07:00
joeduffy
0b6e262b46 Rename resource provider methods
This change renames two provider methods:

    * Read becomes Get.

    * UpdateImpact becomes PreviewUpdate.

These just read a whole lot nicer than the old names.
2017-04-20 14:09:00 -07:00
joeduffy
94e072c653 Add a TryLoadDynamicExpression IL opcode
This change introduces TryLoadDynamicExpression.  This is similar to
the existing LoadDynamicExpression opcode, except that it will return
null in response to a missing member (versus the default of raising
an exception).  This is to enable languages like JavaScript to encode
operations properly (which always yields undefined/nulls), while still
catering to languages like Python (which throw exceptions).
2017-04-19 16:49:59 -07:00
joeduffy
8eecc2c05b Flatten Fission metadata property
This change flattens the Fission metadata property on the Coconut
side so that it is nicer to consume.  Due to struct embedding in Go,
we need to perform translation at the serialization boundary anyway,
so the less-usable approach we had before didn't really buy us anything.
2017-04-19 16:05:33 -07:00
joeduffy
da73d81dd0 Implement the Fission function provider 2017-04-19 15:52:17 -07:00
joeduffy
47a9a6ee08 Implement Kubernetes functions
This change detects when the target scheduler is Kubernetes, and prefers
to schedule atop Fission functions.
2017-04-19 15:36:48 -07:00
joeduffy
cc6dd63556 Use inter-resource references for Fission resources
The Fission internal data structures reference one another by
name/URI.  Coconut needs the graph with dependencies, however, so
we prefer to model things using real object references in these
situations.  (This also makes it much nicer to program.)  This
changes to fit the Coconut model better; the provider will then
turn around and present names/URIs to the various APIs.
2017-04-19 15:33:33 -07:00
joeduffy
0977477f95 Improve an assertion message 2017-04-19 15:23:05 -07:00
joeduffy
cbd3692d3d Initialize the logging libraries 2017-04-19 15:05:18 -07:00
joeduffy
53e7bfbb86 Rearrange the way stderr/stdout is handled for plugins
The order of operations for stderr/stdout monitoring with plugins
managed to hide some important errors.  For example, if something
was written to stderr *before* the port was parsed from stdout, a
very possible scenario if the plugin fails before it has even
properly sarted, then we would silently drop the stderr on the floor
leaving behind no indication of what went wrong.  The new ordering
ensures that stderr is never ignored with some minor improvements
in the case that part of the port is parsed from stdout but it
ultimately ends in an error (e.g., if an EOF occurs prematurely).
2017-04-19 15:01:04 -07:00
joeduffy
f429bc6a0c Use github.com/pkg/errors for errors
This change moves us over to the github.com/pkg/errors package to
encourage the addition of more context associated with failures.
2017-04-19 14:46:50 -07:00
joeduffy
958d67d444 Move NewCheckResponse into coconut/pkg/resource package 2017-04-19 14:25:49 -07:00
joeduffy
366e236853 Implement the Fission provider scaffolding; and Environments 2017-04-19 14:23:01 -07:00
joeduffy
936d3f45e6 Fix another package name reference 2017-04-19 14:20:12 -07:00
joeduffy
1abda1b49e Glide the AWS provider package 2017-04-19 12:04:24 -07:00
joeduffy
ad87006611 Add an @asset decorator, and a Code asset type
This change introduces an @asset decorator in addition to a Code
asset type.  The idea here is that compilers will recognize the
presence of @asset and react by spewing the code as a textual
asset, rather than an AST and/or actually getting compiled into IL.
2017-04-19 11:43:33 -07:00
joeduffy
98961b706a Permit package name hyphens in one more place
...missed one.
2017-04-19 11:34:36 -07:00
joeduffy
48677d8809 Fix some paths
😳
2017-04-19 11:26:41 -07:00
joeduffy
cc37a8078d Implement a basic Fission (Kubernetes FaaS) resource package
This change introduces some resource IDL types for the Fission
project (https://github.com/fission/fission), which is a
functions-as-a-service (FaaS) substrate on top of Kubernetes.

It has a single configuration variable,
`kube-fission:config:controller`, which controls the address of
the Fission controller in the target Kubernetes cluster.

Someday, ideally we would rebuild this atop a more general
Kubernetes resource package, and add some sort of discovery of
this controller, to minimize the amount of manual configuration
required and to make it more resilient to self-healing operations.

This is IDL-only.  The provider is coming soon.
2017-04-19 11:21:35 -07:00
joeduffy
034fc3bf67 Permit dashes in package names
Right now, we reject dashes in package names.  I've hit this a few times and it annoys
me each time.  (It would seem to makes sense to permit hyphens in package names, given
that [almost?] every other package manager on Earth does...)

No more!  Hyphens welcome!
2017-04-19 10:53:14 -07:00
joeduffy
847d74c9f6 Implement rudimentary decorator support
This change introduces decorator support for CocoJS and the corresponding
IL/AST changes to store them on definition nodes.  Nothing consumes these
at the moment, however, I am looking at leveraging this to indicate that
certain program fragments are "code" and should be serialized specially
(in support of Functions-as-lambdas).
2017-04-18 16:53:26 -07:00
joeduffy
4989e70425 Upgrade to TypeScript 2.2.2 2017-04-18 15:57:13 -07:00
joeduffy
c51fbd57c3 Introduce a new cross-cloud package, Mantle
This introduces a new experimental cross-cloud package, Mantle.  This is part
of getting pulumi/coconut#134 up and running.

For now, there isn't much here, except for configuration of the target cloud and
scheduler, plus the ability to create a cross-cloud function.  (OK, it's not truly
cross-cloud yet, since we have only implemented AWS Lambdas.)

Mantle is, of course, a placeholder name.  Since this is a higher level
abstraction than the unopinionated core primitives, the name mantle corresponds
to the Earth's upper layers (and I thought "Crust" would be a rather odd name).
2017-04-18 15:10:13 -07:00
joeduffy
da75f62865 Retry lambda creation until IAM role is available
Per Amazon's own documentation,
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html#launch-instance-with-role,
IAM roles may take "several seconds" to propagate.  In the meantime, we
are apt to get the dreaded "role defined for this function cannot be assumed"
error message.  In response, we'll do what the AWS documentation suggests:
wait a bit and retry.
2017-04-18 13:56:19 -07:00
joeduffy
74772f89e4 Use the role ARN, not its name, for lambda creation 2017-04-18 13:18:04 -07:00
joeduffy
973fccf09d Implement AWS Lambda resource provider
This change introduces a basic AWS Lambda resource provider.  It supports
C--D, but not -RU-, yet.
2017-04-18 11:02:04 -07:00
joeduffy
7878a44ef5 Export inlinePolicy fields 2017-04-17 18:06:12 -07:00
joeduffy
e8d7ef620f Add a couple missing trace errs 2017-04-17 18:05:12 -07:00
joeduffy
351981a409 Implement AWS IAM Role provider 2017-04-17 17:54:20 -07:00
joeduffy
4c5e13f241 Rearrange asset modules to read nicer 2017-04-17 17:24:08 -07:00