Commit graph

177 commits

Author SHA1 Message Date
joeduffy 7c848bfff4 Add config to the basic/minimal test 2017-09-04 11:35:21 -07:00
joeduffy 590e9e539b Rename Lumi.yaml to Pulumi.yaml
And also eliminate lots of accumulated cruft around "packfiles", etc.
in the workspace code.
2017-09-04 11:35:21 -07:00
joeduffy 1df1b6d572 Get integration tests passing
This makes a few tweaks to get the integration tests passing:

* Add `runtime: nodejs` to the minimal example's `Lumi.yaml` file.

* Remove usage of `@lumi/lumirt { printf }` and just use `console.log`.

* Remove calls to `lumijs` in the integration test framework and
  the minimal example's package.json.  Instead, we just run
  `yarn run build`, which itself internally just invokes `tsc`.

* Add package validation logic and eliminate the pkg/compiler/metadata
  library, in favor of the simpler code in pkg/engine.

* Simplify the Node.js langhost plugin CLI, and simply take an
  argument rather than requiring required and optional --flags.

* Use a default path of "." if the program path isn't provided.  This
  is a legal scenario if you've passed a pwd and just want to load
  the package's default module ("./index.js" or whatever main says).

* Add an executable script, lumi-langhost-nodejs, that fires up the
  `bin/cmd/langhost/index.js` file to serve the Node.js language plugin.
2017-09-04 11:35:21 -07:00
joeduffy 2f4d7a9789 Run integration tests
This change runs the examples integration tests for every test
run.  They used to be split out because the AWS tests take so long,
but now those are in their own separate package.  Running the
integration tests here more frequently will prevent breaking the
most basic Lumi CLI commands and capabilities.
2017-08-06 08:26:20 -07:00
joeduffy 35aa6b7559 Rename pulumi/lumi to pulumi/pulumi-fabric
We are renaming Lumi to Pulumi Fabric.  This change simply renames the
pulumi/lumi repo to pulumi/pulumi-fabric, without the CLI tools and other
changes that will follow soon afterwards.
2017-08-02 09:25:22 -07:00
joeduffy d6b6cbf4ff Build/test the minimal Lumi program as a nightly integration test 2017-07-23 12:12:43 -06:00
joeduffy e35bfc35b2 Remove AWS examples
These are moving to their respective repos.
2017-07-21 14:00:30 -07:00
joeduffy 87b7091b35 Remove serverless examples
These have moved to [pulumi/aws-serverless/examples](
https://github.com/pulumi/aws-serverless/examples/).
2017-07-21 14:00:29 -07:00
Luke Hoban 92418cf687 Refactor integration testing framework
Generalizes Lumi program validation so that it can be applied
to integration testing for other packages (such as the
pulumi/lumi-platform package examples).
2017-07-13 12:20:57 -07:00
Luke Hoban 1d7792cd83 Allow additional configuration on aws.serverless.Function
We need to provide higher level abstractions with the
ability to set additional properties on a Lambda Function
even when using the closure serialization support of
aws.serverless.Function.

Note that this is an API breaking change, and may require
updates in any other libraries dependent on this API.
2017-07-09 21:03:31 -07:00
Luke Hoban dc6594f06b Temporary workaround for #276
Disable invocation of `lumi plan` during examples
integration testing, pending resolution of #276 to
support planning in the face of output properties.
2017-06-29 10:49:05 -07:00
Luke Hoban 9ba8567d59 Support for AWS Cloudwatch Logs
Adds support for LogGroup and LogSubscriptionFilter
resources.
2017-06-28 15:42:42 -07:00
joeduffy 2daea4c3d8 Clarify aspects of using the DCO 2017-06-26 14:46:34 -07:00
joeduffy 3c1041af49 Update license headers 2017-06-23 14:53:41 -07:00
Luke Hoban 201a3d1a51 Additional nightly test fixes
Address several issues with running the Beanstalk
example in newer AWS regions with different requirements.

Ensures S3 bucket names adhere to required naming patterns
outside of us-east-1.

Also add InstanceProfile and ServiceRole configuration to the
beanstalk example as required in newer regions.
2017-06-22 16:59:16 -07:00
Luke Hoban 72bb2b2309 Further improvements for nightly tests
Make RestAPI more robust to TooManyRequestsException.

Fix imports in minimal example.

Make printing for examples test more explicit to help with diagnostics during parallel test execution.
2017-06-21 17:33:47 -07:00
Britton Forsyth 233c5a8c52 [#245, #226, #185] Finished linting (#257)
* Added typescript linting to Makefiles
* Added tslint to CI and ran on examples
* Fixed tslint calling in Makefiles
* Deleted unnecessary files
2017-06-21 11:58:22 -07:00
Luke Hoban bcc37d78a8 Nightly test fixes
Fix a couple of issues that have been preventing nightlies
from running cleanly.
2017-06-20 12:29:04 -07:00
Luke Hoban ac4a56bb6d Additional tslint cleanup in examples 2017-06-19 17:09:55 -07:00
Luke Hoban e6509e3814 Make examples tslint clean 2017-06-16 13:44:29 -07:00
Luke Hoban 6840dba051 Move examples test to seperate package
We need to run examples tests only after building and
installing all Lumi commands.
2017-06-16 09:24:31 -07:00
Luke Hoban 639a2d323d Test more examples
Tests all of our commonly used examples.

Also sets test parallelism to 10 by default
since we are I/O bound on API calls to
the resource providers.

Also avoids using larger EC2 examples in
our samples so that we can keep our test
costs lower :-).
2017-06-16 09:24:31 -07:00
Luke Hoban ae03d69645 Wire up APIs to lambdas using output properties
We now have enough output properties implementation
working to change our API gateway examples and API
wrapper to correctly wire the API routes to the ARNs of
lambdas passed in to them.

We both wire up the lambda to the route, but also create
a permission specific to each route to assign to the
corresponding lambda - providing least privelege needed
for the API definition.

Also adds `string#toUpperCase` and fixes NewUniqueHex
to match how we are using it.
2017-06-15 16:01:00 -07:00
Luke Hoban 1ba0bf7f95 Fix the serverless API example
The aws.serverless.API component was previously relying
on the fact that Lumi delayed resource creation until the
program was done executing.  With the changes to execution
for output properties, this no longer works.

For now, we will address this by change API to create the
RestAPI resource at the time of `publish`, after all of the
routes are already defined.
2017-06-15 09:06:40 -07:00
Luke Hoban 282f40d3e3 Merge branch 'master' into bforsyth927-gometalinter 2017-06-13 16:28:12 -07:00
Britton Forsyth 01003ad48b Implemented highlighted edits 2017-06-13 11:01:23 -07:00
Luke Hoban 29fcde459b Add demo script and raw serverless example
Adds an initial cut at a demo script along with
a raw version of the serverless example that
is a better stepping stone between the low-level
AWS infrastructure providers and the high-level
`aws.serverless` APIs.
2017-06-12 15:20:37 -07:00
Luke Hoban 9bd441be05 Support for nested lambdas and node_modules
LumiJS lambdas can now be serialized when they include calls to other LumiJS lambdas.  The chain of lambda dependencies is jointly serialized into the target Lambda.

Also, LumiJS lambdas now include `node_modules` automatically in the AWS Lambda, ensuring the the runtime execution environment more closely matches the deployment time environment.

An early version of the gh-cicd example supporting #134 is added which uses these capabilities, currently including a mocked GitHub resource provider.
2017-06-12 10:15:20 -07:00
joeduffy 97468ef38d Add an invoke.sh script for easier serverless demos 2017-06-10 15:59:53 -07:00
Britton Forsyth 69e4834f63 Merge branch 'master' into gometalinter 2017-06-09 14:34:51 -07:00
Britton Forsyth 13dbcdbafc Finalized tslint edits and deletes 2017-06-09 08:54:05 -07:00
Luke Hoban d77c51ff7f Allow runtime lambda to reference globals.
For lambdas which will execute at runtime,
we want to allow them to reference Node.js
global variables, like `console`.

This change makes Lumijs generated IL
incrementally more dynamic by preferring to
generate `TryLoadDynamic` over `LoadLocation`
for references to global variables (except for
references to imports).

Also introduces `console.log` in LumiJS, though
it is not yet attached to a Lumi global environment.

Fixes #174.
2017-06-08 22:06:41 -07:00
Britton Forsyth 11c74f12d9 Fixed tslint issues 2017-06-08 16:40:12 -07:00
Luke Hoban e838c6ff2d Allow lambdas to capture reference to module scope variables
The scope chain currently does not include module-scope
vairables, which are instead stored on a module object.  For
now, we are capturing this module object along with the
scope chain as part of a Lambda object so that we can use
it when evaluating variable references within a lambda
expression.

Fixes #175.
2017-06-08 15:56:50 -07:00
Luke Hoban 771a30c688 Save build artifacts aftifacts for Go builds
In the places we run `go build`, we should use
`go build -i` to save the `.a` files generated
during the build.  This ensures the artifacts
are availble for other Go tools (linters, IDEs), and
should also improve build speeds.
2017-06-07 17:03:07 -07:00
Luke Hoban 73e4be9f5e Some cleanup on AWS examples 2017-06-07 16:16:55 -07:00
joeduffy c2f2fbd2ff Regenerate some code 2017-06-04 19:37:27 -07:00
Luke Hoban 92a9925201 Merge branch 'master' into apigateway 2017-06-03 14:58:23 -07:00
Luke Hoban 5358080ca6 Output property improvements for AWS Function and Role
The AssumeRolePolicyDocument property returned by the AWS IAM GetRole API returns
a URL-encoded JSON string, so we need to decode this before JSON unmarshalling.

The Code property returned by AWS Lambda GetFunction provides a pre-signed S3 URL,
which changes on each call, and is of a different format to what is provided by the user.
For now, we'll not store this back into the Function object.

Add additional output properties to AWS Lambda Function that are stable values returned
from GetFunction.

Also corrects a gap where some property delete operations were not being correctly reported.
2017-06-01 15:04:37 -07:00
Luke Hoban c117b43ae4 Update serverless API programming model
Updates  the higher level AWS APIGateway programming model
in aws.serverless.API  to use an Express-like imperative API.
2017-06-01 10:54:26 -07:00
joeduffy 4d63e6d672 Add a few more RawResources: trues 2017-06-01 08:39:48 -07:00
Luke Hoban 01af21a1e4 Support for multiple methods on route in aws.serverless.API
Also adds length property to String objects and a toLowerCase method to the String prototype.
2017-05-31 11:45:02 -07:00
Luke Hoban 715a26bfba Introduce aws.serverless package with API and Function
This new package is similar to the AWS Serverless Application Model, offering
higher-level interfaces to manage serverless resources. This will be a candidate
for moving into its own package in the future.

The FunctionX class has been moved into this module, and a new API class has
been added

The API class manages a collection of an API Gateway RestAPI, Stage and Deployment,
based on a collection of routes linked to Functions.  On changes to the API specification,
it updates the RestAPI, replaces the Deployment and updates the Stage to point to
the updated Deployment.

This change also reorganizes some of the intrinsics.
2017-05-31 11:45:02 -07:00
Luke Hoban 1ba42954f4 WIP on AWS ApiGateway resource providers
Adds support for RestApi, Deployment and Stage resources.

Resolves #180.
2017-05-31 11:42:58 -07:00
Luke Hoban 8bbf48bf87 Support for AWS DynamoDB Table GlobalSecondaryIndexes
Adds support for global secondary indexes on DynamoDB Tables.

Also adds a HashSet API to the AWS provider library.  This handles part of #178,
providing a standard way for AWS provider implementations to compute set-based
diffs. This new API is used in both aws.dynamodb.Table and aws.elasticbeanstalk.Environment
currently.
2017-05-26 14:54:35 -07:00
Luke Hoban 7f8b1e59c1 Support for lambdas (#158)
Resolves #137.

This is an initial pass for supporting JavaScript lambda syntax for defining an AWS Lambda Function.

A higher level API for defining AWS Lambda Function objects `aws.lambda.FunctionX` is added which accepts a Lumi lambda as an argument, and uses that lambda to generate the AWS Lambda Function code package.

LumiJS lambdas are serialized as the JavaScript text of the lambda body, along with a serialized version of the environment that is deserialized at runtime and used as the context for the body of the lambda.

Remaining work to further improve support for lambdas is being tracked in #173, #174, #175, and #177.
2017-05-25 16:55:14 -07:00
Luke Hoban b8d978b22c Move Intrinsic into eval/rt package
Unifies the notion of BuiltinFunctions with the existing Intrinsic.

Intrinsic is now only a wrapper type, used to indicate the need to lookup the symbol in the
eval pacakges table of registered intrinsics.  It does not carry the invoker function used
to eval the intrinsic.
2017-05-25 12:06:13 -07:00
Luke Hoban a625117e72 Add a length property to Array objects
Also adds a `lumi.runtime.printf` function for debugging Lumi scripts and fixes a couple issues with getter/setter references.
2017-05-25 12:06:13 -07:00
Luke Hoban 9a898d88fd Add AWS DynamoDB Table provider (#151)
Adds support for Table resources, along with the beginnings of a Serverless example that uses Tables.
2017-05-21 22:17:56 -07:00
Luke Hoban 0f99762e2e Add AWS Elastic Beanstalk resource providers (#154)
Includes support for:
* Application
* ApplicationVersion
* Environment
2017-05-21 21:45:28 -07:00
joeduffy 4108c51549 Reclassify Lumi under the Apache 2.0 license
This is part of pulumi/lumi#147.
2017-05-18 14:51:52 -07:00
joeduffy b7f3d447a1 Preserve the lumi prefix on our CLI tools
This change keeps the lumi prefix on our CLI tools.

As @lukehoban pointed out in person, as soon as we do pulumi/coconut#98,
most people (other than compiler authors themselves) won't actually be
typing the commands.  And, furthermore, the commands aren't all that bad.

Eventually I assume we'll want something like `lumi-js`, or
`lumi-js-compiler`, so that binaries are discovered dynamically in a way
that is extensible for future languages.  We can tackle this during #98.
2017-05-18 12:38:58 -07:00
joeduffy dafeb77dff Rename Coconut to Lumi
This is part of pulumi/coconut#147.

After it has landed, I will rename the repo on GitHub.
2017-05-18 11:38:28 -07:00
joeduffy 4da37edf1c Remove stale submodules 2017-05-15 10:33:22 -07:00
joeduffy 06b6ce3e44 Add a GH CI/CD sample
This is the code we want to get working with pulumi/coconut#134.
My goal this week is to make that happen!
2017-04-30 09:05:02 -07:00
joeduffy 25fb74803a Add a sample "Hello, World" for Mantle
This is a sample serverless function, written in Node.js, and exposed
over an HTTP API gateway endpoint.

It even works!  (For Kubernetes Fission; the AWS support is dependent
upon projecting the various API gateway resource providers...)
2017-04-20 18:46:21 -07:00
Joe Duffy d4b5f95a0d Merge branch 'master' into webserver-comp-fix 2017-04-17 17:00:09 -07:00
joeduffy 9c1ea1f161 Fix some poor hygiene
A few linty things crept in; this addresses them.
2017-04-08 07:44:02 -07:00
joeduffy f0c1100ad6 Add basic web server example in Python 2017-04-03 17:22:37 -07:00
joeduffy 35f46dcdf7 Rename the infosec analyzer to contoso/infosec 2017-03-23 10:38:53 -07:00
joeduffy 2bda2d5b2d Tidy up the webserver example to match slides 2017-03-21 11:02:20 -07:00
joeduffy 98119f917e Rename ACMECorp security analyzer to InfoSec
This changes the example security analyzer from acmecorp/security
to just infosec, to reinforce that we will have certain analyzers
"out of the box" (infosec, cost, etc.)
2017-03-21 10:57:36 -07:00
Luke Hoban cf81603f60 Fix webserver-comp example
Coconut package name must currently be a valid identifier.
2017-03-20 22:33:03 -07:00
joeduffy aee7f0ab92 Make a few minor renames in the webserver example 2017-03-16 09:17:52 -07:00
joeduffy 963df58912 Make a few nice modifications to the webserver example
* Rename the sample from ec2instance to webserver.

* Factor out the AMI map stuff into the AWS library, rather than the sample.

* Strongly type the instance type parameter using the aws.ec2.InstanceType union.

* Add a new webserver-comp example that demonstrates a bit of the ability to do
  encapsulation, componentization, and multi-instantiation.
2017-03-15 19:55:56 -07:00
joeduffy 95f59273c8 Update copyright notices from 2016 to 2017 2017-03-14 19:26:14 -07:00
joeduffy 9e9f38014e Tidy up the ec2instance sample a bit more
And also just use HTTP port 80 to mirror the corresponding AWS docs example.
2017-03-14 04:43:37 -07:00
joeduffy 774ed45ccc Add an example cpuwatch package
This change includes a simple cpuwatch package, as a demonstration of
how easy it can be to add CPU-level monitoring to an existing stack.

It contains a single API, enableAlarm, that takes an instance and
CPU % threshold; if the CPU ever exceeds that threshold for a sustained
period of time (3 consecutive minutes), an email will be generated.

To use this, first simply install the package as usual, e.g.

    $ coco pack get cpuwatch

From there, you will need to configure the email address to send to:

    $ coco env config <env> cpuwatch:config:emailAddress joe@pulumi.com

And finally, add an import plus call to enableAlarm for all instances:

    import * as cpuwatch from "cpuwatch";
    ..
    let instance = new aws.ec2.Instance(...);
    cpuwatch.enableAlarm(instance, 90); // email if >90% CPU utilization.

As part of this, I've added the typing projections for the AWS SNS topic
and CloudWatch alarm resource types (but no providers just yet).
2017-03-13 12:26:33 -07:00
joeduffy 30171c9752 Remove port 80 access from the sample 2017-03-13 10:36:37 -07:00
joeduffy 56ffba23c6 Tidy up the sample code 2017-03-13 10:35:40 -07:00
joeduffy 8b8c1931f7 Add a simple ACMECorp security analyzer
This adds a simple ACMECorp security analyzer example.  It doesn't
actually do anything other than reject any AWS EC2 instance, claiming
it is vulnerable.  Eventually we should do something smart.
2017-03-11 10:09:11 -08:00
joeduffy 384e347115 No more nuts! 2017-03-10 13:27:19 -08:00
joeduffy cfd06083f5 Add a basic Riffmart sample
This checks in part of the Riffmart sample.

warning: this is a work-in-progress, and is incomplete.
2017-03-10 09:49:20 -08:00
joeduffy 86dc13ed5b More term rotations
This changes a few naming things:

* Rename "husk" to "environment" (`coco env` for short).

* Rename NutPack/NutIL to CocoPack/CocoIL.

* Rename the primary Nut.yaml/json project file to Coconut.yaml/json.

* Rename the compiled Nutpack.yaml/json file to Cocopack.yaml/json.

* Rename the package asset directory from nutpack/ to .coconut/.
2017-03-06 14:32:39 +00:00
joeduffy 800f3dd388 Eliminate superfluous copy of ec2instance 2017-03-06 13:04:25 +00:00
joeduffy 954fd9ae78 Use real configuration in ec2instance example
This change eliminates the hard-coded region from the ec2instance
example, and instead uses the new `aws.config.region` configuration
variable.  This makes the code more amenable to multi-instancing.
2017-02-28 16:14:18 -08:00
joeduffy 977b16b2cc Add basic targeting capability
This change partially implements pulumi/coconut#94, by adding the
ability to name targets during creation and reuse those names during
deletion and update.  This simplifies the management of deployment
records, checkpoints, and snapshots.

I've opted to call these things "husks" (perhaps going overboard with
joy after our recent renaming).  The basic idea is that for any
executable Nut that will be deployed, you have a nutpack/ directory
whose layout looks roughly as follows:

    nutpack/
        bin/
            Nutpack.json
            ... any other compiled artifacts ...
        husks/
            ... one snapshot per husk ...

For example, if we had a stage and prod husk, we would have:

    nutpack/
        bin/...
        husks/
            prod.json
            stage.json

In the prod.json and stage.json files, we'd have the most recent
deployment record for that environment.  These would presumably get
checked in and versioned along with the overall Nut, so that we
can use Git history for rollbacks, etc.

The create, update, and delete commands look in the right place for
these files automatically, so you don't need to manually supply them.
2017-02-25 09:24:52 -08:00
joeduffy 32379da4f5 Fix a few lingering issues from rename 2017-02-25 07:51:29 -08:00
joeduffy fbb56ab5df Coconut! 2017-02-25 07:25:33 -08:00
joeduffy 14e3f19437 Implement name property in AWS provider/library 2017-02-24 15:41:56 -08:00
joeduffy 271eaefb5c Fix ec2instance CIDR property 2017-02-23 15:04:07 -08:00
joeduffy 39d2fd4e96 Change mu.Bucket to mu.x.Bucket in Thumbnaile 2017-02-19 09:26:25 -08:00
joeduffy 2e8b04f2a7 Get the minimal AWS sample compiling/verifying/evaluating 2017-02-11 13:33:11 -08:00
joeduffy 0b299f0ae5 Update ec2instance so that it compiles/verifies/evals 2017-02-10 09:10:13 -08:00
joeduffy c821634761 Restructure the examples
This restructures the examples directory a bit, into three buckets:

* basic/: simplistic examples, like hello world and whatnot.

* conversions/: actual conversions from existing samples (with the source cited).

* scenarios/: more complex examples that demonstrate various features of the system.
2017-02-09 16:07:45 -08:00
joeduffy e8f54f3f4d Add a minimal MuJS blueprint example 2017-02-09 13:36:18 -08:00
joeduffy a0de263930 Add a tsconfig to the Thumbnailer example 2017-02-03 12:14:40 -08:00
joeduffy 7854e0d393 Merge branch 'master' of github.com:marapongo/mu 2017-02-03 12:09:56 -08:00
joeduffy 03b065015f Add a missing } 2017-02-03 12:09:48 -08:00
Joe Duffy dbf86736e1 Move ec2instance example underneath aws/
I just introduced the examples/aws/ directory yesterday to hold some
of our basic AWS examples, as I expect us to start accumulating more
of them.  So this just moves ec2instance underneath that directory.
We should probably think about further reorganizing these (e.g., into
demos/, scenarios/, etc.), ass we accumulate more examples.
2017-02-03 05:38:06 -08:00
Luke Hoban 464bd4fe28 Simple EC2 instance example 2017-02-02 22:03:12 -08:00
joeduffy e953c68074 Add a minimal AWS VPC blueprint to the examples 2017-02-02 10:53:35 -08:00
joeduffy d1986910b9 Check in the thumbnailer example from our docs 2017-01-28 11:33:15 -08:00
joeduffy 5374a2e3fa Add a example web crawler Mu blueprint
This adds an example web crawler blueprinted, derived from the AOSA
book's asynchronous I/O coroutines example.
2017-01-12 14:59:18 -08:00
joeduffy 6a23300d21 Reformat example; use "new()" in place of "resources" 2016-12-16 11:15:33 -08:00
joeduffy 284cec204d Experiment with intersection types
This change experiments with using intersection types to cut down
on some of the boilerplate -- and make reuse easier -- in the
service definitions.
2016-12-16 10:04:46 -08:00
joeduffy ed1eebe7e5 Clean up imports 2016-12-15 19:42:48 -08:00
joeduffy 2e941bbc57 Make an initial attempt at a better factoring
This splits the overall example rack service into many sub-services.
This leads to a much cleaner factoring of the code.  Note that there are
some missing properties -- it's hard to eyeball this without a real compiler.
But the essence of the example is pretty spot on.
2016-12-15 19:40:34 -08:00
joeduffy 68a3d27a73 Use "prop: value" instead of "prop = value"
I personally prefer this syntax.  It is more "declarative" and, particularly
because we support true assignments, I feel like it's more representative of
what's going on.  It's easier to scan for real imperative assignments.  This
approach also eliminates a single horizontal character per assignment.
2016-12-15 17:57:36 -08:00
joeduffy 14471c0ac7 Experiment with different syntax for resource creation
In this change, ":= new" declares a new resource.

In addition, we change arrays to be Go-like, in that zero-initialized
ones can be appended to safely, such that declaring a new instance is
done merely by stating "var arr: T[]".
2016-12-15 17:50:10 -08:00