When merging inputs and defaults in order to construct the set of inputs
for a call to `Create`, we must recursively merge each property value:
the provided defaults may contain nested values that must be present in
the merged result.
These changes introduce a new field, `Raw`, to `diag.Message`. This
field indicates that the contents of the message are not a format string
and should not be rendered via `Sprintf` during stringification.
The plugin std{out,err} readers have been updated to use raw messages,
and the event reader in `pulumi` has been fixed s.t. it does not format
event payloads before display.
Fixes#551.
Add the ability to upload data and timing for test runs to S3. Uploaded data is designed to be queried via a service like AWS Athena and these queries can then be imported into BI tools (Excel, QuickSight, PowerBI, etc.)
Initially hook this up to the `minimal` test as a baseline.
This adds a minimal runtime verification test to our basic
test suite, to at least exercise the portions of the integration
test library that load up and parse checkpoint files.
The `nodejs` language support is implemented as two programs: one that
manages the initial connection to the engine and provides the language
serivce itself, and another that the language service invokes in order
to run a `nodejs` Pulumi program. The latter is responsible for running
the user's program and communicating its resource requests to the
engine. Currently, `run` effectively assumes that the user's program
will run synchronously from start to finish, and will disconnect from
the engine once the user's program has completed. This assumption breaks
if the user's program requires multiple turns of the event loop to
finish its root resource requests. For example, the following program
would fail to create its second resource because the engine will be
disconnected once it reaches its `await`:
```
(async () => {
let a = new Resource();
await somePromise();
let = new Resource();
})();
```
These changes fix this issue by disconnecting from the engine during
process shutdown rather than after the user's program has finished its
first turn through the event loop.
We now encrypt secrets at rest based on a key derived from a user
suplied passphrase.
The system is designed in a way such that we should be able to have a
different decrypter (either using a local key or some remote service
in the Pulumi.com case in the future).
Care is taken to ensure that we do not leak decrypted secrets into the
"info" section of the checkpoint file (since we currently store the
config there).
In addtion, secrets are "pay for play", a passphrase is only needed
when dealing with a value that's encrypted. If secure config values
are not used, `pulumi` will never prompt you for a
passphrase. Otherwise, we only prompt if we know we are going to need
to decrypt the value. For example, `pulumi config <key>` only prompts
if `<key>` is encrypted and `pulumi deploy` and friends only prompt if
you are targeting a stack that has secure configuration assoicated
with it.
Secure values show up as unecrypted config values inside the language
hosts and providers.
During the course of a `pulumi update`, it is possible for a resource to
become slated for deletion. In the case that this deletion is part of a
replacement, another resource with the same URN as the to-be-deleted
resource will have been created earlier. If the `update` fails after the
replacement resource is created but before the original resource has been
deleted, the snapshot must capture that the original resource still exists
and should be deleted in a future update without losing track of the order
in which the deletion must occur relative to other deletes. Currently, we
are unable to track this information because the our checkpoints require
that no two resources have the same URN.
To fix this, these changes introduce to the update engine the notion of a
resource that is pending deletion and change checkpoint serialization to
use an array of resources rather than a map. The meaning of the former is
straightforward: a resource that is pending deletion should be deleted
during the next update.
This is a fairly major breaking change to our checkpoint files, as the
map of resources is no more. Happily, though, it makes our checkpoint
files a bit more "obvious" to any tooling that might want to grovel
or rewrite them.
Fixes#432, #387.
A dynamic resource is a resource whose provider is implemented alongside
the resource itself. This provider may close over and use orther
resources in the implementation of its CRUD operations. The provider
itself must be stateless, as each CRUD operation for a particular
dynamic resource type may use an independent instance of the provider.
Changes to the definition of a resource's provider result in replacement
of the resource itself (rather than a simple update), as this allows the
old provider definition to delete the old resource and the new provider
definition to create an appropriate replacement.
Previously, you had to fully qualify configuration values (e.g
example:config:message). As a convience, let's support adding
configuration values where the key is not a fully qualified module
member. In this case, we'll treat the key as if
`<program-name>:config:` had been prepended to it.
In addition, when we print config, shorten keys of the form
`<program-name>:config:<key-name>` to `<key-name>`.
I've updated one integration test to use the new syntax and left the
other as is to ensure both continue to work.
This changes a few things about "components":
* Rename what was previously ExternalResource to CustomResource,
and all of the related fields and parameters that this implies.
This just seems like a much nicer and expected name for what
these represent. I realize I am stealing a name we had thought
about using elsewhere, but this seems like an appropriate use.
* Introduce ComponentResource, to make initializing resources
that merely aggregate other resources easier to do correctly.
* Add a withParent and parentScope concept to Resource, to make
allocating children less error-prone. Now there's no need to
explicitly adopt children as they are allocated; instead, any
children allocated as part of the withParent callback will
auto-parent to the resource provided. This is used by
ComponentResource's initialization function to make initialization
easier, including the distinction between inputs and outputs.
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
* Remove the bitrotted and useless basic/abc/ test.
* No need for the basic/ subdirectory. Move minimal to the top.
* Update TypeScript to 2.5.3.
* Check in lockfiles to ensure repeatability in Travis tests.
This resource provider accepts a single configuration parameter, `testing:provider:module`, that is the path to a Javascript module that implements CRUD operations for a set of resource types. This allows e.g. a test case to provide its own implementation of these operations that may succeed or fail in interesting ways.
Fixes#338.
This includes a few changes:
* The repo name -- and hence the Go modules -- changes from pulumi-fabric to pulumi.
* The Node.js SDK package changes from @pulumi/pulumi-fabric to just pulumi.
* The CLI is renamed from lumi to pulumi.
This makes a few tweaks to get the integration tests passing:
* Add `runtime: nodejs` to the minimal example's `Lumi.yaml` file.
* Remove usage of `@lumi/lumirt { printf }` and just use `console.log`.
* Remove calls to `lumijs` in the integration test framework and
the minimal example's package.json. Instead, we just run
`yarn run build`, which itself internally just invokes `tsc`.
* Add package validation logic and eliminate the pkg/compiler/metadata
library, in favor of the simpler code in pkg/engine.
* Simplify the Node.js langhost plugin CLI, and simply take an
argument rather than requiring required and optional --flags.
* Use a default path of "." if the program path isn't provided. This
is a legal scenario if you've passed a pwd and just want to load
the package's default module ("./index.js" or whatever main says).
* Add an executable script, lumi-langhost-nodejs, that fires up the
`bin/cmd/langhost/index.js` file to serve the Node.js language plugin.
This change runs the examples integration tests for every test
run. They used to be split out because the AWS tests take so long,
but now those are in their own separate package. Running the
integration tests here more frequently will prevent breaking the
most basic Lumi CLI commands and capabilities.
We are renaming Lumi to Pulumi Fabric. This change simply renames the
pulumi/lumi repo to pulumi/pulumi-fabric, without the CLI tools and other
changes that will follow soon afterwards.
Generalizes Lumi program validation so that it can be applied
to integration testing for other packages (such as the
pulumi/lumi-platform package examples).
We need to provide higher level abstractions with the
ability to set additional properties on a Lambda Function
even when using the closure serialization support of
aws.serverless.Function.
Note that this is an API breaking change, and may require
updates in any other libraries dependent on this API.
Disable invocation of `lumi plan` during examples
integration testing, pending resolution of #276 to
support planning in the face of output properties.
Address several issues with running the Beanstalk
example in newer AWS regions with different requirements.
Ensures S3 bucket names adhere to required naming patterns
outside of us-east-1.
Also add InstanceProfile and ServiceRole configuration to the
beanstalk example as required in newer regions.
Make RestAPI more robust to TooManyRequestsException.
Fix imports in minimal example.
Make printing for examples test more explicit to help with diagnostics during parallel test execution.
Tests all of our commonly used examples.
Also sets test parallelism to 10 by default
since we are I/O bound on API calls to
the resource providers.
Also avoids using larger EC2 examples in
our samples so that we can keep our test
costs lower :-).
We now have enough output properties implementation
working to change our API gateway examples and API
wrapper to correctly wire the API routes to the ARNs of
lambdas passed in to them.
We both wire up the lambda to the route, but also create
a permission specific to each route to assign to the
corresponding lambda - providing least privelege needed
for the API definition.
Also adds `string#toUpperCase` and fixes NewUniqueHex
to match how we are using it.
The aws.serverless.API component was previously relying
on the fact that Lumi delayed resource creation until the
program was done executing. With the changes to execution
for output properties, this no longer works.
For now, we will address this by change API to create the
RestAPI resource at the time of `publish`, after all of the
routes are already defined.
Adds an initial cut at a demo script along with
a raw version of the serverless example that
is a better stepping stone between the low-level
AWS infrastructure providers and the high-level
`aws.serverless` APIs.
LumiJS lambdas can now be serialized when they include calls to other LumiJS lambdas. The chain of lambda dependencies is jointly serialized into the target Lambda.
Also, LumiJS lambdas now include `node_modules` automatically in the AWS Lambda, ensuring the the runtime execution environment more closely matches the deployment time environment.
An early version of the gh-cicd example supporting #134 is added which uses these capabilities, currently including a mocked GitHub resource provider.
For lambdas which will execute at runtime,
we want to allow them to reference Node.js
global variables, like `console`.
This change makes Lumijs generated IL
incrementally more dynamic by preferring to
generate `TryLoadDynamic` over `LoadLocation`
for references to global variables (except for
references to imports).
Also introduces `console.log` in LumiJS, though
it is not yet attached to a Lumi global environment.
Fixes#174.
The scope chain currently does not include module-scope
vairables, which are instead stored on a module object. For
now, we are capturing this module object along with the
scope chain as part of a Lambda object so that we can use
it when evaluating variable references within a lambda
expression.
Fixes#175.
In the places we run `go build`, we should use
`go build -i` to save the `.a` files generated
during the build. This ensures the artifacts
are availble for other Go tools (linters, IDEs), and
should also improve build speeds.
The AssumeRolePolicyDocument property returned by the AWS IAM GetRole API returns
a URL-encoded JSON string, so we need to decode this before JSON unmarshalling.
The Code property returned by AWS Lambda GetFunction provides a pre-signed S3 URL,
which changes on each call, and is of a different format to what is provided by the user.
For now, we'll not store this back into the Function object.
Add additional output properties to AWS Lambda Function that are stable values returned
from GetFunction.
Also corrects a gap where some property delete operations were not being correctly reported.
This new package is similar to the AWS Serverless Application Model, offering
higher-level interfaces to manage serverless resources. This will be a candidate
for moving into its own package in the future.
The FunctionX class has been moved into this module, and a new API class has
been added
The API class manages a collection of an API Gateway RestAPI, Stage and Deployment,
based on a collection of routes linked to Functions. On changes to the API specification,
it updates the RestAPI, replaces the Deployment and updates the Stage to point to
the updated Deployment.
This change also reorganizes some of the intrinsics.
Adds support for global secondary indexes on DynamoDB Tables.
Also adds a HashSet API to the AWS provider library. This handles part of #178,
providing a standard way for AWS provider implementations to compute set-based
diffs. This new API is used in both aws.dynamodb.Table and aws.elasticbeanstalk.Environment
currently.
Resolves#137.
This is an initial pass for supporting JavaScript lambda syntax for defining an AWS Lambda Function.
A higher level API for defining AWS Lambda Function objects `aws.lambda.FunctionX` is added which accepts a Lumi lambda as an argument, and uses that lambda to generate the AWS Lambda Function code package.
LumiJS lambdas are serialized as the JavaScript text of the lambda body, along with a serialized version of the environment that is deserialized at runtime and used as the context for the body of the lambda.
Remaining work to further improve support for lambdas is being tracked in #173, #174, #175, and #177.
Unifies the notion of BuiltinFunctions with the existing Intrinsic.
Intrinsic is now only a wrapper type, used to indicate the need to lookup the symbol in the
eval pacakges table of registered intrinsics. It does not carry the invoker function used
to eval the intrinsic.
This change keeps the lumi prefix on our CLI tools.
As @lukehoban pointed out in person, as soon as we do pulumi/coconut#98,
most people (other than compiler authors themselves) won't actually be
typing the commands. And, furthermore, the commands aren't all that bad.
Eventually I assume we'll want something like `lumi-js`, or
`lumi-js-compiler`, so that binaries are discovered dynamically in a way
that is extensible for future languages. We can tackle this during #98.
This is a sample serverless function, written in Node.js, and exposed
over an HTTP API gateway endpoint.
It even works! (For Kubernetes Fission; the AWS support is dependent
upon projecting the various API gateway resource providers...)
This changes the example security analyzer from acmecorp/security
to just infosec, to reinforce that we will have certain analyzers
"out of the box" (infosec, cost, etc.)
* Rename the sample from ec2instance to webserver.
* Factor out the AMI map stuff into the AWS library, rather than the sample.
* Strongly type the instance type parameter using the aws.ec2.InstanceType union.
* Add a new webserver-comp example that demonstrates a bit of the ability to do
encapsulation, componentization, and multi-instantiation.
This change includes a simple cpuwatch package, as a demonstration of
how easy it can be to add CPU-level monitoring to an existing stack.
It contains a single API, enableAlarm, that takes an instance and
CPU % threshold; if the CPU ever exceeds that threshold for a sustained
period of time (3 consecutive minutes), an email will be generated.
To use this, first simply install the package as usual, e.g.
$ coco pack get cpuwatch
From there, you will need to configure the email address to send to:
$ coco env config <env> cpuwatch:config:emailAddress joe@pulumi.com
And finally, add an import plus call to enableAlarm for all instances:
import * as cpuwatch from "cpuwatch";
..
let instance = new aws.ec2.Instance(...);
cpuwatch.enableAlarm(instance, 90); // email if >90% CPU utilization.
As part of this, I've added the typing projections for the AWS SNS topic
and CloudWatch alarm resource types (but no providers just yet).
This adds a simple ACMECorp security analyzer example. It doesn't
actually do anything other than reject any AWS EC2 instance, claiming
it is vulnerable. Eventually we should do something smart.
This changes a few naming things:
* Rename "husk" to "environment" (`coco env` for short).
* Rename NutPack/NutIL to CocoPack/CocoIL.
* Rename the primary Nut.yaml/json project file to Coconut.yaml/json.
* Rename the compiled Nutpack.yaml/json file to Cocopack.yaml/json.
* Rename the package asset directory from nutpack/ to .coconut/.
