Commit graph

163 commits

Author SHA1 Message Date
CyrusNajmabadi 9d0dc65f49
Provide helper to compute whihc sub-packages should be included even if we would exclude a higher package. (#1883) 2018-09-05 12:54:28 -07:00
CyrusNajmabadi ddd83fafc4
Support serializing regex instances. (#1870) 2018-09-03 23:14:00 -07:00
joeduffy 70e1dbfe19 Incorporate PR feedback
* Use Promise.resolve.

* Use `Inputs | Promise<Inputs> | Output<Inputs>` rather than
  `Input<Inputs>`, which looks supremely bizarre.

* Update ComponentResource.registerOutputs also.
2018-09-02 12:15:58 -07:00
joeduffy 44c17e4877 Allow promise exports
This change partly addresses pulumi/pulumi#1611, by permitting you
to export a promise at the top-level, and have it be recognized as
a stack output. In other words, you can now say things like

    async function main() {
        ...
        return {
            a: "x",
            ...,
            z: 42,
        };
    }

    module.exports = main();

and your Pulumi program will record distinct outputs as you'd hope:

    ---outputs:---
    a: "x"
    ...
    z: 42

This is arguably just a bug in the way we implemented stack outputs.
The remainder of the requests in #1611 will remain open for future
design and discussion, as they have more subtle ramifications.
2018-09-02 10:41:04 -07:00
Joe Duffy f8d0ff87fd
Merge pull request #1834 from pulumi/joeduffy/1671_more_config_types
Add more config helpers
2018-08-28 21:57:34 -07:00
joeduffy c2258be408 Adopt code review feedback for config constraints
This change adopts the code review suggestion to use a bag of options
for config constraints rather than having overloaded function names.
This is a much cleaner approach, lets us use more descriptive names,
and is far more future proof in case we decide to add more capabilities.
2018-08-28 18:07:26 -07:00
joeduffy 834987bcd3 Add more config helpers
Everytime I convert a CloudFormation template to Pulumi, I inevitably
run into the fact that CloudFormation has advanced "schema" capabilities
for input variables, like min/max for numbers and string lengths, enums,
and regex pattern matching. This is always cumbersome to convert.

In this change, I've added a number of config helpers for these cases:

For string enums:

    getEnum(key: string, values: string[]): string | undefined;
    requireEnum(key: string: values: string[]): string;

For min/max strlen:

    getMinMax(key: string, min: number, max: number): string | undefined;
    requireMinMax(key: string, min: number, max: number): string;

For regex patterns:

    getPattern(key: string, regexp: string | RegExp): string | undefined;
    requirePattern(key: string, regexp: string | RegExp): string;

For min/max strlen _and_ regex patterns:

    getMinMaxPattern(key: string, min: number, max: number,
        regexp: string | RegExp): string | undefined;
    requireMinMaxPattern(key: string, min: number, max: number,
        regexp: string | RegExp): string;

For min/max numbers:

    getNumberMinMax(key: string, min: number, max: number): number | undefined;
    requireNumberMinMax(key: string, min: number, max: number): number;

Each function throws a detailed RunError-derived exception type if the
configuration value doesn't meet the constraint.

This fixes pulumi/pulumi#1671.
2018-08-28 13:22:53 -07:00
CyrusNajmabadi f4e04e4a5d
Add test to demonstrate issue is fixed. (#1829) 2018-08-27 14:25:37 -07:00
Sean Gillespie a0cf415179
Fix an issue with NodeJS host logging (#1819)
* Fix an issue with NodeJS host logging

Related to pulumi/pulumi#1694. This issue prevented the language host
from being aware that an engine (logging endpoint) was available and
thus no log messages were sent to the engine. By default, the language
host wrote them to standard out instead, which resulted in a pretty bad
error experience.

This commit fixes the PR and adds machinery to the NodeJS langhost tests
for testing the engine RPC endpoint. It is now possible to give a "log"
function to tests which will be hooked up to the "log" RPC endpoint
normally provided by the Pulumi engine.

* Remove accidental console.log
2018-08-24 16:50:09 -07:00
CyrusNajmabadi e05cad9424
Emit export at the end-of-file for factory functions. (#1812) 2018-08-22 12:33:01 -07:00
CyrusNajmabadi b927b5726d
Add support for serializing 'factory' functions. (#1804) 2018-08-21 12:29:30 -07:00
Pat Gavlin 58a75cbbf4
Pull default options from a resource's parent. (#1748)
If a resource's options bag does not specify `protect` or `provider`,
pull a default value from the resource's parent.

In order to allow a parent resource to specify providers for multiple
resource types, component resources now accept an optional map from
package name to provider instance. When a custom resource needs a
default provider from its parent, it checks its parent provider bag for
an entry under its package. If a component resource does not have a
provider bag, its pulls a default from its parent.

These changes also add a `parent` field to `InvokeOptions` s.t. calls to
invoke can use the same behavior as resource creation w.r.t. providers.

Fixes #1735, #1736.
2018-08-10 16:18:21 -07:00
Matt Ellis c8b1872332
Merge pull request #1698 from pulumi/ellismg/fix-1581
Allow eliding name in pulumi.Config .ctor
2018-08-08 14:16:20 -07:00
CyrusNajmabadi 152fde0867
Strip off the node_modules prefix from our require() calls. (#1729) 2018-08-07 20:27:02 -04:00
Matt Ellis 153729683a Allow eliding name in pulumi.Config .ctor
When this argument is not provided, we'll default to the value of
pulumi.getProject(). This is what you want for application level code
anyway and it matches the CLI behavior where if you don't qualify a
key with a package we use the name of the current project.

Fixes #1581
2018-08-06 16:03:54 -07:00
Matt Ellis 5eb78af779 Do not lazy initialize config or settings
The pulumi runtime used to lazily load and parse both config and
settings data set by the language host. The initial reason for this
design was that we wanted the runtime to be usable in a normal node
environment, but we have moved away from supporting that.

In addition, while we claimed we loaded these value "lazily", we
actually forced their loading quite eagerly when we started
up. However, when capturing config (or settings, as we now do), we
would capture all the logic about loading these values from the
environment.

Even worse, in the case where you had two copies of @pulumi/pulumi
loaded, it would be possible to capture a config object which was not
initialized and then at runtime the initialization logic would try to
read PULUMI_CONFIG from the process environment and fail.

So we adopt a new model where configuration and settings are parsed as
we load their containing modules. In addition, to support SxS
scinerios, we continue to use `process.env` as a way to control both
configuration and settings. This means that `run.ts` must now ensure
that these values are present in the environment before either the
config or runtime modules have been loaded.
2018-08-06 15:53:38 -07:00
CyrusNajmabadi c57aef785b
Ensure we can capture non-built-in modules with 'require'. (#1685) 2018-08-02 16:25:49 -04:00
CyrusNajmabadi d19942f2b0
Go back to capturing *non-user* modules by 'require' reference. (#1655) 2018-07-31 11:37:46 -04:00
CyrusNajmabadi 48df5bfe1e
Update pulumi/pulumi to run on Nodejs v10. (#1658) 2018-07-25 16:55:20 -07:00
CyrusNajmabadi 4cebc381ae
Do a better job preventing serialization of unnecessary objects in closure serialization (#1543) 2018-06-20 12:57:57 -07:00
Luke Hoban 076d8887c9
Compute required packages during closure serialization (#1457)
Closure serialization now keeps track of the `require`d packages it sees in the function bodies that are serialized during a call to `serializeFunction`.

Also, replaces `serializeFunctionAsync` with `serializeFunction` which accepts richer parameters and return type, deprecating the former API (but leaving it available for now to avoid a breaking change).
2018-06-03 21:55:37 -07:00
Sean Gillespie 38cf2de39e
Issue a better error message if you capture a V8 intrinsic (#1423) 2018-05-24 11:54:31 -07:00
Pat Gavlin a2c30f75ed
Resolve missing outputs as undefined. (#1427)
Before the changes in #1414, all output properties were guaranteed to
have values after deserialization. After #1414, any properties with no
value were no longer resolved, which was treated as an error. These
changes resolve all missing proprties to `undefined`. If a property is
missing during an update, its `undefined` value is marked as known.
2018-05-24 11:22:08 -07:00
Pat Gavlin a16a880518
Discriminate unknown values in the JS runtime. (#1414)
These changes add support for distinguishing an output property with
an unknown value from an output property with a known value that is
undefined.

In a broad sense, the Pulumi property type system is just JSON with the
addition of unknown values. Notably absent, however, are undefined
values. As it stands, our marshalers between JavaScript and Pulumi
property values treat all undefined JavaScript values as unknown Pulumi
values. Unfortunately, this conflates two very different concepts:
unknown Pulumi values are intended to represent values of output
properties that are unknown at time of preview, _not_ values that are
known but undefined. This results in difficulty reasoning about when
transforms are run on output properties as well as confusing output in
the `diff` view of Pulumi preview (user-specifed undefined values are
rendered as unknown values).

As it turns out, we already have a way to decide whether or not an
Output value is known or not: Output.performApply. These changes rename
this property to `isKnown`, clarify its meaning, and take advantage of
the result to decide whether or not an Output value should marshal as
an unknown Pulumi value.

This also allowed these changes to improve the serialization of
undefined object keys and array elements s.t. we better match JavaScript
to JSON serialization behavior (undefined object keys are omitted;
undefined array elements are marshaled as `null`).

Fixes https://github.com/pulumi/pulumi-cloud/issues/483.
2018-05-23 14:47:40 -07:00
joeduffy 5967259795 Add license headers 2018-05-22 15:02:47 -07:00
Sean Gillespie 82ac202139
Improve the promise leak experience (#1374)
* Fix a bug in promise leak detection that leaked promises when errors occur

* Add an opt-in to the super-verbose debug error message on promise leaks

* Fix a bad merge

* was/were grammar improvement in error message

* Fail the deployment if a debuggable promise leaks
2018-05-17 15:32:39 -07:00
CyrusNajmabadi 4807de038c
Capture 'exports' if referenced in user code. (#1359) 2018-05-11 16:35:41 -07:00
CyrusNajmabadi 6de1cead5a
Fix issue where we were not properly serializing out the prototype for an object. (#1352) 2018-05-11 15:53:16 -07:00
CyrusNajmabadi 5387e78cfa
Support async function serialization. (#1311) 2018-05-03 12:25:52 -07:00
Matt Ellis 409477b951 Invoke node directly from the language host
Instead of using a shell script to jump from the language host into
node, just invoke node directly. This makes our start-up path a little
simpler to understand and indirectly fixes pulumi/home#156, where we
would fail on Windows if the `-exec` script was in a folder that had
spaces in it (due to a subtle interaction between how go launches cmd
files and how cmd.exe parses arguments).
2018-05-02 11:16:58 -07:00
Pat Gavlin 16f1930069
Pass "special" properties to Invoke. (#1277)
Rather than filtering out the `id` and `urn` properties when serializing
the inputs to an invoke, pass these properties along. This enables the
use of invoke endpoints that accepts these as inputs (e.g. the endpoint
that backs `aws.ec2.getSubnet`).
2018-05-01 15:05:42 -07:00
joeduffy 6f4423895c Don't use instanceof for RTTI
This change moves us away from using JavaScript RTTI, by way of
`instanceof`, for built-in Pulumi types.  If we use `instanceof`,
then the same logical type loaded from separate copies of the
SDK package -- as will happen in SxS scenarios -- are considered
different.  This isn't actually what we want.  The solution is
simple: implement our own quasi-RTTI solution, using __pulumi*
properties and manual as* and is* functions.  Note that we could
have skipped the as* and is* functions, but I found that they led
to slightly easier to read code.

There is one strange thing in here, which I spoke to
@CyrusNajmabadi about: SerializedOutput<T>, because it implements
Output<T> as an _interface_, did not previously masquerade as an
actual Output<T>.  In other words, `instanceof` would have returned
false, and indeed a few important properties (like promise) are
missing.  This change preserves that behavior, although I'll admit
that this is slightly odd.  I suspect we'll want to revisit this as
part of https://github.com/pulumi/pulumi/issues/1074.

Fixes https://github.com/pulumi/pulumi/issues/1203.
2018-04-16 14:08:10 -07:00
Sean Gillespie 1f5be5f8cd
Bring the magic of Pulumi to Node v8.11.1, v9.11.1, v6.10.3 (#1167)
* Introduce a simple repetition operator to match expected error messages against actual ones

* Convert required and optional objects to use a Map (node v9 compat), improve the error formatting for failed tests

* Test node v6, v8, and v9 in CI

* Get rid of PULUMI_API env in .travis.yml, it's set from the Travis console now
2018-04-14 11:50:01 -07:00
Joe Duffy 150f57168a
Fix SxS config (#1175)
We weren't properly lazily loading everywhere we need to.
2018-04-13 11:26:01 -07:00
CyrusNajmabadi 97c1344035
Disallow capturing 'this' inside a lambda (#1138) 2018-04-09 15:57:39 -07:00
Joe Duffy 62042cd3ed
Simplify resource lookup (#1133)
As I began to write code using the ability to perform resource
lookups, especially in the code-generators, I realized the way it
was surfaced as an argument to the Resource base constructor would
lead to overload explosion.  Instead of doing that, let's pass it
in the ResourceOptions bag.
2018-04-07 10:15:58 -07:00
Joe Duffy 28033c22bc
Allow multiple Pulumi SDKs side-by-side (#1132)
Prior to this change, if you ended up with multiple Pulumi SDK
packages loaded side-by-side, we would fail in obscure ways.  The
reason for this was that we initialize and store important state
in static variables.  In the case that you load the same library
twice, however, you end up with separate copies of said statics,
which means we would be missing engine RPC addresses and so on.

This change adds the ability to recover from this situation by
mirroring the initialized state in process-wide environment
variables.  By doing this, we can safely recover simply by reading
them back when we detect that they are missing.  I think we can
eventually go even further here, and eliminate the entry point
launcher shim altogether by simply having the engine launch the
Node program with the right environment variables.  This would
be a nice simplification to the system (fewer moving pieces).

There is still a risk that the separate copy is incompatible.
Presumably the reason for loading multiple copies is that the
NPM/Yarn version solver couldn't resolve to a shared version.
This may yield obscure failure modes should RPC interfaces change.
Figuring out what to do here is part of pulumi/pulumi#957.

This fixes pulumi/pulumi#777 and pulumi/pulumi#1017.
2018-04-07 08:02:59 -07:00
joeduffy 5e28a4ab07 Add the ability to read an existing resource
This change wires up the new Read RPC method in such a manner that
Pulumi programs can invoke it.  This is technically not required for
refreshing state programmatically (as in pulumi/pulumi#1081), however
it's a feature we had eons ago and have wanted since (see
pulumi/pulumi#83), and will allow us to write code like

    let vm = aws.ec2.Instance.get("my-vm", "i-07043cd97bd2c9cfc");
    // use any property from here on out ...

The way this works is simply by bridging the Pulumi program via its
existing RPC connection to the engine, much like Invoke and
RegisterResource RPC requests already do, and then invoking the proper
resource provider in order to read the state.  Note that some resources
cannot be uniquely identified by their ID alone, and so an extra
resource state bag may be provided with just those properties required.

This came almost for free (okay, not exactly) and will come in handy as
we start gaining experience with reading live state from resources.
2018-04-05 09:48:09 -07:00
Pat Gavlin a23b10a9bf
Update the copyright end date to 2018. (#1068)
Just what it says on the tin.
2018-03-21 12:43:21 -07:00
CyrusNajmabadi d719e7966e
Only avoid running transformations on outputs we truly do not have values for. (#921) 2018-03-18 00:15:22 -07:00
CyrusNajmabadi 134941dd22
Improve error text when lambda functions are involved. (#1046) 2018-03-13 13:21:07 -07:00
CyrusNajmabadi 5b244dbdb1
Use a class for Output serialization to ensure that .apply exists on it. (#1040)
Also, rename/cleanup a bunch of serialization code.

Also, generate better environment names in the serialized closure code. Thsi code should be much easier to make sense of as hte names will better track to the original names in the user code.

Also, dedupe simple non-capturing functions. This helps ensure we don't spit out N copies of __awaiter (one per file it is declared in).
2018-03-12 16:27:00 -07:00
CyrusNajmabadi 850130e30f
Capture modules through normal value capture. (#1030) 2018-03-11 00:11:53 -08:00
Matt Ellis 1515889a40 Remove the need for the :config: part of a config bag
We now unify new Config("package") and new Config("package:config"),
printing a warning when the new Config("package:config") form is
used and pointing consumers towards just new Config("package")

I've updated our examples to use the newer syntax, but I've added a
test ot the langhost to ensure both forms work.

Fixes #923
2018-03-08 10:52:25 -08:00
CyrusNajmabadi c544accfa6
Only attempt to serialize the properties of an object that are actually used. (#1000) 2018-03-07 21:10:12 -08:00
CyrusNajmabadi 410351f571
Do not throw if 'typeof' references an undeclared variable. (#998) 2018-03-05 12:58:25 -08:00
CyrusNajmabadi e7c0e4cdaa
Make many fixes to closure serialization (#944)
Make many fixes to closure serialization

Primary things that i've done as part of this change:

    Added support for cyclic objects.
    Properly serialize objects that are shared across different function. previously you would get multiple copies, now you properly reference the same copy.
    Remove the usages of 'hashes' for functions. Because we track identity of objects, we no longer need them.
    Serialize properties of functions (if they have any).
    Handle Objects/Functions with different __proto__s than normal. i.e. classes/constructors. but also anything the user may have done themselves to the object.
    Handle generator functions.
    Handle functions with 'computed' names.
    Handle functions with 'symbol' names.
    Handle serializing Promises as Promises.
    Removed the dual Closure/AsyncClosure tree. One existed solely so we could have a tree without promises (for use in testing maybe?). Because this all exists in a part of our codebase that is entirely async, it's fine to have promises in the tree, and to await them when serializing the Closure to a string.
    Handle serializing class-constructors and methods. Including properly handling 'super' calls.
2018-03-01 00:32:01 -08:00
Sean Gillespie b84320b45e
Code review feedback:
1. Various idiomatic Go and TypeScript fixes
    2. Add an integration test that end-to-end roundtrips dependency
    information for a simple Pulumi program
    3. Add an additional test assert that tests that dependency information
    comes from the language host as expected
2018-02-22 13:33:50 -08:00
joeduffy 548c22d014 Reimplement GetRequiredPlugins in Go
This brings back the Node.js language plugin's GetRequiredPlugins
function, reimplemented in Go now that the language host has been
rewritten from JavaScript.  Fairly rote translation, along with
some random fixes required to get tests passing again.
2018-02-18 08:08:15 -08:00
joeduffy c04341edb2 Consult the program for its list of plugins
This change adds a GetRequiredPlugins RPC method to the language
host, enabling us to query it for its list of plugin requirements.
This is language-specific because it requires looking at the set
of dependencies (e.g., package.json files).

It also adds a call up front during any update/preview operation
to compute the set of plugins and require that they are present.
These plugins are populated in the cache and will be used for all
subsequent plugin-related operations during the engine's activity.

We now cache the language plugins, so that we may load them
eagerly too, which we never did previously due to the fact that
we needed to pass the monitor address at load time.  This was a
bit bizarre anyhow, since it's really the Run RPC function that
needs this information.  So, to enable caching and eager loading
-- which we need in order to invoke GetRequiredPlugins -- the
"phone home" monitor RPC address is passed at Run time.

In a subsequent change, we will switch to faulting in the plugins
that are missing -- rather than erroring -- in addition to
supporting the `pulumi plugin install` CLI command.
2018-02-18 08:08:15 -08:00
Sean Gillespie e87204d3e1
Move language host logic from Node to Go (#901)
* experimental: separate language host from node

* Remove langhost details from the NodeJS SDK runtime

* Cleanup

* Work around an issue where Node sometimes loads the same module twice in two different contexts, resulting in two distinct module objects. Some additional cleanup.

* Add some tests

* Fix up the Windows script

* Fix up the install scripts and Windows build

* Code review feedback

* Code review feedback: error capitalization
2018-02-10 02:15:04 +00:00
CyrusNajmabadi 296151e088
Support serializing methods to the inside layer. (#904) 2018-02-09 14:22:03 -08:00
CyrusNajmabadi 275670692b
Introduce Output<T> and update Resource construction code to properly handle it. (#834)
This PR adds a new formalisms at the Resource layer.  First all inputs to a Resource are typed as ```Input<T>```.  This is either a T, ```Promise<T>``
2018-02-05 14:44:23 -08:00
CyrusNajmabadi f35f991140
Extract out serialization functionality from transfer function. (#835) 2018-01-25 13:34:21 -08:00
pat@pulumi.com 4cb7703676 Add a test. 2017-12-13 17:30:43 -08:00
Pat Gavlin 370c926c80
Merge pull request #680 from pulumi/TestDeserializeAssets
Test the asset deserialization changes from #677.
2017-12-08 16:04:19 -08:00
pat@pulumi.com 5ef0dcf598 Test the asset deserialization changes from #677.
Just what it says on the tin.
2017-12-08 15:37:30 -08:00
CyrusNajmabadi 6e68163cc5
Fix 'this' capture in async functions and lambdas. (#678) 2017-12-08 14:57:51 -08:00
Joe Duffy 69f5882b97
Fix two closure bugs (#664)
This fixes two closure bugs.

First, we had special cased `__awaiter` from days of yore, when we had
special cased its capture.  I also think we were confused at some point
and instead of fixing the fact that we captured `this` for non-arrow
functions, which `__awaiter` would trigger, we doubled down on this
incorrect hack.  This means we missed a real bonafide `this` capture.

Second, we had a global cache of captured variable objects.  So, if a
free variable resolved to the same JavaScript object, it always resolved
to the first serialization of that object.  This is clearly wrong if
the object had been mutated in the meantime.  The cache is required to
reach a fixed point during mutually recursive captures, but we should
only be using it for the duration of a single closure serialization
call.  That's precisely what this commit does.

Also add a fix for this case.

This fixes pulumi/pulumi#663.
2017-12-07 16:21:28 -08:00
joeduffy a4c7c05e27 Simplify RPC changes
This change simplifies the necessary RPC changes for components.
Instead of a Begin/End pair, which complicates the whole system
because now we have the opportunity of a missing End call, we will
simply let RPCs come in that append outputs to existing states.
2017-11-29 12:08:01 -08:00
joeduffy c5b7b6ef11 Bring back component outputs
This change brings back component outputs to the overall system again.
In doing so, it generally overhauls the way we do resource RPCs a bit:

* Instead of RegisterResource and CompleteResource, we call these
  BeginRegisterResource and EndRegisterResource, which begins to model
  these as effectively "asynchronous" resource requests.  This should also
  help with parallelism (https://github.com/pulumi/pulumi/issues/106).

* Flip the CLI/engine a little on its head.  Rather than it driving the
  planning and deployment process, we move more to a model where it
  simply observes it.  This is done by implementing an event handler
  interface with three events: OnResourceStepPre, OnResourceStepPost,
  and OnResourceComplete.  The first two are invoked immediately before
  and after any step operation, and the latter is invoked whenever a
  EndRegisterResource comes in.  The reason for the asymmetry here is
  that the checkpointing logic in the deployment engine is largely
  untouched (intentionally, as this is a sensitive part of the system),
  and so the "begin"/"end" nature doesn't flow through faithfully.

* Also make the engine more event-oriented in its terminology and the
  way it handles the incoming BeginRegisterResource and
  EndRegisterResource events from the language host.  This is the first
  step down a long road of incrementally refactoring the engine to work
  this way, a necessary prerequisite for parallelism.
2017-11-29 07:42:14 -08:00
joeduffy 7e48e8726b Add (back) component outputs
This change adds back component output properties.  Doing so
requires splitting the RPC interface for creating resources in
half, with an initial RegisterResource which contains all of the
input properties, and a final CompleteResource which optionally
contains any output properties synthesized by the component.
2017-11-20 17:38:09 -08:00
Luke Hoban 96e4b74b15
Support for stack outputs (#581)
Adds support for top-level exports in the main script of a Pulumi Program to be captured as stack-level output properties.

This create a new `pulumi:pulumi:Stack` component as the root of the resource tree in all Pulumi programs.  That resources has properties for each top-level export in the Node.js script.

Running `pulumi stack` will display the current value of these outputs.
2017-11-17 15:22:41 -08:00
CyrusNajmabadi 36a692390d
Properly capture 'arguments' when creating our serialization closure. (#569)
* Simplify how we capture 'this' in our serialization logic.
* Properly capture 'arguments'

* add tests for 'arguments' capture.

* Properly serialize out 'arguments'
* Invert 'with' and function closure.
2017-11-15 11:31:17 -08:00
CyrusNajmabadi 89b5a4be71
remove use of 'eval' in tests. (#510)
* remove use of 'eval' in tests.

* Remove another eval.

* Remove usage of eval.
2017-10-31 14:41:58 -07:00
Joe Duffy cdb2c79e8e
Exit with an error code in the face of unhandled errors (#495)
As part of fixing the exit bug recently, we accidentally made errors
lead to zero exit codes.  As a result, the Pulumi CLI thought the
prgoram exited ordinarily, and proceeded to do its usual planning and
deployment, rather than terminating abruptly.

This is a byproduct of how Node's process.uncaughtException handler
works.  It hijacks and replaces all usual error logic, including the
process.exit part.  This change simply adds back the non-zero exit.

I also added a test (and fixed one other that began failing
afterwards), so that we can prevent regressions down the road.
2017-10-28 17:05:05 -07:00
joeduffy c61bce3e41 Permit undefined in more places
The prior code was a little too aggressive in rejected undefined
properties, because it assumed any occurrence indicated a resource
that was unavailable due to planning.  This is a by-produt of our
relatively recent decision to flow undefineds freely during planning.

The problem is, it's entirely legitimate to have undefined values
deep down in JavaScript structures, entirely unrelated to resources
whose property values are unknown due to planning.

This change flows undefined more freely.  There really are no
negative consequences of doing so, and avoids hitting some overly
aggressive assertion failures in some important scenarios.  Ideally
we would have a way to know statically whether something is a resource
property, and tighten up the assertions just to catch possible bugs
in the system, but because this is JavaScript, and all the assertions
are happening at runtime, we simply lack the necessary metadata to do so.
2017-10-23 16:02:28 -07:00
Luke Hoban ba98f5e837 Fix bugs in free variable analysis (#444)
Properties and methods were not being traversed correctly.

Fixes #442.
2017-10-19 23:20:57 -07:00
joeduffy 599ca8ea43 Add accessors to fetch the Pulumi project and stack names
This change adds functions, `pulumi.getProject()` and `pulumi.getStack()`,
to fetch the names of the project and stack, respectively.  These can be
handy in generating names, specializing areas of the code, etc.

This fixes pulumi/pulumi#429.
2017-10-19 08:26:57 -07:00
CyrusNajmabadi d929c169de Enable tslinting of the nodejs sdk. (#433) 2017-10-18 15:03:56 -07:00
CyrusNajmabadi d007f040b9 Move from acorn to TypeScript as the parser we use when computing free variables. (#431) 2017-10-18 13:29:53 -07:00
joeduffy 301739c6b5 Add auto-parenting
This changes a few things about "components":

* Rename what was previously ExternalResource to CustomResource,
  and all of the related fields and parameters that this implies.
  This just seems like a much nicer and expected name for what
  these represent.  I realize I am stealing a name we had thought
  about using elsewhere, but this seems like an appropriate use.

* Introduce ComponentResource, to make initializing resources
  that merely aggregate other resources easier to do correctly.

* Add a withParent and parentScope concept to Resource, to make
  allocating children less error-prone.  Now there's no need to
  explicitly adopt children as they are allocated; instead, any
  children allocated as part of the withParent callback will
  auto-parent to the resource provided.  This is used by
  ComponentResource's initialization function to make initialization
  easier, including the distinction between inputs and outputs.
2017-10-15 04:38:26 -07:00
joeduffy fbfca58a3f Implement components
This change implements core support for "components" in the Pulumi
Fabric.  This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.

In a nutshell, resources no longer imply external providers.  It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.

For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.

To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.

All resources now have the ability to adopt children.  This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).

Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output.  For instance:

    + aws:serverless:Function: (create)
      [urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
      => urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
      => urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
      => urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda

The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 18:30:59 -07:00
CyrusNajmabadi b713990b5e Enable 'use const' linter rule. (#405)
* Enable 'use const' linter rule.
2017-10-10 14:50:55 -07:00
CyrusNajmabadi 556345c68e Include environment when computing the hash of closure. (#403)
* Include environment when computing the hash of closure.
2017-10-09 21:46:24 -07:00
Luke Hoban 4af1345d4c Move serialization of Closures to JavaScript text to pulumi SDK (#402)
This logic was previously in the `@pulumi/aws` pacakge.  Moving it into the `pulumi` SDK as part of the overall closure serialization logic to make it more broadly accessible, and to centralize this functionality.

Now that it's all in one place, we may decide to remove the publically exposed `Closure` abstraction completely, which may also enable significant simplicifcation to the logic in closure serialization.

Also add one initial test case for this code.

Fixes pulumi/pulumi-aws#14.
2017-10-07 14:29:56 -07:00
joeduffy 2e9e0d2a98 Add a simple invoke test case 2017-09-30 14:53:27 -04:00
Luke Hoban ad5ee5bc04 Support module capture without serialization (#375)
This change adds first class support for capturing objects which are references to loaded Node modules.

If an object to be serialized is found as a loaded module which can be referenced as `require(<name>)`, then is is not serialized and is passed as a new kind of environment entry - `module` which will be de-serialized as a `require` statement.

Supports three cases:
1. built-in modules such as `http` and `path`
2. dependencies in the `node_modules` folder
3. other user-defined modules in the source folder

This allows natural use of `import`s with "inside" code.  For example - note the use of `$` in the outside scope only on the "inside".

```typescript
import * as cloud from "@pulumi/cloud";
import * as $ from "cheerio";
let queue = new pulumi.Topic<string>("sites_to_process");
queue.subscribe("foreachurl", async (url) => {
    let x = $("a", "<a href='foo'>hello</a>");
});
```

Also fixes free variable capture of `this` in arrow functions.

Fixes #342.
2017-09-28 16:44:00 -07:00
Luke Hoban e65348e246 Support for destructing patterns in free variable computation (#365)
Also runs `sdk/nodejs` tests by default during build.
2017-09-25 15:01:31 -07:00
pat@pulumi.com 69341fa7c8 push is dead; long live update.
After discussion with Joe and Luke, we've decided to use `update` instead
of `push` as it more intuitively fits the operation being performed.
2017-09-22 17:23:40 -07:00
pat@pulumi.com 597db186ec Renames: plan -> preview, deploy -> push.
Part of #353.

These changes also remove all command aliases from the `pulumi` command.
2017-09-22 15:28:03 -07:00
Joe Duffy f6e694c72b Rename pulumi-fabric to pulumi
This includes a few changes:

* The repo name -- and hence the Go modules -- changes from pulumi-fabric to pulumi.

* The Node.js SDK package changes from @pulumi/pulumi-fabric to just pulumi.

* The CLI is renamed from lumi to pulumi.
2017-09-21 19:18:21 -07:00
joeduffy 1c2c972d37 Add back Computed<T> as a short-hand
This adds back Computed<T> as a short-hand for Promise<T | undefined>.
Subtly, all resource properties need to permit undefined flowing through
during planning  Rather than forcing the long-hand version, which is easy
to forget, we'll keep the convention of preferring Computed<T>.  It's
just a typedef and the runtime type is just a Promise.
2017-09-20 09:59:32 -07:00
joeduffy f8ee6c570e Eliminate Computed/Property in favor of Promises
As part of pulumi/pulumi-fabric#331, we've been exploring just using
undefined to indicate that a property value is absent during planning.
We also considered blocking the message loop to simplify the overall
programming model, so that all asynchrony is hidden.

It turns out ThereBeDragons 🐲 anytime you try to block the
message loop.  So, we aren't quite sure about that bit.

But the part we are convicted about is that this Computed/Property
model is far too complex.  Furthermore, it's very close to promises, and
yet frustratingly so far away.  Indeed, the original thinking in
pulumi/pulumi-fabric#271 was simply to use promises, but we wanted to
encourage dataflow styles, rather than control flow.  But we muddied up
our thinking by worrying about awaiting a promise that would never resolve.

It turns out we can achieve a middle ground: resolve planning promises to
undefined, so that they don't lead to hangs, but still use promises so
that asynchrony is explicit in the system.  This also avoids blocking the
message loop.  Who knows, this may actually be a fine final destination.
2017-09-20 09:59:32 -07:00
joeduffy 087deb7643 Add optional dependsOn to Resource constructors
This change adds an optiona dependsOn parameter to Resource constructors,
to "force" a fake dependency between resources.  We have an extremely strong
desire to resort to using this only in unusual cases -- and instead rely
on the natural dependency DAG based on properties -- but experience in other
resource provisioning frameworks tells us that we're likely to need this in
the general case.  Indeed, we've already encountered the need in AWS's
API Gateway resources... and I suspect we'll run into more especially as we
tackle non-serverless resources like EC2 Instances, where "ambient"
dependencies are far more commonplace.

This also makes parallelism the default mode of operation, and we have a
new --serialize flag that can be used to suppress this default behavior.
Full disclosure: I expect this to become more Make-like, i.e. -j 8, where
you can specify the precise width of parallelism, when we tackle
pulumi/pulumi-fabric#106.  I also think there's a good chance we will flip
the default, so that serial execution is the default, so that developers
who don't benefit from the parallelism don't need to worry about dependsOn
in awkward ways.  This tends to be the way most tools (like Make) operate.

This fixes pulumi/pulumi-fabric#335.
2017-09-15 16:38:52 -07:00
joeduffy 94207acf65 Upgrade to TypeScript ^2.5.2 2017-09-14 15:02:41 -07:00
joeduffy 8ce07617c9 Implement recursive closure captures
This change implements recursive closure captures.  This permits
cases like the following

    {
        function f() { g(); }
        function g() { f(); }
    }

and the slightly more useful

    class C {
        this.x = 42;
        this.f = () => x;
    }

To do this requires caching the environment objects and permitting
cycles in the resulting environment graph.  The closure emitter code
already knows how to handle this.

In addition, we must mark captures of `this` as free variables.

This resolves pulumi/pulumi-fabric#333.
2017-09-10 07:40:53 -07:00
joeduffy f9995159c6 Fix a handful of things, mostly logging
* Initialize the diganostics logger with opts.Debug when doing
  a Deploy, like we do Plan.

* Don't spew leaked promises if there were Log.errors.

* Serialize logging RPC calls so that they can't appear out of order.

* Print stack traces in more places and, in particular, remember
  the original context for any errors that may occur asynchronously,
  like resource registration and calls to mapValue.

* Include origin stack traces generally in more error messages.

* Add some more mapValue test cases.

* Only undefined-propagate mapValue values during dry-runs.
2017-09-09 13:43:51 -07:00
joeduffy aefe297aa1 Harden dependent resolutions
This fixes a few problems with dependent resolutions and hardens
even more promises-related error paths, so we swallow precisely zero
errors (or at least we hope so).  This also digs through multi-level
chains of promises and computed properties as needed for nested mapValues.
2017-09-06 14:29:17 -07:00
joeduffy 240b54b5be Add typings and tests for mapValues that return computeds 2017-09-06 08:28:11 -07:00
joeduffy 6630de503c Support capturing Computed<T>s and Promise<T>s
This change adds support for awaiting any Computed<T> and Promise<T>s
that were captured inside of a function's closure.  This preserves our
ability to capture, for example, resource state that ends up getting
serialized as the final resource state, rather than a snapshot of the
(mostly unresolved) resource state at the time of serialization.
2017-09-06 07:36:19 -07:00
joeduffy 726e48e094 Add an extra test for nested functions 2017-09-05 15:50:47 -07:00
joeduffy 3164572b6e Fix some free variable capture logic
* Use `global.hasOwnProperty(ident)`, rather than `global[ident] !== undefined`,
  to avoid classifying references to globals as free variables.  Surprise(!!),
  the prior logic wouldn't work for `undefined` itself... 😒

* Expand this check to include the built-in Node.js module variables, namely
  `__dirname`, `__filename`, `exports`, `module`, and `require`, so that
  references to them don't get classified as serializable free variables either.

* Place catch variables in scope, so that `catch (err) { ... }` won't yield
  free variables for references to `err` within `...`.

* Place recursive function definitions into the top-level `var`-like scope of
  variables so that we don't consider references to them free.

* Harden all error pathways in the native C++ add-on so that we terminate
  anytime an exception is in-flight, rather than limping along and making
  things worse...
2017-09-05 15:21:14 -07:00
joeduffy e3a6695399 Depend only on vendored protos 2017-09-05 11:52:33 -07:00
joeduffy a1ab56fc28 Prettify properties
This change makes a few simplifications to how properties are exposed in
the system, mostly in the name of usability, but also to feel a bit more
like "idiomatic JavaScript".  Namely:

* Rename `then` to `mapValue`.  This hopefully helps to suggest that this
  is meant for a dataflow style of programming.

* Move Property<T> into the runtime module, and remove PropertyState<T>,
  collapsing back down to a single type.  This also eliminates some of the
  messy internal runtime casting, accessing of internal members, etc.

* Export a Computed<T> interface from the root of the module.  This is
  the entirety of the public-facing surface area for properties, and
  exposes that single `mapValue` member function.  The internal runtime
  logic understands how to handle Property<T>s specifically in addition
  to Computed<T>s more generally (in case someone writes their own).
2017-09-05 10:55:09 -07:00
joeduffy 2a22a71116 Tidy up resource properties
This changes a few aspects of resource properties:

* Move all runtime-related goo into the runtime module, in an
  internal PromiseState class.  This encapsulates the internal
  state transitions and protects against misuse.  It also allows
  us to clean up the public API for the Property<T> type so that
  it's entirely suitable for external usage.

* Track input and output property values distinctly.  It turns
  out we want to key off events differently.  For example, to marshal
  property values to a resource provider, we only care about the
  inputs.  For final property values that are used in, say, thens
  or as inputs to other properties, we want the output property value.

* Be more precise about when an output is truly final, and known, or
  unknown due to planning/dry-runs.  Note that this does mean that
  we'll encounter unknown values more frequently because, aside from
  IDs and URNs, we can't say for sure that arbitrary properties will never
  change post-creation.  We have ideas on how to denote this; see
  pulumi/pulumi-fabric#330 for more details.
2017-09-05 09:31:03 -07:00
joeduffy b80b6afcf1 Lint the test files 2017-09-04 11:35:21 -07:00
joeduffy 7c7610848f Rename asset classes
This change renames String, File, and Remote to StringAsset, FileAsset,
and RemoteAsset, largely to avoid conflicting with the built-in JavaScript
String type, but also because it mirrors our Archive naming strategy.
2017-09-04 11:35:21 -07:00
joeduffy d7688da5e3 Fix a few minor pathing things 2017-09-04 11:35:21 -07:00
joeduffy 3427647f93 Implement free variable calculations
This change implements free variable calculations and wires it up
to closure serialization.  This is recursive, in the sense that
the serializer may need to call back to fetch free variables for
nested functions encountered during serialization.

The free variable calculation works by parsing the serialized
function text and walking the AST, applying the usual scoping rules
to determine what is free.  In particular, it respects nested
function boundaries, and rules around var, let, and const scoping.

We are using Acorn to perform the parsing.  I'd originally gone
down the path of using V8, so that we have one consistent parser
in the game, however unfortunately neither V8's parser nor its AST
is a stable API meant for 3rd parties.  Unlike the exising internal
V8 dependencies, this one got very deep very quickly, and I became
nervous about maintaining all those dependencies.  Furthermore,
by doing it this way, we can write the free variable logic in
JavaScript, which means one fewer C++ component to maintain.

This also includes a fairly significant amount of testing, all
of which passes! 🎉
2017-09-04 11:35:21 -07:00