This change switches from child lists to parent pointers, in the
way resource ancestries are represented. This cleans up a fair bit
of the old parenting logic, including all notion of ambient parent
scopes (and will notably address pulumi/pulumi#435).
This lets us show a more parent/child display in the output when
doing planning and updating. For instance, here is an update of
a lambda's text, which is logically part of a cloud timer:
* cloud:timer:Timer: (same)
[urn=urn:pulumi:malta::lm-cloud:☁️timer:Timer::lm-cts-malta-job-CleanSnapshots]
* cloud:function:Function: (same)
[urn=urn:pulumi:malta::lm-cloud:☁️function:Function::lm-cts-malta-job-CleanSnapshots]
* aws:serverless:Function: (same)
[urn=urn:pulumi:malta::lm-cloud::aws:serverless:Function::lm-cts-malta-job-CleanSnapshots]
~ aws:lambda/function:Function: (modify)
[id=lm-cts-malta-job-CleanSnapshots-fee4f3bf41280741]
[urn=urn:pulumi:malta::lm-cloud::aws:lambda/function:Function::lm-cts-malta-job-CleanSnapshots]
- code : archive(assets:2092f44) {
// etc etc etc
Note that we still get walls of text, but this will be actually
quite nice when combined with pulumi/pulumi#454.
I've also suppressed printing properties that didn't change during
updates when --detailed was not passed, and also suppressed empty
strings and zero-length arrays (since TF uses these as defaults in
many places and it just makes creation and deletion quite verbose).
Note that this is a far cry from everything we can possibly do
here as part of pulumi/pulumi#340 (and even pulumi/pulumi#417).
But it's a good start towards taming some of our output spew.
Adds support for top-level exports in the main script of a Pulumi Program to be captured as stack-level output properties.
This create a new `pulumi:pulumi:Stack` component as the root of the resource tree in all Pulumi programs. That resources has properties for each top-level export in the Node.js script.
Running `pulumi stack` will display the current value of these outputs.
* Simplify how we capture 'this' in our serialization logic.
* Properly capture 'arguments'
* add tests for 'arguments' capture.
* Properly serialize out 'arguments'
* Invert 'with' and function closure.
Adds OpenTracing in the Pulumi engine and plugin + langhost subprocesses.
We currently create a single root span for any `Enging.plan` operation - which is a single `preview`, `update`, `destroy`, etc.
The only sub-spans we currently create are at gRPC boundaries, both on the client and server sides and on both the langhost and provider plugin interfaces.
We could extend this to include spans for any other semantically meaningful sections of compute inside the engine, though initial examples show we get pretty good granularity of coverage by focusing on the gRPC boundaries.
In the future, this should be easily extensible to HTTP boundaries and to track other bulky I/O like datastore read/writes once we hook up to the PPC and Pulumi Cloud.
We expose a `--trace <endpoint>` option to enable tracing on the CLI, which we will aim to thread through to subprocesses.
We currently support sending tracing data to a Zipkin-compatible endpoint. This has been validated with both Zipkin and Jaeger UIs.
We do not yet have any tracing inside the TypeScript side of the JS langhost RPC interface. There is not yet automatic gRPC OpenTracing instrumentation (though it looks like it's in progress now) - so we would need to manually create meaningful spans on that side of the interface.
This change remembers that we failed due to an uncaught exception,
and defers the process.exit(1) until we actually reach the process's
exit event. This ensures that we drain the message queue before
exiting, which ensures that outbound messages actually reach their
destination.
As part of fixing the exit bug recently, we accidentally made errors
lead to zero exit codes. As a result, the Pulumi CLI thought the
prgoram exited ordinarily, and proceeded to do its usual planning and
deployment, rather than terminating abruptly.
This is a byproduct of how Node's process.uncaughtException handler
works. It hijacks and replaces all usual error logic, including the
process.exit part. This change simply adds back the non-zero exit.
I also added a test (and fixed one other that began failing
afterwards), so that we can prevent regressions down the road.
The `nodejs` language support is implemented as two programs: one that
manages the initial connection to the engine and provides the language
serivce itself, and another that the language service invokes in order
to run a `nodejs` Pulumi program. The latter is responsible for running
the user's program and communicating its resource requests to the
engine. Currently, `run` effectively assumes that the user's program
will run synchronously from start to finish, and will disconnect from
the engine once the user's program has completed. This assumption breaks
if the user's program requires multiple turns of the event loop to
finish its root resource requests. For example, the following program
would fail to create its second resource because the engine will be
disconnected once it reaches its `await`:
```
(async () => {
let a = new Resource();
await somePromise();
let = new Resource();
})();
```
These changes fix this issue by disconnecting from the engine during
process shutdown rather than after the user's program has finished its
first turn through the event loop.
The prior code was a little too aggressive in rejected undefined
properties, because it assumed any occurrence indicated a resource
that was unavailable due to planning. This is a by-produt of our
relatively recent decision to flow undefineds freely during planning.
The problem is, it's entirely legitimate to have undefined values
deep down in JavaScript structures, entirely unrelated to resources
whose property values are unknown due to planning.
This change flows undefined more freely. There really are no
negative consequences of doing so, and avoids hitting some overly
aggressive assertion failures in some important scenarios. Ideally
we would have a way to know statically whether something is a resource
property, and tighten up the assertions just to catch possible bugs
in the system, but because this is JavaScript, and all the assertions
are happening at runtime, we simply lack the necessary metadata to do so.
This change adds functions, `pulumi.getProject()` and `pulumi.getStack()`,
to fetch the names of the project and stack, respectively. These can be
handy in generating names, specializing areas of the code, etc.
This fixespulumi/pulumi#429.
This changes a few things about "components":
* Rename what was previously ExternalResource to CustomResource,
and all of the related fields and parameters that this implies.
This just seems like a much nicer and expected name for what
these represent. I realize I am stealing a name we had thought
about using elsewhere, but this seems like an appropriate use.
* Introduce ComponentResource, to make initializing resources
that merely aggregate other resources easier to do correctly.
* Add a withParent and parentScope concept to Resource, to make
allocating children less error-prone. Now there's no need to
explicitly adopt children as they are allocated; instead, any
children allocated as part of the withParent callback will
auto-parent to the resource provided. This is used by
ComponentResource's initialization function to make initialization
easier, including the distinction between inputs and outputs.
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
This change adds environment variable fallbacks for configuration
variables, such that you can either set them explicitly, as a specific
variable PULUMI_CONFIG_<K>, or an entire JSON serialized bag via
PULUMI_CONFIG.
This is convenient when simply invoking programs at the command line,
via node, e.g.
PULUMI_CONFIG_AWS_CONFIG_REGION=us-west-2 node bin/index.js
Our language host also now uses this to communicate config when invoking
a Run RPC, rather than at the command line. This fixespulumi/pulumi#336.
This resource provider accepts a single configuration parameter, `testing:provider:module`, that is the path to a Javascript module that implements CRUD operations for a set of resource types. This allows e.g. a test case to provide its own implementation of these operations that may succeed or fail in interesting ways.
Fixes#338.
This exposes the existing runtime logging functionality in a way meant
for 3rd-parties to consume. This can be useful if we want to introduce
debug logging, warnings, or other things, that fit nicely with the
Pulumi CLI and overall developer workflow.
This logic was previously in the `@pulumi/aws` pacakge. Moving it into the `pulumi` SDK as part of the overall closure serialization logic to make it more broadly accessible, and to centralize this functionality.
Now that it's all in one place, we may decide to remove the publically exposed `Closure` abstraction completely, which may also enable significant simplicifcation to the logic in closure serialization.
Also add one initial test case for this code.
Fixespulumi/pulumi-aws#14.
This change adds the capability for a resource provider to indicate
that, where an action carried out in response to a diff, a certain set
of properties would be "stable"; that is to say, they are guaranteed
not to change. As a result, properties may be resolved to their final
values during previewing, avoiding erroneous cascading impacts.
This avoids the ever-annoying situation I keep running into when demoing:
when adding or removing an ingress rule to a security group, we ripple
the impact through the instance, and claim it must be replaced, because
that instance depends on the security group via its name. Well, the name
is a great example of a stable property, in that it will never change, and
so this is truly unfortunate and always adds uncertainty into the demos.
Particularly since the actual update doesn't need to perform replacements.
This resolvespulumi/pulumi#330.
This wires up the Node.js SDK to the newly added Invoke function
on the resource monitor and provider gRPC interfaces, letting us
expose functions that are implemented by the providers to user code.
This change adds first class support for capturing objects which are references to loaded Node modules.
If an object to be serialized is found as a loaded module which can be referenced as `require(<name>)`, then is is not serialized and is passed as a new kind of environment entry - `module` which will be de-serialized as a `require` statement.
Supports three cases:
1. built-in modules such as `http` and `path`
2. dependencies in the `node_modules` folder
3. other user-defined modules in the source folder
This allows natural use of `import`s with "inside" code. For example - note the use of `$` in the outside scope only on the "inside".
```typescript
import * as cloud from "@pulumi/cloud";
import * as $ from "cheerio";
let queue = new pulumi.Topic<string>("sites_to_process");
queue.subscribe("foreachurl", async (url) => {
let x = $("a", "<a href='foo'>hello</a>");
});
```
Also fixes free variable capture of `this` in arrow functions.
Fixes#342.
This includes a few changes:
* The repo name -- and hence the Go modules -- changes from pulumi-fabric to pulumi.
* The Node.js SDK package changes from @pulumi/pulumi-fabric to just pulumi.
* The CLI is renamed from lumi to pulumi.
There were two problems:
- node-gyp configure was failing because of different shell syntax
between windows and *nix.
- MSVC 2015 is not smart enough to understand our use of strlen actually
results in a constant value and prevents us from using it to create an
array, move to a macro based solution.
This adds back Computed<T> as a short-hand for Promise<T | undefined>.
Subtly, all resource properties need to permit undefined flowing through
during planning Rather than forcing the long-hand version, which is easy
to forget, we'll keep the convention of preferring Computed<T>. It's
just a typedef and the runtime type is just a Promise.
As part of pulumi/pulumi-fabric#331, we've been exploring just using
undefined to indicate that a property value is absent during planning.
We also considered blocking the message loop to simplify the overall
programming model, so that all asynchrony is hidden.
It turns out ThereBeDragons 🐲 anytime you try to block the
message loop. So, we aren't quite sure about that bit.
But the part we are convicted about is that this Computed/Property
model is far too complex. Furthermore, it's very close to promises, and
yet frustratingly so far away. Indeed, the original thinking in
pulumi/pulumi-fabric#271 was simply to use promises, but we wanted to
encourage dataflow styles, rather than control flow. But we muddied up
our thinking by worrying about awaiting a promise that would never resolve.
It turns out we can achieve a middle ground: resolve planning promises to
undefined, so that they don't lead to hangs, but still use promises so
that asynchrony is explicit in the system. This also avoids blocking the
message loop. Who knows, this may actually be a fine final destination.
This change flips the polarity on parallelism: rather than having a
--serialize flag, we will have a --parallel=P flag, and by default
we will shut off parallelism. We aren't benefiting from it at the
moment (until we implement pulumi/pulumi-fabric#106), and there are
more hidden dependencies in places like AWS Lambdas and Permissions
than I had realized. We may revisit the default, but this allows
us to bite off the messiness of dependsOn only when we benefit from
it. And in any case, the --parallel=P capability will be useful.
This change adds an optiona dependsOn parameter to Resource constructors,
to "force" a fake dependency between resources. We have an extremely strong
desire to resort to using this only in unusual cases -- and instead rely
on the natural dependency DAG based on properties -- but experience in other
resource provisioning frameworks tells us that we're likely to need this in
the general case. Indeed, we've already encountered the need in AWS's
API Gateway resources... and I suspect we'll run into more especially as we
tackle non-serverless resources like EC2 Instances, where "ambient"
dependencies are far more commonplace.
This also makes parallelism the default mode of operation, and we have a
new --serialize flag that can be used to suppress this default behavior.
Full disclosure: I expect this to become more Make-like, i.e. -j 8, where
you can specify the precise width of parallelism, when we tackle
pulumi/pulumi-fabric#106. I also think there's a good chance we will flip
the default, so that serial execution is the default, so that developers
who don't benefit from the parallelism don't need to worry about dependsOn
in awkward ways. This tends to be the way most tools (like Make) operate.
This fixespulumi/pulumi-fabric#335.
This change implements recursive closure captures. This permits
cases like the following
{
function f() { g(); }
function g() { f(); }
}
and the slightly more useful
class C {
this.x = 42;
this.f = () => x;
}
To do this requires caching the environment objects and permitting
cycles in the resulting environment graph. The closure emitter code
already knows how to handle this.
In addition, we must mark captures of `this` as free variables.
This resolvespulumi/pulumi-fabric#333.
This ensures RPC channels stay alive until logs finish. It also
makes provisions for logs that come in *after* shutdown has begun,
but before it has finished, by observing that the keepalive promise
has changed between the time of initiating the callback and running it.
* Initialize the diganostics logger with opts.Debug when doing
a Deploy, like we do Plan.
* Don't spew leaked promises if there were Log.errors.
* Serialize logging RPC calls so that they can't appear out of order.
* Print stack traces in more places and, in particular, remember
the original context for any errors that may occur asynchronously,
like resource registration and calls to mapValue.
* Include origin stack traces generally in more error messages.
* Add some more mapValue test cases.
* Only undefined-propagate mapValue values during dry-runs.
