This change fixes up a few things so that updates correctly deal
with output properties. This involves a few things:
1) All outputs stored on the pre snapshot need to get propagated
to the post snapshot during planning at various points. This
ensures that the diffing logic doesn't need to be special cased
everywhere, including both the Lumi and the provider sides.
2) Names are changed to "input" properties (using a new `lumi` tag
option, `in`). These are properties that providers are expected
to know nothing about, which we must treat with care during diffs.
3) We read back properties, via Get, after doing an Update just like
we do after performing a Create. This ensures that if an update
has a cascading impact on other properties, it will be detected.
4) Inspecting a change, prior to updating, must be done using the
computed property set instead of the real one. This is to avoid
mutating the resource objects ahead of actually applying a plan,
which would be wrong and misleading.
This change skips printing output<T> properties as we perform a
deployment, instead showing the real values inline after the resource
has been created. (output<T> is still shown during planning, of course.)
The change to flow logging to plugins is nice, however, it can be
annoying because all writes to stderr are interepreted on the Lumi
side as errors. After this change, we will only flow if
--logflow is passed, e.g. as in
$ lumi --logtostderr --logflow -v=9 deploy ...
This change prepares for integrating more planning and deployment logic
closer to the runtime itself. For historical reasons, we ended up with these
in the env.go file which really has nothing to do with deployments anymore.
This change introduces the notion of a computed versus an output
property on resources. Technically, output is a subset of computed,
however it is a special kind that we want to treat differently during
the evaluation of a deployment plan. Specifically:
* An output property is any property that is populated by the resource
provider, not code running in the Lumi type system. Because these
values aren't available during planning -- since we have not yet
performed the deployment operations -- they will be latent values in
our runtime and generally missing at the time of a plan. This is no
problem and we just want to avoid marshaling them in inopportune places.
* A computed property, on the other hand, is a different beast altogehter.
Although true one of these is missing a value -- by virtue of the fact
that they too are latent values, bottoming out in some manner on an
output property -- they will appear in serializable input positions.
Not only must we treat them differently during the RPC handshake and
in the resource providers, but we also want to guarantee they are gone
by the time we perform any CRUD operations on a resource. They are
purely a planning-time-only construct.
We need to smuggle metadata from the resource IDL all the way through
to the runtime, so that it knows which things are output properties. In
order to do this, we'll leverage decorators and the support for serializing
them as attributes. This change adds support for the various kinds
(class, property, method, and parameter), in addition to test cases.
This change includes approximately 1/3rd of the change necessary
to support output properties, as per pulumi/lumi#90.
In short, the runtime now has a new hidden type, Latent<T>, which
represents a "speculative" value, whose eventual type will be T,
that we can use during evaluation in various ways. Namely,
operations against Latent<T>s generally produce new Latent<U>s.
During planning, any Latent<T>s that end up in resource properties
are transformed into "unknown" property values. An unknown property
value is legal only during planning-time activities, such as Check,
Name, and InspectChange. As a result, those RPC interfaces have
been updated to include lookaside maps indicating which properties
have unknown values. My intent is to add some helper functions to
make dealing with this circumstance more correct-by-construction.
For now, using an unresolved Latent<T> in a conditional will lead
to an error. See pulumi/lumi#67. Speculating beyond these -- by
supporting iterative planning and application -- is something we
want to support eventually, but it makes sense to do that as an
additive change beyond this initial support. That is a missing 1/3.
Finally, the other missing 1/3rd which will happen much sooner
than the rest is restructuing plan application so that it will
correctly observe resolution of Latent<T> values. Right now, the
evaluation happens in one single pass, prior to the application, and
so Latent<T>s never actually get witnessed in a resolved state.
Adds support for global secondary indexes on DynamoDB Tables.
Also adds a HashSet API to the AWS provider library. This handles part of #178,
providing a standard way for AWS provider implementations to compute set-based
diffs. This new API is used in both aws.dynamodb.Table and aws.elasticbeanstalk.Environment
currently.
Resolves#137.
This is an initial pass for supporting JavaScript lambda syntax for defining an AWS Lambda Function.
A higher level API for defining AWS Lambda Function objects `aws.lambda.FunctionX` is added which accepts a Lumi lambda as an argument, and uses that lambda to generate the AWS Lambda Function code package.
LumiJS lambdas are serialized as the JavaScript text of the lambda body, along with a serialized version of the environment that is deserialized at runtime and used as the context for the body of the lambda.
Remaining work to further improve support for lambdas is being tracked in #173, #174, #175, and #177.
This change keeps the lumi prefix on our CLI tools.
As @lukehoban pointed out in person, as soon as we do pulumi/coconut#98,
most people (other than compiler authors themselves) won't actually be
typing the commands. And, furthermore, the commands aren't all that bad.
Eventually I assume we'll want something like `lumi-js`, or
`lumi-js-compiler`, so that binaries are discovered dynamically in a way
that is extensible for future languages. We can tackle this during #98.
This change implements property accessors (getters and setters).
The approach is fairly basic, but is heavily inspired by the ECMAScript5
approach of attaching a getter/setter to any property slot (even if we don't
yet fully exploit this capability). The evaluator then needs to track and
utilize the appropriate accessor functions when loading locations.
This change includes CocoJS support and makes a dent in pulumi/coconut#66.
This change, part of pulumi/coconut#62, adds support for ECMAScript
local functions. This leverages the recent support for lambdas.
The change also adds some new test cases for the various forms.
Here are some examples of supported forms:
function outer() {
// simple named inner function:
function inner1() { .. };
// anonymous inner function (just a lambda):
let inner2 = function() { ... };
// named and bound inner function:
let inner3 = function inner4() { ... };
}
These merely compile into lambdas that have been bound to local
variables with the appropriate names.
The previous shape of SequenceExpression only permitted expressions
in the sequence. This is pretty common in most ILs, however, it usually
leads to complicated manual spilling in the event that a statement is needed.
This is often necessary when, for example, a compiler is deeply nested in some
expression production, and then realizes the code expansion requires a
statement (e.g., maybe a new local variable must be declared, etc).
Instead of requiring complicated code-gen, this change permits SequenceExpression
to contain an arbitrary mixture of expression/statement prelude nodes, terminating
with a single, final Expression which yields the actual expression value. The
runtime bears the burden of implementing this which, frankly, is pretty trivial.
This change recognizes and emits lambdas correctly in CocoJS (as part
of pulumi/coconut#62). The existing CocoIL representation for lambdas
worked just fine for functions, lambdas, and local functions. There
still isn't runtime support, but that comes next.
I've tripped over pulumi/coconut#141 a few times now, particularly with
the sort of dynamic payloads required when creating lambdas and API gateways.
This change implements support for computed property initializers.
This change correctly implements package/module resolution in CIDLC.
For now, this only works for intra-package imports, which is sufficient
for now. Eventually we will need to support this (see pulumi/coconut#138).
* Use --out-rpc, rather than --out-provider, since rpc/ is a peer to provider/.
* Use strongly typed tokens in more places.
* Append "rpc" to the generated RPC package names to avoid conflicts.
* Change the Check function to return []mapper.FieldError, rather than
mapper.DecodeError, to make the common "no errors" case easier (and to eliminate
boilerplate resulting in needing to conditionally construct a mapper.DecodeError).
* Rename the diffs argument to just diff, matching the existing convention.
* Automatically detect changes to "replaces" properties in the PreviewUpdate
function. This eliminates tons of boilerplate in the providers and handles the
90% common case for resource recreation. It's still possible to override the
PreviewUpdate logic, of course, in case there is more sophisticated recreation
logic necessary than just whether a property changed or not.
* Add some comments on some generated types.
* Generate property constants for the names as they will appear in weakly typed
property bags. Although the new RPC interfaces are almost entirely strongly
typed, in the event that diffs must be inspected, this often devolves into using
maps and so on. It's much nicer to say `if diff.Changed(SecurityGroup_Description)`
than `if diff.Changed("description")` (and catches more errors at compile-time).
* Fix resource ID generation logic to properly fetch the Underlying() type on
named types (this would sometimes miss resources during property analysis, emitting
for example `*VPC` instead of `*resource.ID`).
This is an initial implementation of the Coconut IDL Compiler (CIDLC).
This is described further in
https://github.com/pulumi/coconut/blob/master/docs/design/idl.md,
and the work is tracked by coconut/pulumi#133.
I've been kicking the tires with this locally enough to checkpoint the
current version. There are quite a few loose ends not yet implemented,
most of them minor, with the exception of the RPC stub generation which
I need to flesh out more before committing.
This change introduces TryLoadDynamicExpression. This is similar to
the existing LoadDynamicExpression opcode, except that it will return
null in response to a missing member (versus the default of raising
an exception). This is to enable languages like JavaScript to encode
operations properly (which always yields undefined/nulls), while still
catering to languages like Python (which throw exceptions).
This change introduces decorator support for CocoJS and the corresponding
IL/AST changes to store them on definition nodes. Nothing consumes these
at the moment, however, I am looking at leveraging this to indicate that
certain program fragments are "code" and should be serialized specially
(in support of Functions-as-lambdas).
This uses %q to quote property values when printing them. This ensures
that control characters are escaped (like \n), in addition to replacing any
unprintable characters with the appropriate escape sequence. Both show up
nicer in the output for planning commands, etc.
This change introduces the scaffolding for a new cmd/cocogo tool,
as part of pulumi/coconut#133. The idea here is to do some very
rudimentary code-gen on a subset of Go, to ease the task of writing
providers. The README describes this in more detail. Eventually
this will presumably expand to being a peer language to CocoPy,
etc., in that real code can be written, but for now it's mostly IDL.
At the moment, the tool really doesn't do anything useful, other
than loading up, parsing, semantically validating, and spewing
some information about the Go packages passed at the command line.
This change restructures the overall structure for commands so that
all top-level tools are in the cmd/ directory, alongside the primary
coco command. This is more "idiomatic Go" in its layout, and makes
room for additional command line tools (like cocogo for IDL).
The old model for imports was to use top-level declarations on the
enclosing module itself. This was a laudible attempt to simplify
matters, but just doesn't work.
For one, the order of initialization doesn't precisely correspond
to the imports as they appear in the source code. This could incur
some weird module initialization problems that lead to differing
behavior between a language and its Coconut variant.
But more pressing as we work on CocoPy support, it doesn't give
us an opportunity to dynamically bind names in a correct way. For
example, "import aws" now needs to actually translate into a variable
declaration and assignment of sorts. Furthermore, that variable name
should be visible in the environment block in which it occurs.
This change switches imports to act like statements. For the most
part this doesn't change much compared to the old model. The common
pattern of declaring imports at the top of a file will translate to
the imports happening at the top of the module's initializer. This
has the effect of initializing the transitive closure just as it
happened previously. But it enables alternative models, like imports
inside of functions, and -- per the above -- dynamic name binding.
This rearranges the way dynamic loads work a bit. Previously, they¬
required an object, and did a dynamic lookup in the object's property¬
map. For real dynamic loads -- of the kind Python uses, obviously,¬
but also ECMAScript -- we need to search the "environment".
This change searches the environment by looking first in the lexical¬
scope in the current function. If a variable exists, we will use it.¬
If that misses, we then look in the module scope. If a variable exists¬
there, we will use it. Otherwise, if the variable is used in a non-lval
position, an dynamic error will be raised ("name not declared"). If
an lval, however, we will lazily allocate a slot for it.
Note that Python doesn't use block scoping in the same way that most
languages do. This behavior is simply achieved by Python not emitting
any lexically scoped blocks other than at the function level.
This doesn't perfectly achieve the scoping behavior, because we don't
yet bind every name in a way that they can be dynamically discovered.
The two obvious cases are class names and import names. Those will be
covered in a subsequent commit.
Also note that we are getting lucky here that class static/instance
variables aren't accessible in Python or ECMAScript "ambiently" like
they are in some languages (e.g., C#, Java); as a result, we don't need
to introduce a class scope in the dynamic lookup. Some day, when we
want to support such languages, we'll need to think about how to let
languages control the environment probe order; for instance, perhaps
the LoadDynamicExpression node can have an "environment" property.
We already had the ability to manually execute a CocoPack and generate
a DOT from its object graph. However, for demo purposes we also want
to be able to generate one from the plan. This adds a --dot flag to plan.
This change eliminates the need to constantly type in the environment
name when performing major commands like configuration, planning, and
deployment. It's probably due to my age, however, I keep fat-fingering
simple commands in front of investors and I am embarrassed!
In the new model, there is a notion of a "current environment", and
I have modeled it kinda sorta just like Git's notion of "current branch."
By default, the current environment is set when you `init` something.
Otherwise, there is the `coco env select <env>` command to change it.
(Running this command w/out a new <env> will show you the current one.)
The major commands `config`, `plan`, `deploy`, and `destroy` will prefer
to use the current environment, unless it is overridden by using the
--env flag. All of the `coco env <cmd> <env>` commands still require the
explicit passing of an environment which seems reasonable since they are,
after all, about manipulating environments.
As part of this, I've overhauled the aging workspace settings cruft,
which had fallen into disrepair since the initial prototype.
