These changes introduce a new field, `Raw`, to `diag.Message`. This
field indicates that the contents of the message are not a format string
and should not be rendered via `Sprintf` during stringification.
The plugin std{out,err} readers have been updated to use raw messages,
and the event reader in `pulumi` has been fixed s.t. it does not format
event payloads before display.
Fixes#551.
I noticed in our Docker builds, we often end up seeing %(MISSING)!
style messages, which were an indication we were trying to format
them. The reason was the presence of %c's in the stream, and the
fact that we passed said messages to Fprintf. We were careful in
all other layers to use the message on the "right hand side" of
any *f calls, but in this instance, we used Fprintf and passed the
message on the "left hand side", triggering formatting. It turns
out we've already formatted everything by the time we get here,
so there's no need -- we can just use Fprint instead.
Previously, the engine would write to io.Writer's to display output.
When hosted in `pulumi` these writers were tied to os.Stdout and
os.Stderr, but other applications hosting the engine could send them
other places (e.g. a log to be sent to an another application later).
While much better than just using the ambient streams, this was still
not the best. It would be ideal if the engine could just emit strongly
typed events and whatever is hosting the engine could care about
displaying them.
As a first step down that road, we move to a model where operations on
the engine now take a `chan engine.Event` and during the course of the
operation, events are written to this channel. It is the
responsibility of the caller of the method to read from the channel
until it is closed (singifying that the operation is complete).
The events we do emit are still intermingle presentation with data,
which is unfortunate, but can be improved over time. Most of the
events today are just colorized in the client and printed to stdout or
stderr without much thought.
This change improves our output formatting by generally adding
fewer prefixes. As shown in pulumi/pulumi#359, we were being
excessively verbose in many places, including prefixing every
console.out with "langhost[nodejs].stdout: ", displaying full
stack traces for simple errors like missing configuration, etc.
Overall, this change includes the following:
* Don't prefix stdout and stderr output from the program, other
than the standard "info:" prefix. I experimented with various
schemes here, but they all felt gratuitous. Simply emitting
the output seems fine, especially as it's closer to what would
happen if you just ran the program under node.
* Do NOT make writes to stderr fail the plan/deploy. Previously
we assumed that any console.errors, for instance, meant that
the overall program should fail. This simply isn't how stderr
is treated generally and meant you couldn't use certain
logging techniques and libraries, among other things.
* Do make sure that stderr writes in the program end up going to
stderr in the Pulumi CLI output, however, so that redirection
works as it should. This required a new Infoerr log level.
* Make a small fix to the planning logic so we don't attempt to
print the summary if an error occurs.
* Finally, add a new error type, RunError, that when thrown and
uncaught does not result in a full stack trace being printed.
Anyone can use this, however, we currently use it for config
errors so that we can terminate with a pretty error message,
rather than the monstrosity shown in pulumi/pulumi#359.
This includes a few changes:
* The repo name -- and hence the Go modules -- changes from pulumi-fabric to pulumi.
* The Node.js SDK package changes from @pulumi/pulumi-fabric to just pulumi.
* The CLI is renamed from lumi to pulumi.
This refactors the engine so all of the APIs on it are instance
methods on the type instead of raw methods that float around and use
data from a global engine.
A mechcanical change as we remove the global `E` and then make
anything that interacted with that in pkg/engine to be an instance
method and the dealing with the fallout.
We are renaming Lumi to Pulumi Fabric. This change simply renames the
pulumi/lumi repo to pulumi/pulumi-fabric, without the CLI tools and other
changes that will follow soon afterwards.
This change introduces a --debug option to the plan, deploy, and
destroy commands. Unlike --logtostderr, which merely hooks into the
copious Glogging that we perform (and is therefore meant for developers
of the tools themselves and not end users), --debug hooks into the
user-facing debug stream. This now includes any debug messages coming
from the resource providers as they perform their tasks.
This change adds an engine gRPC interface, and associated implementation,
so that plugins may do interesting things that require "phoning home".
Previously, the engine would fire up plugins and talk to them directly,
but there was no way for a plugin to ask the engine to do anything.
The motivation here is so that plugins can read evaluator state, such
as config information, but this change also allows richer logging
functionality than previously possible. We will still auto-log any
stdout/stderr writes; however, explicit errors, warnings, informational,
and even debug messages may be written over the Log API.
This change does a few things:
* First and foremost, it tracks configuration variables that are
initialized, and optionally prints them out as part of the
prelude/header (based on --show-config), both in a dry-run (plan)
and in an actual deployment (apply).
* It tidies up some of the colorization and messages, and includes
nice banners like "Deploying changes:", etc.
* Fix an assertion.
* Issue a new error
"One or more errors occurred while applying X's configuration"
just to make it easier to distinguish configuration-specific
failures from ordinary ones.
* Change config keys to tokens.Token, not tokens.ModuleMember,
since it is legal for keys to represent class members (statics).
This change introduces a new informational message category to the
overall diagnostics infrastructure, and then wires up the resource
provider plugins stdout/stderr streams to it. In particular, a
write to stdout implies an informational message, whereas a write to
stderr implies an error. This is just a very simple and convenient
way for plugins to provide progress reporting; eventually we may
need something more complex, due to parallel evaluation of resource
graphs, however I hope we don't have to deviate too much from this.
Per Eric's suggestion, this moves the colors we use into a central
location so that it'll be easier someday down the road to reconfigure
and/or disable them, etc. This does not include a --no-colors option
although we should really include this soon before it gets too hairy.
This change introduces a new package, pkg/resource, that will form
the foundation for actually performing deployment plans and applications.
It contains the following key abstractions:
* resource.Provider is a wrapper around the CRUD operations exposed by
underlying resource plugins. It will eventually defer to resource.Plugin,
which itself defers -- over an RPC interface -- to the actual plugin, one
per package exposing resources. The provider will also understand how to
load, cache, and overall manage the lifetime of each plugin.
* resource.Resource is the actual resource object. This is created from
the overall evaluation object graph, but is simplified. It contains only
serializable properties, for example. Inter-resource references are
translated into serializable monikers as part of creating the resource.
* resource.Moniker is a serializable string that uniquely identifies
a resource in the Mu system. This is in contrast to resource IDs, which
are generated by resource providers and generally opaque to the Mu
system. See marapongo/mu#69 for more information about monikers and some
of their challenges (namely, designing a stable algorithm).
* resource.Snapshot is a "snapshot" taken from a graph of resources. This
is a transitive closure of state representing one possible configuration
of a given environment. This is what plans are created from. Eventually,
two snapshots will be diffable, in order to perform incremental updates.
One way of thinking about this is that a snapshot of the old world's state
is advanced, one step at a time, until it reaches a desired snapshot of
the new world's state.
* resource.Plan is a plan for carrying out desired CRUD operations on a target
environment. Each plan consists of zero-to-many Steps, each of which has
a CRUD operation type, a resource target, and a next step. This is an
enumerator because it is possible the plan will evolve -- and introduce new
steps -- as it is carried out (hence, the Next() method). At the moment, this
is linearized; eventually, we want to make this more "graph-like" so that we
can exploit available parallelism within the dependencies.
There are tons of TODOs remaining. However, the `mu plan` command is functioning
with these new changes -- including colorization FTW -- so I'm landing it now.
This is part of marapongo/mu#38 and marapongo/mu#41.
* Add a TODO as a reminder to implement number toString formatting.
* Change the Loreley delimiters to something obscure ("<{%" and "%}>")
to avoid conflicting with actual characters we might use in messages.
Also, make the assertions more descriptive should Loreley fail.
* Rip out a debug.PrintStack() in verbose logging.
* Check the underlying pointer's object type for +=, not the pointer type.
This change implements stack traces. This is primarily so that we
can print out a full stack trace in the face of an unhandled exception,
and is done simply by recording the full trace during evaluation
alongside the existing local variable scopes.
This change begins to lay the groundwork for doing semantic analysis and
lowering to the cloud target's representation. In particular:
* Split the mu/schema package. There is now mu/ast which contains the
core types and mu/encoding which concerns itself with JSON and YAML
serialization.
* Notably I am *not* yet introducing a second AST form. Instead, we will
keep the parse tree and AST unified for the time being. I envision very
little difference between them -- at least for now -- and so this keeps
things simpler, at the expense of two downsides: 1) the trees will be
mutable (which turns out to be a good thing for performance), and 2) some
fields will need to be ignored during de/serialization. We can always
revisit this later when and if the need to split them arises.
* Add a binder phase. It is currently a no-op.
This change adds a few more compiler tests and rearranges some bits and pieces
that came up while doing so. For example, we now issue warnings for incorrect
casing and/or extensions of the Mufile (and test these conditions). As part of
doing that, it became clear the layering between the mu/compiler and mu/workspace
packages wasn't quite right, so some logic got moved around; additionally, the
separation of concerns between mu/workspace and mu/schema wasn't quite right, so
this has been fixed also (workspace just understands Mufile related things while
schema understands how to unmarshal the specific supported extensions).
This change adds a compiler test that just checks the basic "Mufile is missing"
error checking. The test itself is mostly uninteresting; what's more interesting
is the addition of some basic helper functionality that can be used for future
compiler tests, like capturing of compiler diagnostics for comparisons.
Error messages could get quite lengthy as the code was written previously,
because we always used the complete absolute path for the file in question.
This change "prettifies" this to be relative to whatever contextual path
the user has chosen during compilation. This shortens messages considerably.
This change includes some progress on actual compilation (albeit with several
TODOs remaining before we can actually spit out a useful artifact). There are
also some general cleanups sprinkled throughout. In a nutshell:
* Add a compiler.Context object that will be available during template expansion.
* Introduce a diag.Document abstraction. This is better than passing raw filenames
around, and lets us embellish diagnostics as we go. In particular, we will be
in a better position to provide line/column error information.
* Move IO out of the Parser and into the Compiler, where it can cache and reuse
Documents. This will become important as we start to load up dependencies.
* Rename PosRange to Location. This reads nicer with the new Document terminology.
* Rename the mu/api package to mu/schema. It's likely we will need to introduce a
true AST that is decoupled from the serialization format and contains bound nodes.
As a result, treating the existing types as "schema" is more honest.
* Add in a big section of TODOs at the end of the compiler.Compiler.Build function.
* Rename Meta to Metadata.
* Rename Target's CloudOS and CloudScheduler properties to Cloud
and Scheduler, respectively. Also rename Target's JSON properties
to match (they had drifted); they are now "cloud" and "scheduler".
* Rename Diags() to Diag() on the Compiler and Parser interfaces.
* Rename defaultDiags to defaultSink, to match the interface name.
* Add a few useful logging outputs.
This adds a bunch of general scaffolding and the beginning of a `build` command.
The general engineering scaffolding includes:
* Glide for dependency management.
* A Makefile that runs govet and golint during builds.
* Google's Glog library for logging.
* Cobra for command line functionality.
The Mu-specific scaffolding includes some packages:
* mu/pkg/diag: A package for compiler-like diagnostics. It's fairly barebones
at the moment, however we can embellish this over time.
* mu/pkg/errors: A package containing Mu's predefined set of errors.
* mu/pkg/workspace: A package containing workspace-related convenience helpers.
in addition to a main entrypoint that simply wires up and invokes the CLI. From
there, the mu/cmd package takes over, with the Cobra-defined CLI commands.
Finally, the mu/pkg/compiler package actually implements the compiler behavior.
Or, it will. For now, it simply parses a JSON or YAML Mufile into the core
mu/pkg/api types, and prints out the result.