09c01dd942
This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown. |
||
|---|---|---|
| cmd | ||
| docs | ||
| examples | ||
| lib | ||
| pkg | ||
| sdk | ||
| tools/mujs | ||
| .gitignore | ||
| .gitmodules | ||
| glide.lock | ||
| glide.yaml | ||
| main.go | ||
| Makefile | ||
| README.md | ||
Mu
Mu is a framework and toolset for creating reusable stacks of services.
If you are learning about Mu for the first time, please see the overview document.
Architecture
Prerequisites
Mu is written in Go and uses Glide for dependency management. They must be installed:
If you wish to use the optional lint make target, you'll also need to install Golint:
go get -u github.com/golang/lint/golint
Building and Testing
To build Mu, first clone it into a standard Go workspace:
$ mkdir -p $GOPATH/src/github.com/marapongo
$ git clone git@github.com:marapongo/mu $GOPATH/src/github.com/marapongo/mu
A good default value for GOPATH is ~/go.
Mu needs to know where to look for its runtime, library, etc. By default, it will look in /usr/local/mu, however you
can override this with the MUPATH variable. Normally it's easiest just to create a symlink:
$ ln -s $GOPATH/src/github.com/marapongo/mu /usr/local/mu
There is one additional build-time dependency, golint, which can be installed using:
$ go get -u github.com/golang/lint/golint
And placed on your path by:
$ export PATH=$PATH:$GOPATH/bin
At this point you should be able to build and run tests from the root directory:
$ cd $GOPATH/src/github.com/marapongo/mu
$ glide update
$ make
This installs the mu binary into $GOPATH/bin, which may now be run provided make exited successfully.
Compilers
The Mu compilers are built and tested independently from the Mu tool and runtime written in Go. Please see the respective pages for details on this process for each compiler:
Debugging
The Mu tools have extensive logging built in. In fact, we encourage liberal logging in new code, and addding new logging when debugging problems. This helps to ensure future debugging endeavors benefit from your sleuthing.
All logging is done using Google's Glog library. It is relatively barebones, and adds basic leveled logging, stack dumping, and other capabilities beyond what Go's built-in logging routines offer.
The Mu command line has two flags that control this logging and that can come in handy when debugging problems. The
--logtostderr flag spews directly to stderr, rather than the default of logging to files in your temp directory. And
the --verbose=n flag (-v=n for short) sets the logging level to n. Anything greater than 3 is reserved for
debug-level logging, greater than 5 is going to be quite verbose, and anything beyond 7 is extremely noisy.
For example, the command
$ mu compile blueprint.yaml --logtostderr -v=5
is a pretty standard starting point during debugging that will show a fairly comprehensive trace log of a compilation.