133 lines
8.9 KiB
Markdown
133 lines
8.9 KiB
Markdown
# Coconut Resources
|
|
|
|
Coconut describes cloud resources and programs for purposes of provisioning, updating, and deleting. From the CocoPack
|
|
and CocoIL documentation, it won't yet be clear how this happens. The CocoPack and CocoIL formats are intentionally
|
|
general purpose and themselves contain no specific understanding of cloud resources. This design note fills that gap.
|
|
|
|
## Overview
|
|
|
|
Each CocoPack contains program logic that encodes program logic and data. This data comes in the form of the usual
|
|
programming language runtime constructs, namely primitive data (numbers, strings, etc) and objects. For example:
|
|
|
|
let bucket = new s3.Bucket("pictures", { deletionPolicy: "retain" });
|
|
|
|
The evaluation of an executable package, and the resulting runtime objects and data, drives the creation of a CocoGL
|
|
resource graph. This is a directed acyclic graph (DAG) whose nodes represent resources and whose edges represent
|
|
resource dependencies. From this graph, the Coconut toolchain can determine how to provision the desired resources.
|
|
|
|
The question is, how does it "provision" them?
|
|
|
|
## What is a Resource, Anyway
|
|
|
|
First thing's first. For this to work, we must first identify what a "resource" is.
|
|
|
|
A resource in the programming model is simply identified by any class that derives from the `coconut.Resource` base
|
|
class. (The specific manifestation of this depends on your language SDK; examples below are in CocoJS.) This class
|
|
is primarily a marker class in that it doesn't offer much on its own (except for a few advanced methods, see below).
|
|
|
|
import * as coconut from "@coconut/coconut";
|
|
class MyResouce extends coconut.Resource {
|
|
constructor(name: string, ...) {
|
|
super(name);
|
|
// ...
|
|
}
|
|
// ...
|
|
}
|
|
|
|
The only property that `Resource` demands is a "name". This is a friendly name that is used to create resource URNs.
|
|
|
|
To create a CocoGL graph, Coconut evaluates the program. As it does so, the Coconut runtime monitors object creation
|
|
and property assignments. The runtime recognizes which of those objects are resources and records associated activity.
|
|
|
|
Each resource may have any number of properties. Some properties may form dependencies on other resources, either by
|
|
storing a reference to another resource object, or by allocating one (directly or indirectly). These dependencies are
|
|
ultimately used to form the CocoGL DAG. In some rare circumstances, such as dynamic name-based dependencies -- a design
|
|
pattern not used by Coconut but by other cloud providers -- these dependencies may need to be registered explicitly.
|
|
|
|
Some properties may not have values until the resulting CocoGL graph is actually applied to a target environment; these
|
|
are called *provisioned properties* and will receive default values from the resource provider in some manner, possibly
|
|
by contacting the target cloud provider (e.g., default VM sizes, etc). It is possible to use these properties, however,
|
|
we must tread with care. A provisioned property's value cannot be known during planning. Any conditional logic that
|
|
depends upon a provisioned property would mean an inability to plan, and so is rejected by default.
|
|
|
|
Most of the magic of a true resource is available in its *resource provider* plugin. This provider implements create,
|
|
read, update, and delete (CRUD) operations for that resource type, in addition to some optional facilities, such as
|
|
logging and operational performance counters. These plugins are registered and loaded dynamically (see more below).
|
|
|
|
## Resource Names, IDs, and URNs
|
|
|
|
Each resource has two unique IDs associated with it:
|
|
|
|
1. A provider-assigned ID that is generally opaque to Coconut.
|
|
2. A globally-unique Coconut-assigned
|
|
[Uniform Resource Name (URN)](https://en.wikipedia.org/wiki/Uniform_Resource_Name).
|
|
|
|
As an example of the provider ID, AWS resources will have [ARNs](
|
|
http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html). For example, an AWS EC2 instance may have
|
|
a provider ID of `arn:aws:ec2:us-west-1:123456789012:instance/i-0c5192a1d67810e1a` assigned to it. Its Coconut URN, on
|
|
the other hand, might be `urn:coconut:prod/acmecorp::acmeinfra::index::aws:ec2/instance:Instance::master-node`. The
|
|
provider ID is a physical moniker while the Coconut URN is a logical, more stable, moniker.
|
|
|
|
Naming is trickier than it may seem. Coconut must be able to correlate nodes in the "new" shape of a program's
|
|
resource graph with nodes in the "old" shape, for purposes of reads, updates, and deletes. (And so Coconut doesn't
|
|
erroneously delete and recreate resources due to simple code refactorings.) However, a Coconut program may be updated
|
|
after its initial provisioning, altering the graph's shape in a way that confuses automatic naming and correlation.
|
|
|
|
As a result, we wish the Coconut URN to be reasonably stable in the face of refactoring. This is done by concatenating
|
|
aspects of the scope in which a resource object is allocated. As you can see above, there are many parts to one:
|
|
|
|
* The standard URN prefix with Coconut namespace: `urn:coconut`.
|
|
* A series of tokens in the URN name, delimited by the `::` sequence of characters:
|
|
- The stack name: `prod/acemcorp`.
|
|
- The executable package responsible for creating this stack: `acmeinfra`.
|
|
- The module within this package that created the resource: `index`.
|
|
- The type of resource: `aws:ec2/instance:Instance`.
|
|
- The friendly name of the resource assigned in the package source code: `master-node`.
|
|
|
|
Despite this relatively specific context, it is possible for the same logical resource's name to change due to simple
|
|
refactoring. For instance, if we move a resource allocation from inside of one module and into another one, its name
|
|
will change. It is difficult for Coconut to automatically understand such situations; that said, there is a
|
|
`coco rename <old> <new>` command that will rename an old URN to a new one and move all old references accordingly. The
|
|
deployment command also attempts to recognize and suggest potential moves, although it won't perform them automatically.
|
|
|
|
## Resource Providers and Extensibility
|
|
|
|
Each resource provider plugin corresponds to a single package and handles one or more resource types. The rule for
|
|
loading a plugin is quite simple: a binary of the name `coco-resource-<pkg>` is loaded, either from the path, or from
|
|
one of [the standard installation locations](deps.md). `<pkg>` is the package token with any `/`s replaced with `_`s.
|
|
|
|
There is no requirement around a resource provider, other than that it implement a specific HTTP/2 protocol. This
|
|
protocol is described by a [set of gRPC interfaces](
|
|
https://github.com/pulumi/coconut/blob/master/sdk/proto/provider.proto). In particular:
|
|
|
|
service ResourceProvider {
|
|
// Check validates that the given property bag is valid for a resource of the given type.
|
|
rpc Check(CheckRequest) returns (CheckResponse) {}
|
|
// Name names a given resource. Sometimes this will be assigned by a developer, and so the provider
|
|
// simply fetches it from the property bag; other times, the provider will assign this based on its own
|
|
// algorithm. In any case, resources with the same name must be safe to use interchangeably with one another.
|
|
rpc Name(NameRequest) returns (NameResponse) {}
|
|
// Create allocates a new instance of the provided resource and returns its unique ID afterwards. (The input ID
|
|
// must be blank.) If this call fails, the resource must not have been created (i.e., it is "transacational").
|
|
rpc Create(CreateRequest) returns (CreateResponse) {}
|
|
// Read reads the instance state identified by ID, returning a resource object, or an error if not found.
|
|
rpc Read(ReadRequest) returns (ReadResponse) {}
|
|
// Update updates an existing resource with new values.
|
|
rpc Update(UpdateRequest) returns (google.protobuf.Empty) {}
|
|
// UpdateImpact checks what impacts a hypothetical update will have on the resource's properties.
|
|
rpc UpdateImpact(UpdateRequest) returns (UpdateImpactResponse) {}
|
|
// Delete tears down an existing resource with the ID. If it fails, the resource is assumed to still exist.
|
|
rpc Delete(DeleteRequest) returns (google.protobuf.Empty) {}
|
|
}
|
|
|
|
Describing the full RPC interface here is outside of the scope of this document. However, all existing resources are
|
|
addressed by the resource type token plus a provider ID. All properties are encoded as dynamically typed bags.
|
|
|
|
Eventually, we envision CocoPacks can contain both the CocoPack/CocoIL descriptions of resources, plus the associated
|
|
resource provider logic, alongside one another and in a consistent language.
|
|
|
|
There is no reason the two need to be the same, although undoubtedly that will be the most convenient approach. Due to
|
|
the decoupling afforded by the gRPC interface, however, the toolchain takes no stance on this topic. As such, it is
|
|
common in today's code to author resource providers in Go, where the full Coconut SDK is available to them.
|
|
|