2017-02-25 16:25:33 +01:00
|
|
|
# Coconut Stacks
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
This document describes how Coconut Stacks and Services show up in the various [formats](formats.md) (CocoLangs,
|
2017-03-06 15:32:39 +01:00
|
|
|
CocoPack, CocoIL, and CocoGL). Those are the definitive resources on the low-level formats, but this document describes
|
2017-02-25 16:25:33 +01:00
|
|
|
the overall programming model that a developer will encounter. For more details on how this results in concrete
|
|
|
|
resources provisioned in the target cloud provider, please refer to [the cloud targeting design document](clouds.md).
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2016-11-04 19:23:59 +01:00
|
|
|
## Overview
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2017-03-06 15:32:39 +01:00
|
|
|
The following are the basic steps to creating a new CocoPack:
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
* Pick your favorite CocoLang language.
|
2017-01-02 03:06:03 +01:00
|
|
|
* Create a new project folder in your workspace.
|
2017-03-06 15:32:39 +01:00
|
|
|
* Create a Cocofile (`Coconut.yaml`) containing the top-level metadata.
|
2017-02-25 16:25:33 +01:00
|
|
|
* Install any dependencies using the `coco get` command line.
|
|
|
|
* Author stacks by subclassing the `Stack` base class in the Coconut SDK.
|
|
|
|
* Build the package using `coco build`, rinse and repeat, and then publish it.
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
For illustration purposes within this document, we shall choose Coconut's JavaScript subset, CocoJS. Please also note
|
|
|
|
that, though metadata examples are in YAML, it is generally valid to use JSON instead if preferred.
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
TODO: this document needs some good examples!
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
## Metadata
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2017-03-06 15:32:39 +01:00
|
|
|
Most CocoPacks will contain a Cocofile to help direct the compilation process. It is conventionally named
|
|
|
|
`Coconut.yaml` and is checked into the workspace.
|
2016-11-02 21:04:27 +01:00
|
|
|
|
2017-03-06 15:32:39 +01:00
|
|
|
Each Cocofile contains metadata for the package that cannot be derived from the source code. (Please refer to
|
2017-03-10 22:27:19 +01:00
|
|
|
[the CocoPack document](packages.md) for a complete listing of what metadata can appear here.) In the case that all
|
2017-03-06 15:32:39 +01:00
|
|
|
metadata can be derived from the program alone -- e.g., thanks to the use of attributes/decorators -- then the Cocofile
|
2017-02-25 16:25:33 +01:00
|
|
|
might be omitted. This is specific to your CocoLang compiler, so please consult its documentation.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
In the case of CocoJS, and most CocoLang compilers, the top-level "package manager"-like metadata -- such as name,
|
2017-01-02 03:06:03 +01:00
|
|
|
description, and so on -- must be explicitly provided; for example:
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
name: acmecorp/elk
|
|
|
|
description: A fully functioning ELK stack (Elasticsearch, Logstash, Kibana).
|
|
|
|
author: Joe Smith <joesmith@acmecorp.com>
|
|
|
|
website: https://acmecorp.github.io/elk
|
|
|
|
keywords: [ elasticsearch, logstash, kibana ]
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
In addition to basic metadata like this, any dependency packages must also be listed explicitly:
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
dependencies:
|
|
|
|
- aws/ec2#^1.0.6
|
Rename parameters to properties
The more I live with the current system, the more I prefer "properties" to
"parameters" for stacks and services. Although it is true that these things
are essentially construction-time arguments, they manifest more like properties
in the way they are used; in fact, if you think of the world in terms of primary
constructors, the distinction is pretty subtle anyway.
For example, when creating a new service, we say the following:
services:
private:
some/service:
a: 0
b: true
c: foo
This looks like a, b, and c are properties of the type some/service. If, on
the other hand, we kept calling these parameters, then you'd arguably prefer to
see the following:
services:
private:
some/service:
arguments:
a: 0
b: true
c: foo
This is a more imperative than declarative view of the world, which I dislike
(especially because it is more verbose).
Time will tell whether this is the right decision or not ...
2016-11-19 19:34:51 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
Each dependency package should consist of the following elements:
|
2016-11-29 21:36:02 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
* An optional protocol (e.g., `https://`).
|
2017-02-25 16:25:33 +01:00
|
|
|
* An optional base URL (e.g., `cocohub.com/`, `github.com/`, etc).
|
2017-01-02 03:06:03 +01:00
|
|
|
* A required namespace and/or name part (e.g., `acmecorp/worker`, `aws/s3/bucket`, etc).
|
|
|
|
* An optional `#` followed by version number (e.g., `#^1.0.6`, `#6f99088`, `#latest`, etc).
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
If protocol and base URL are absent, Coconut will default to `https://cocohub.com/`. If the version is omitted, it will
|
2017-01-02 03:06:03 +01:00
|
|
|
default to `latest`, which just means "tip"; in other words, the most recent version is used at compile-time.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
Please refer to the [dependencies design document](deps.md) for details on the format for these references in addition
|
|
|
|
to the overall package resolution process.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
TODO: it's unclear where and how [security information](security.md) should appear.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
## Defining Stacks
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
As we saw above, a stack is any subclass of Coconut's base `Stack` class:
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
export class Registry extends coconut.Stack {
|
|
|
|
private table: coconut.Table;
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
constructor() {
|
2017-02-25 16:25:33 +01:00
|
|
|
this.table = new coconut.Table("names");
|
2017-01-02 03:06:03 +01:00
|
|
|
}
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
@api public register(name: string): Promise<number> {
|
|
|
|
return this.table.insert({ name: name });
|
|
|
|
}
|
|
|
|
}
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
Any additional stacks instantiated by this stack will get transformed into CocoGL services at planning and deployment
|
2017-01-02 03:06:03 +01:00
|
|
|
time. The `Registry` above is very simple, since it doesn't accept any properties or constructor arguments, and doesn't
|
|
|
|
expose any properties of its own. But clearly each could be an interesting extension. We will see examples shortly.
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
A stack is capable of representing many different kinds of cloud "services": infrastructure, databases, containers,
|
|
|
|
event-oriented systems, web service applications, microservices, SaaS dependencies, ..., and so on. This consistency
|
|
|
|
facilitates a degree of composition and reuse, plus consistency in the way they are created, configured, and managed.
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
### Subclassing
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
A stack may subclass any other stack, specializing aspects of it as appropriate. This facilitates reuse. For instance,
|
|
|
|
perhaps my company wishes to enforce that certain best practices and standards are adhered to, for all stacks. Or
|
|
|
|
imagine someone in the community has published a best-in-breed Node.js application blueprint, leveraging Express,
|
|
|
|
MongoDB, and the ELK stack, and I merely want to plug in my own application logic and leverage the overall Stack.
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
### Capabilities
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
Any reference to a service instance is called a "capability".
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
A capability is an unforgeable reference to another running service and can be used to interact with it, either through
|
|
|
|
configuration, RPC, or otherwise. By defining interfaces in terms of capabilities, we enable a more formal way of
|
|
|
|
expressing runtime dependencies, in a way that the system can understand and leverage in its management of the system
|
|
|
|
(like ensuring services are created in the right order).
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
The more statically typed approach of using service capabilities also eliminates some of the fragility common to weakly
|
|
|
|
typed and dynamic approaches, which can be prone to race conditions, requiring manual sleeps, retries, etc.
|
2016-12-06 03:11:56 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
Capabilities can also benefit from the abstraction and encapsulation provided by Coconut. For example, imagine we want
|
|
|
|
a key/value store. The `coconut/x` namespace offers such a `KVStore` abstraction, but it is abstract. By declaring in
|
|
|
|
a constructor that we require a `KVStore`, we leave open the possibility that a caller might provide an instance of
|
|
|
|
etcd, Consul, Zookeeper, or their favorite key/value store provider.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
The references between services forms a DAG and the system topologically sorts that DAG in order to determine the order
|
2017-02-25 16:25:33 +01:00
|
|
|
in which to create and destroy services, during CocoGL planning time. There must be no cycles. Resource providers
|
|
|
|
understand liveness and health, so that developers needn't worry about races, liveness, or retries in CocoLang code.
|
2016-11-29 21:36:02 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
### Exporting Services
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
By default, services created by a stack are private implementation details of the enclosing service definition. It is
|
|
|
|
possible to export instances for public usage as a capability, however, simply by assigning them to output properties.
|
|
|
|
After constructing a service with outputs, they will be available for read access by callers.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
### RPC
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
Every stack may choose to expose protocols. These can be standard "unconstrained" network interfaces, such as "HTTP
|
|
|
|
over port 80", or can take on a more structured form, like structured RPC or REST APIs.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
The above example demonstrates this. The `@api` annotation above makes the `register` function available as an API
|
|
|
|
at runtime. This is convenient because we can simply close over `this.table` to reference it at runtime, versus the
|
|
|
|
common practice of keeping configuration and code separate, and then needing to use "loose binding" through a
|
|
|
|
combination of dynamic lookup, environment variable-based configuration, manual establishment of channels, etc.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
The benefits of declaring the full interfaces are that the RPC semantics are known to the system, facilitating advanced
|
|
|
|
management capabilities such as diagnostics, monitoring, fuzzing, and self-documentation, in addition to RPC code-
|
|
|
|
generation. This also adds a sort of "strong typing" to the connections between stacks.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
In addition to `@api`, the `@event` annotation lets a stack export an event. This event can be subscribed to and used
|
|
|
|
to schedule serverless lambda invocations, among other things. All events are subject to the same restrictions as APIs.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
All RPC functions must deal solely in terms of simple schema types on the wire, since they map to HTTP/2-based RPC and
|
|
|
|
Internet protocols. Please refer to the [RPC design document](rpc.md) for details on how this works.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
### Readonly and Perturbing Properties
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
A readonly property cannot be changed after provisioning a resource without replacing it.
|
2016-11-29 21:36:02 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
This is often used by core "infrastructure" that cannot change certain properties after creation, for example, the
|
|
|
|
data-center, virtual private cloud, or physical machine size. Although the tools allow you to change these, the mental
|
|
|
|
model is that of creating a "new" object, and wiring up all of its dependencies all over again. As a result, the
|
|
|
|
deployment process is more delicate, and may trigger a cascading recreation of many resources.
|
2016-11-29 21:36:02 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
How a readonly property is expressed is CocoLang language-specific, however for languages like CocoJS that support a
|
2017-01-02 03:06:03 +01:00
|
|
|
`readonly` property modifier, that is how it's done.
|
2016-11-29 21:36:02 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
A "perturbing" property is one that can be changed after provisioning, but doing so requires perturbing the existing
|
2016-11-29 23:29:34 +01:00
|
|
|
service in a way that may interrupt the live service. Modifying this isn't quite as impactful to the deployment process
|
|
|
|
as modifying a readonly property, however it too must be treated with care.
|
|
|
|
|
2016-11-29 21:36:02 +01:00
|
|
|
TODO(joe): CloudFormation distinguishes between three modes: update w/out interruption, update w/ interruption, and
|
|
|
|
replacement; I personally like the logical nature of readonly, however it's possible we should adopt something closer to
|
|
|
|
it: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-updating-stacks-update-behaviors.html.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2016-11-04 19:23:59 +01:00
|
|
|
## Configuration
|
|
|
|
|
|
|
|
TODO(joe): write this section.
|
2016-11-03 23:47:52 +01:00
|
|
|
|
2017-01-02 03:06:03 +01:00
|
|
|
## Common Stack Types
|
2016-11-04 19:23:59 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
Coconut offers a complete set of infrastructure stacks for each cloud provider.
|
2016-11-04 00:11:28 +01:00
|
|
|
|
2017-02-25 16:25:33 +01:00
|
|
|
Coconut also provides the `coconut/x` package, which contains a set of logical stack types, like `Container`, `Lambda`,
|
|
|
|
`Table`, `Volume`, and so on, offering a framework of higher-level, cloud-agnostic abstractions. Please consult
|
|
|
|
[the cross-cloud design document](x-cloud.md) for more details on this package and its contents.
|
2017-01-02 01:57:12 +01:00
|
|
|
|