pulumi/docs/overview.md

Mu

The core concepts in Mu are:

1. Stack: A blueprint that describes a topology of cloud resources.
2. Service: A grouping of stateless or stateful logic with an optional API.
3. Function: A single stateless function that is unbundled with a single "API": invoke.
4. Trigger: A subscription that calls a Service or Function in response to an event.

// TODO(joe): map to Kube concepts; do we need "more" (e.g., Controller)?

Each Stack "instantiates" one or more Services, Functions, and Triggers to create cloud functionality. This can include databases, queues, containers, pub/sub topics, and overall container-based microservices, to name a few examples. These constructs compose nicely, such that a Service may create a Stack if it wishes to encapsulate its own resource needs.

A Service may be stateless or stateful depending on the scenario's state and scale requirements. Multiple kinds of Services exist and may be backed by different physical facilities: Docker containers, VMs, AWS Lambdas, and/or cloud hosted SaaS services, to name a few. The programming model remains consistent across them. A Service may export APIs for RPC-based consumption by other Services or even exported as an HTTP/2 endpoint for external consumption.

A Function is actually just a special kind of Service, however they feature prominently enough to call them out as a top-level construct in the system. Many of the same policies that apply to stateless Services also apply to Functions.

A rich ecosystem of Trigger events exists so that you can write reactive, serverless code where convenient without managing whole Services. This includes the standard ones -- like CRUD operations in your favorite NoSQL database -- in addition to more novel ones -- like SalesForce customer events -- to deliver a uniform event-driven programming model.

Here is a brief example of Stack that represents a voting service, authored in Node.js:

var mu = require("mu");

var votes = new mu.Table();
var voteCounts = new mu.Table();

// Create a HTTP endpoint Service that receives votes from an API:
var voteAPI = new mu.HTTPGateway();
voteAPI.post("/vote", (req, res) => {
    votes.push({ color: req.info.color, count: 1 });
});

// Keep our aggregated counts up-to-date:
votes.forEach(vote => {
    voteCounts.updateIncrement(vote.color, vote.count);
});

Imagining this were in a single file, voting_stack.js, the single command

$ mu up ./voting_stack.js

would provision all of the requisite cloud resources and make the service come to life.

Let's quickly look at two slight variants of this same code.

First, we can use a custom service to encapsulate resources and logic. The above example is comprised of nothing but Functions, which is great for simple scenarios. Sometimes, however, the situation calls for more structure:

var mu = require("mu");

// Create a HTTP endpoint Service that receives votes from an API:
var voteAPI = new mu.HTTPGateway();
var votingService = new VotingService();
voteAPI.register(votingService);

// Define a Service that creates a Stack, wires up Functions to Triggers, and exposes an API:
class VotingService {
    constructor() {
        this.votes = new mu.Table();
        this.voteCounts = new mu.Table();
        this.votes.forEach(vote => {
            // Keep our aggregated counts up-to-date:
            this.voteCounts.updateIncrement(vote.color, vote.count);
        });
    }
    
    vote(info) {
        this.votes.push({ color: info.color, count: 1 });
    }
}

Instead of registering the API manually with HTTPGateway's get function, we have registered all of VotingService's methods using register. The votes and voteCounts Tables are also entirely encapsulated inside of VotingService. In addition to the benefits of encapsulation, this brings us to our second variant of the code: multi-instancing.

Imagine that we want to offer voting for each of the 50 states. We can simply create many VotingServices:

var mu = require("mu");

// Create a HTTP endpoint Service that receives votes from an API:
var voteAPI = new mu.HTTPGateway();

for (var state of [ "AL", "AK", ... "WI", "WY" ]) {
    var votingService = new VotingService();
    voteAPI.register(`/${state}`, votingService);
}

// VotingService is unchanged from above.

Instead of a single /vote endpoint, there will now be endpoints for each of the 50 states -- /vote/AL, /vote/AK, ..., /vote/WI, and /vote/WY -- each with its own votes and voteCounts tables. Notice that we didn't even have to change the definition of VotingService to do this. Of course, we may want to, in order to perform state-specific logic, name its internal resources to have state prefixes in their names, and so on. But this demonstrates the power of reusability when we define Services in the manner shown above.

This simple example has demonstrated many facets:

1. Infrastructure as code and application logic living side-by-side, using Stacks.
2. Provisioning cloud-native resources, like HTTPGateway and Table, as ordinary Services.
3. Creating entirely serverless cloud applications using Functions and Triggers.
4. Creating custom Services, like VotingService, that encapsulate cloud resources and exports APIs.

A Teardown

Although a developer wrote very simple code in the introductory example, there is a fair bit of machinery behind making it work. In fact, the specific details differ greatly depending on which cloud orchestration fabric you are targeting (such as AWS native, Google Cloud native, Kubernetes, Docker Swarm, and so on); moreover, multiple backends are available for some providers (such as AWS CloudFormation or Terraform when targeting AWS native deployments).

To illustrate how the projections work, let's pick a single provider: AWS native using CloudFormation.

The above example contains two Stacks:

1. The top-level Stack.
2. The inner Stack allocated by VotingService's constructor.

Each of these maps to a single "Stack" in AWS's CloudFormation terminology. To generate them, run:

$ mu build ./voting_stack.js

Inside of each Stack, there are a number of resources. Let's first take a look at the top-level Stack:

1. A native AWS API Gateway.
2. A native AWS Lambda, containing the code for vote wired up to said API Gateway at /vote.

Next, the inner Stack allocated by VotingService:

1. Two native AWS DynamoDB "no-SQL" tables: votes and voteCounts.
2. A native AWS Lambda, containing the callback wired up to the votes DynamoDB table.

In this particular example, there is little advantage to having two Stacks, since we only ever create one VotingService. It's important to remember, however, that Services can be multi-instanced, as in our 50 states example, so they must remain distinct. Of course, many AWS resources may be generated in like fashion: S3 buckets, Route53 DNS entries, and so on. Furthermore, stateful Services will end up requiring EC2 VMs and/or Docker containers.

In addition to generating the metadata, the code is prepared for deployment. This includes some massaging of the code so that it is in the requisite form (e.g., Docker images, S3 tarballs for AWS Lambdas, and so on).

If you were to change the code, rerunning mu build would regenerate the modified Stack. Leveraging the usual techniques for applying diffs to an existing environment allows incremental changes to be made, rather than needing to destroy and redeploy the entire cluster again. Blue green, staged deployments and high availability are both supported.

For simple scenarios, developers may not care what goes on behind the scenes. In those cases, just writing code like the above and running the CLI is perfect. For complex scenarios, on the other hand -- particularly in multi-tenant environments, hybrid or on-premise clouds, and/or when IT organizations want more control over things -- the contents of this section become more important. In fact, organizations may wish to manage the cloud deployment artifacts more intently, possibly even editing them by hand, and/or checking them into source control. Moreover, it's even possible to author these definitions by hand and map them to the program using a mu.yaml file that sits in the middle.