pulumi/sdk/proto/provider.proto

106 lines
4.7 KiB
Protocol Buffer
Raw Normal View History

// Licensed to Pulumi Corporation ("Pulumi") under one or more
// contributor license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// Pulumi licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
import "google/protobuf/empty.proto";
import "google/protobuf/struct.proto";
package lumirpc;
// TODO: figure out "transactionality" and possibly "tainting".
// ResourceProvider is a service that understands how to create, read, update, or delete resources for types defined
// within a single MuPackage. It is driven by the overall Mu toolchain in response to blueprints and graphs.
service ResourceProvider {
// Check validates that the given property bag is valid for a resource of the given type.
rpc Check(CheckRequest) returns (CheckResponse) {}
Redo object monikers This change overhauls the way we do object monikers. The old mechanism, generating monikers using graph paths, was far too brittle and prone to collisions. The new approach mixes some amount of "automatic scoping" plus some "explicit naming." Although there is some explicitness, this is arguably a good thing, as the monikers will be relatable back to the source more readily by developers inspecting the graph and resource state. Each moniker has four parts: <Namespace>::<AllocModule>::<Type>::<Name> wherein each element is the following: <Namespace> The namespace being deployed into <AllocModule> The module in which the object was allocated <Type> The type of the resource <Name> The assigned name of the resource The <Namespace> is essentially the deployment target -- so "prod", "stage", etc -- although it is more general purpose to allow for future namespacing within a target (e.g., "prod/customer1", etc); for now this is rudimentary, however, see marapongo/mu#94. The <AllocModule> is the token for the code that contained the 'new' that led to this object being created. In the future, we may wish to extend this to also track the module under evaluation. (This is a nice aspect of monikers; they can become arbitrarily complex, so long as they are precise, and not prone to false positives/negatives.) The <Name> warrants more discussion. The resource provider is consulted via a new gRPC method, Name, that fetches the name. How the provider does this is entirely up to it. For some resource types, the resource may have properties that developers must set (e.g., `new Bucket("foo")`); for other providers, perhaps the resource intrinsically has a property that explicitly and uniquely qualifies the object (e.g., AWS SecurityGroups, via `new SecurityGroup({groupName: "my-sg"}`); and finally, it's conceivable that a provider might auto-generate the name (e.g., such as an AWS Lambda whose name could simply be a hash of the source code contents). This should overall produce better results with respect to moniker collisions, ability to match resources, and the usability of the system.
2017-02-24 23:50:02 +01:00
// Name names a given resource. Sometimes this will be assigned by a developer, and so the provider
// simply fetches it from the property bag; other times, the provider will assign this based on its own algorithm.
// In any case, resources with the same name must be safe to use interchangeably with one another.
rpc Name(NameRequest) returns (NameResponse) {}
// Create allocates a new instance of the provided resource and returns its unique ID afterwards. (The input ID
// must be blank.) If this call fails, the resource must not have been created (i.e., it is "transacational").
rpc Create(CreateRequest) returns (CreateResponse) {}
// Get reads the instance state identified by ID, returning a populated resource object, or an error if not found.
rpc Get(GetRequest) returns (GetResponse) {}
// InspectChange checks what impacts a hypothetical update will have on the resource's properties.
rpc InspectChange(ChangeRequest) returns (InspectChangeResponse) {}
// Update updates an existing resource with new values.
rpc Update(ChangeRequest) returns (google.protobuf.Empty) {}
// Delete tears down an existing resource with the given ID. If it fails, the resource is assumed to still exist.
rpc Delete(DeleteRequest) returns (google.protobuf.Empty) {}
}
message CheckRequest {
string type = 1; // the type token of the resource.
google.protobuf.Struct properties = 2; // the full properties to use for validation.
}
message CheckResponse {
repeated CheckFailure failures = 1; // the name of the resource.
}
message CheckFailure {
string property = 1; // the property that failed validation.
string reason = 2; // the reason that the property failed validation.
}
Redo object monikers This change overhauls the way we do object monikers. The old mechanism, generating monikers using graph paths, was far too brittle and prone to collisions. The new approach mixes some amount of "automatic scoping" plus some "explicit naming." Although there is some explicitness, this is arguably a good thing, as the monikers will be relatable back to the source more readily by developers inspecting the graph and resource state. Each moniker has four parts: <Namespace>::<AllocModule>::<Type>::<Name> wherein each element is the following: <Namespace> The namespace being deployed into <AllocModule> The module in which the object was allocated <Type> The type of the resource <Name> The assigned name of the resource The <Namespace> is essentially the deployment target -- so "prod", "stage", etc -- although it is more general purpose to allow for future namespacing within a target (e.g., "prod/customer1", etc); for now this is rudimentary, however, see marapongo/mu#94. The <AllocModule> is the token for the code that contained the 'new' that led to this object being created. In the future, we may wish to extend this to also track the module under evaluation. (This is a nice aspect of monikers; they can become arbitrarily complex, so long as they are precise, and not prone to false positives/negatives.) The <Name> warrants more discussion. The resource provider is consulted via a new gRPC method, Name, that fetches the name. How the provider does this is entirely up to it. For some resource types, the resource may have properties that developers must set (e.g., `new Bucket("foo")`); for other providers, perhaps the resource intrinsically has a property that explicitly and uniquely qualifies the object (e.g., AWS SecurityGroups, via `new SecurityGroup({groupName: "my-sg"}`); and finally, it's conceivable that a provider might auto-generate the name (e.g., such as an AWS Lambda whose name could simply be a hash of the source code contents). This should overall produce better results with respect to moniker collisions, ability to match resources, and the usability of the system.
2017-02-24 23:50:02 +01:00
message NameRequest {
string type = 1; // the type token of the resource.
google.protobuf.Struct properties = 2; // the full properties to use for name creation.
}
message NameResponse {
string name = 1; // the name of the resource.
}
message CreateRequest {
string type = 1; // the type token of the resource.
google.protobuf.Struct properties = 2; // the properties to set during creation.
}
message CreateResponse {
string id = 1; // the ID of the resource created.
google.protobuf.Struct outputs = 2; // any properties populated at creation time.
}
message GetRequest {
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
string id = 1; // the ID of the resource to read.
string type = 2; // the type token of the resource.
}
message GetResponse {
google.protobuf.Struct properties = 1; // the properties read from the resource.
}
message ChangeRequest {
string id = 1; // the ID of the resource to update.
Begin resource modeling and planning This change introduces a new package, pkg/resource, that will form the foundation for actually performing deployment plans and applications. It contains the following key abstractions: * resource.Provider is a wrapper around the CRUD operations exposed by underlying resource plugins. It will eventually defer to resource.Plugin, which itself defers -- over an RPC interface -- to the actual plugin, one per package exposing resources. The provider will also understand how to load, cache, and overall manage the lifetime of each plugin. * resource.Resource is the actual resource object. This is created from the overall evaluation object graph, but is simplified. It contains only serializable properties, for example. Inter-resource references are translated into serializable monikers as part of creating the resource. * resource.Moniker is a serializable string that uniquely identifies a resource in the Mu system. This is in contrast to resource IDs, which are generated by resource providers and generally opaque to the Mu system. See marapongo/mu#69 for more information about monikers and some of their challenges (namely, designing a stable algorithm). * resource.Snapshot is a "snapshot" taken from a graph of resources. This is a transitive closure of state representing one possible configuration of a given environment. This is what plans are created from. Eventually, two snapshots will be diffable, in order to perform incremental updates. One way of thinking about this is that a snapshot of the old world's state is advanced, one step at a time, until it reaches a desired snapshot of the new world's state. * resource.Plan is a plan for carrying out desired CRUD operations on a target environment. Each plan consists of zero-to-many Steps, each of which has a CRUD operation type, a resource target, and a next step. This is an enumerator because it is possible the plan will evolve -- and introduce new steps -- as it is carried out (hence, the Next() method). At the moment, this is linearized; eventually, we want to make this more "graph-like" so that we can exploit available parallelism within the dependencies. There are tons of TODOs remaining. However, the `mu plan` command is functioning with these new changes -- including colorization FTW -- so I'm landing it now. This is part of marapongo/mu#38 and marapongo/mu#41.
2017-02-17 21:31:48 +01:00
string type = 2; // the type token of the resource to update.
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
google.protobuf.Struct olds = 3; // the old values of properties to update.
google.protobuf.Struct news = 4; // the new values of properties to update.
}
message InspectChangeResponse {
repeated string replaces = 1; // if this update requires a replacement, the set of properties triggering it.
google.protobuf.Struct changes = 2; // the set of properties that will be changed (but don't require a replacement).
}
message DeleteRequest {
Begin resource modeling and planning This change introduces a new package, pkg/resource, that will form the foundation for actually performing deployment plans and applications. It contains the following key abstractions: * resource.Provider is a wrapper around the CRUD operations exposed by underlying resource plugins. It will eventually defer to resource.Plugin, which itself defers -- over an RPC interface -- to the actual plugin, one per package exposing resources. The provider will also understand how to load, cache, and overall manage the lifetime of each plugin. * resource.Resource is the actual resource object. This is created from the overall evaluation object graph, but is simplified. It contains only serializable properties, for example. Inter-resource references are translated into serializable monikers as part of creating the resource. * resource.Moniker is a serializable string that uniquely identifies a resource in the Mu system. This is in contrast to resource IDs, which are generated by resource providers and generally opaque to the Mu system. See marapongo/mu#69 for more information about monikers and some of their challenges (namely, designing a stable algorithm). * resource.Snapshot is a "snapshot" taken from a graph of resources. This is a transitive closure of state representing one possible configuration of a given environment. This is what plans are created from. Eventually, two snapshots will be diffable, in order to perform incremental updates. One way of thinking about this is that a snapshot of the old world's state is advanced, one step at a time, until it reaches a desired snapshot of the new world's state. * resource.Plan is a plan for carrying out desired CRUD operations on a target environment. Each plan consists of zero-to-many Steps, each of which has a CRUD operation type, a resource target, and a next step. This is an enumerator because it is possible the plan will evolve -- and introduce new steps -- as it is carried out (hence, the Next() method). At the moment, this is linearized; eventually, we want to make this more "graph-like" so that we can exploit available parallelism within the dependencies. There are tons of TODOs remaining. However, the `mu plan` command is functioning with these new changes -- including colorization FTW -- so I'm landing it now. This is part of marapongo/mu#38 and marapongo/mu#41.
2017-02-17 21:31:48 +01:00
string id = 1; // the ID of the resource to delete.
string type = 2; // the type token of the resource to update.
}