pulumi/pkg/resource/context.go

111 lines
4 KiB
Go
Raw Normal View History

// Licensed to Pulumi Corporation ("Pulumi") under one or more
// contributor license agreements. See the NOTICE file distributed with
// this work for additional information regarding copyright ownership.
// Pulumi licenses this file to You under the Apache License, Version 2.0
// (the "License"); you may not use this file except in compliance with
// the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
package resource
import (
"context"
"github.com/golang/glog"
"github.com/pulumi/lumi/pkg/diag"
"github.com/pulumi/lumi/pkg/eval/rt"
"github.com/pulumi/lumi/pkg/tokens"
"github.com/pulumi/lumi/pkg/util/contract"
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
)
// Context is used to group related operations together so that associated OS resources can be cached, shared, and
// reclaimed as appropriate.
type Context struct {
Diag diag.Sink // the diagnostics sink to use for messages.
Analyzers map[tokens.QName]Analyzer // a cache of analyzer plugins and their processes.
Providers map[tokens.Package]Provider // a cache of provider plugins and their processes.
ObjRes objectResourceMap // the resources held inside of this snapshot.
ObjURN objectURNMap // a convenient lookup map for object to URN.
IDURN idURNMap // a convenient lookup map for ID to URN.
URNRes urnResourceMap // a convenient lookup map for URN to resource.
URNOldIDs urnIDMap // a convenient lookup map for URNs to old IDs.
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
}
type objectURNMap map[*rt.Object]URN
type objectResourceMap map[*rt.Object]Resource
type idURNMap map[ID]URN
type urnResourceMap map[URN]Resource
type urnIDMap map[URN]ID
func NewContext(d diag.Sink) *Context {
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
return &Context{
Diag: d,
Analyzers: make(map[tokens.QName]Analyzer),
Providers: make(map[tokens.Package]Provider),
ObjRes: make(objectResourceMap),
ObjURN: make(objectURNMap),
IDURN: make(idURNMap),
URNRes: make(urnResourceMap),
URNOldIDs: make(urnIDMap),
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
}
}
// Analyzer fetches the analyzer with a given name, possibly lazily allocating the plugins for it. If an analyzer
// could not be found, or an error occurred while creating it, a non-nil error is returned.
func (ctx *Context) Analyzer(name tokens.QName) (Analyzer, error) {
// First see if we already loaded this plugin.
if plug, has := ctx.Analyzers[name]; has {
contract.Assert(plug != nil)
return plug, nil
}
// If not, try to load and bind to a plugin.
plug, err := NewAnalyzer(ctx, name)
if err == nil {
ctx.Analyzers[name] = plug // memoize the result.
}
return plug, err
}
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
// Provider fetches the provider for a given resource, possibly lazily allocating the plugins for it. If a provider
// could not be found, or an error occurred while creating it, a non-nil error is returned.
func (ctx *Context) Provider(pkg tokens.Package) (Provider, error) {
// First see if we already loaded this plugin.
if plug, has := ctx.Providers[pkg]; has {
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
contract.Assert(plug != nil)
return plug, nil
}
// If not, try to load and bind to a plugin.
plug, err := NewProvider(ctx, pkg)
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
if err == nil {
ctx.Providers[pkg] = plug // memoize the result.
Implement resource provider plugins This change adds basic support for discovering, loading, binding to, and invoking RPC methods on, resource provider plugins. In a nutshell, we add a new context object that will share cached state such as loaded plugins and connections to them. It will be a policy decision in server scenarios how much state to share and between whom. This context also controls per-resource context allocation, which in the future will allow us to perform structured cancellation and teardown amongst entire groups of requests. Plugins are loaded based on their name, and can be found in one of two ways: either simply by having them on your path (with a name of "mu-ressrv-<pkg>", where "<pkg>" is the resource package name with any "/"s replaced with "_"s); or by placing them in the standard library installation location, which need not be on the path for this to work (since we know precisely where to look). If we find a protocol, we will load it as a child process. The protocol for plugins is that they will choose a port on their own -- to eliminate races that'd be involved should Mu attempt to pre-pick one for them -- and then write that out as the first line to STDOUT (terminated by a "\n"). This is the only STDERR/STDOUT that Mu cares about; from there, the plugin is free to write all it pleases (e.g., for logging, debugging purposes, etc). Afterwards, we then bind our gRPC connection to that port, and create a typed resource provider client. The CRUD operations that get driven by plan application are then simple wrappers atop the underlying gRPC calls. For now, we interpret all errors as catastrophic; in the near future, we will probably want to introduce a "structured error" mechanism in the gRPC interface for "transactional errors"; that is, errors for which the server was able to recover to a safe checkpoint, which can be interpreted as ResourceOK rather than ResourceUnknown.
2017-02-19 20:08:06 +01:00
}
return plug, err
}
// Request allocates a request sub-context.
func (ctx *Context) Request() context.Context {
// TODO: support cancellation.
return context.TODO()
}
// Close reclaims all resources associated with this context.
func (ctx *Context) Close() error {
for _, plugin := range ctx.Providers {
if err := plugin.Close(); err != nil {
glog.Infof("Error closing '%v' plugin during shutdown; ignoring: %v", plugin.Pkg(), err)
}
}
ctx.Providers = make(map[tokens.Package]Provider) // empty out the plugin map
return nil
}