### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
446 lines
13 KiB
Go
446 lines
13 KiB
Go
// Copyright 2016-2018, Pulumi Corporation.
|
|
//
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
// you may not use this file except in compliance with the License.
|
|
// You may obtain a copy of the License at
|
|
//
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
//
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
// See the License for the specific language governing permissions and
|
|
// limitations under the License.
|
|
|
|
package engine
|
|
|
|
import (
|
|
"reflect"
|
|
"time"
|
|
|
|
"github.com/pulumi/pulumi/pkg/diag"
|
|
"github.com/pulumi/pulumi/pkg/diag/colors"
|
|
"github.com/pulumi/pulumi/pkg/resource"
|
|
"github.com/pulumi/pulumi/pkg/resource/config"
|
|
"github.com/pulumi/pulumi/pkg/resource/deploy"
|
|
"github.com/pulumi/pulumi/pkg/tokens"
|
|
"github.com/pulumi/pulumi/pkg/util/contract"
|
|
"github.com/pulumi/pulumi/pkg/util/logging"
|
|
)
|
|
|
|
// Event represents an event generated by the engine during an operation. The underlying
|
|
// type for the `Payload` field will differ depending on the value of the `Type` field
|
|
type Event struct {
|
|
Type EventType
|
|
Payload interface{}
|
|
}
|
|
|
|
// EventType is the kind of event being emitted.
|
|
type EventType string
|
|
|
|
const (
|
|
CancelEvent EventType = "cancel"
|
|
StdoutColorEvent EventType = "stdoutcolor"
|
|
DiagEvent EventType = "diag"
|
|
PreludeEvent EventType = "prelude"
|
|
SummaryEvent EventType = "summary"
|
|
ResourcePreEvent EventType = "resource-pre"
|
|
ResourceOutputsEvent EventType = "resource-outputs"
|
|
ResourceOperationFailed EventType = "resource-operationfailed"
|
|
)
|
|
|
|
func cancelEvent() Event {
|
|
return Event{Type: CancelEvent}
|
|
}
|
|
|
|
// DiagEventPayload is the payload for an event with type `diag`
|
|
type DiagEventPayload struct {
|
|
URN resource.URN
|
|
Prefix string
|
|
Message string
|
|
Color colors.Colorization
|
|
Severity diag.Severity
|
|
StreamID int32
|
|
}
|
|
|
|
type StdoutEventPayload struct {
|
|
Message string
|
|
Color colors.Colorization
|
|
}
|
|
|
|
type PreludeEventPayload struct {
|
|
IsPreview bool // true if this prelude is for a plan operation
|
|
Config map[string]string // the keys and values for config. For encrypted config, the values may be blinded
|
|
}
|
|
|
|
type SummaryEventPayload struct {
|
|
IsPreview bool // true if this summary is for a plan operation
|
|
MaybeCorrupt bool // true if one or more resources may be corrupt
|
|
Duration time.Duration // the duration of the entire update operation (zero values for previews)
|
|
ResourceChanges ResourceChanges // count of changed resources, useful for reporting
|
|
}
|
|
|
|
type ResourceOperationFailedPayload struct {
|
|
Metadata StepEventMetadata
|
|
Status resource.Status
|
|
Steps int
|
|
}
|
|
|
|
type ResourceOutputsEventPayload struct {
|
|
Metadata StepEventMetadata
|
|
Planning bool
|
|
Debug bool
|
|
}
|
|
|
|
type ResourcePreEventPayload struct {
|
|
Metadata StepEventMetadata
|
|
Planning bool
|
|
Debug bool
|
|
}
|
|
|
|
type StepEventMetadata struct {
|
|
Op deploy.StepOp // the operation performed by this step.
|
|
URN resource.URN // the resource URN (for before and after).
|
|
Type tokens.Type // the type affected by this step.
|
|
Old *StepEventStateMetadata // the state of the resource before performing this step.
|
|
New *StepEventStateMetadata // the state of the resource after performing this step.
|
|
Res *StepEventStateMetadata // the latest state for the resource that is known (worst case, old).
|
|
Keys []resource.PropertyKey // the keys causing replacement (only for CreateStep and ReplaceStep).
|
|
Logical bool // true if this step represents a logical operation in the program.
|
|
Provider string // the provider that performed this step.
|
|
}
|
|
|
|
type StepEventStateMetadata struct {
|
|
// the resource's type.
|
|
Type tokens.Type
|
|
// the resource's object urn, a human-friendly, unique name for the resource.
|
|
URN resource.URN
|
|
// true if the resource is custom, managed by a plugin.
|
|
Custom bool
|
|
// true if this resource is pending deletion due to a replacement.
|
|
Delete bool
|
|
// the resource's unique ID, assigned by the resource provider (or blank if none/uncreated).
|
|
ID resource.ID
|
|
// an optional parent URN that this resource belongs to.
|
|
Parent resource.URN
|
|
// true to "protect" this resource (protected resources cannot be deleted).
|
|
Protect bool
|
|
// the resource's input properties (as specified by the program). Note: because this will cross
|
|
// over rpc boundaries it will be slightly different than the Inputs found in resource_state.
|
|
// Specifically, secrets will have been filtered out, and large values (like assets) will be
|
|
// have a simple hash-based representation. This allows clients to display this information
|
|
// properly, without worrying about leaking sensitive data, and without having to transmit huge
|
|
// amounts of data.
|
|
Inputs resource.PropertyMap
|
|
// the resource's complete output state (as returned by the resource provider). See "Inputs"
|
|
// for additional details about how data will be transformed before going into this map.
|
|
Outputs resource.PropertyMap
|
|
// the resource's provider reference
|
|
Provider string
|
|
}
|
|
|
|
func makeEventEmitter(events chan<- Event, update UpdateInfo) eventEmitter {
|
|
target := update.GetTarget()
|
|
var secrets []string
|
|
if target.Config.HasSecureValue() {
|
|
for _, v := range target.Config {
|
|
if !v.Secure() {
|
|
continue
|
|
}
|
|
secret, err := v.Value(target.Decrypter)
|
|
contract.AssertNoError(err)
|
|
|
|
secrets = append(secrets, secret)
|
|
}
|
|
}
|
|
|
|
logging.AddGlobalFilter(logging.CreateFilter(secrets, "[secret]"))
|
|
|
|
return eventEmitter{
|
|
Chan: events,
|
|
}
|
|
}
|
|
|
|
type eventEmitter struct {
|
|
Chan chan<- Event
|
|
}
|
|
|
|
func makeStepEventMetadata(step deploy.Step, debug bool) StepEventMetadata {
|
|
var keys []resource.PropertyKey
|
|
|
|
if step.Op() == deploy.OpCreateReplacement {
|
|
keys = step.(*deploy.CreateStep).Keys()
|
|
} else if step.Op() == deploy.OpReplace {
|
|
keys = step.(*deploy.ReplaceStep).Keys()
|
|
}
|
|
|
|
return StepEventMetadata{
|
|
Op: step.Op(),
|
|
URN: step.URN(),
|
|
Type: step.Type(),
|
|
Keys: keys,
|
|
Old: makeStepEventStateMetadata(step.Old(), debug),
|
|
New: makeStepEventStateMetadata(step.New(), debug),
|
|
Res: makeStepEventStateMetadata(step.Res(), debug),
|
|
Logical: step.Logical(),
|
|
Provider: step.Provider(),
|
|
}
|
|
}
|
|
|
|
func makeStepEventStateMetadata(state *resource.State, debug bool) *StepEventStateMetadata {
|
|
if state == nil {
|
|
return nil
|
|
}
|
|
|
|
return &StepEventStateMetadata{
|
|
Type: state.Type,
|
|
URN: state.URN,
|
|
Custom: state.Custom,
|
|
Delete: state.Delete,
|
|
ID: state.ID,
|
|
Parent: state.Parent,
|
|
Protect: state.Protect,
|
|
Inputs: filterPropertyMap(state.Inputs, debug),
|
|
Outputs: filterPropertyMap(state.Outputs, debug),
|
|
Provider: state.Provider,
|
|
}
|
|
}
|
|
|
|
func filterPropertyMap(propertyMap resource.PropertyMap, debug bool) resource.PropertyMap {
|
|
mappable := propertyMap.Mappable()
|
|
|
|
var filterValue func(v interface{}) interface{}
|
|
|
|
filterPropertyValue := func(pv resource.PropertyValue) resource.PropertyValue {
|
|
return resource.NewPropertyValue(filterValue(pv.Mappable()))
|
|
}
|
|
|
|
// filter values walks unwrapped (i.e. non-PropertyValue) values and applies the filter function
|
|
// to them recursively. The only thing the filter actually applies to is strings.
|
|
//
|
|
// The return value of this function should have the same type as the input value.
|
|
filterValue = func(v interface{}) interface{} {
|
|
if v == nil {
|
|
return nil
|
|
}
|
|
|
|
// Else, check for some known primitive types.
|
|
switch t := v.(type) {
|
|
case bool, int, uint, int32, uint32,
|
|
int64, uint64, float32, float64:
|
|
// simple types. map over as is.
|
|
return v
|
|
case string:
|
|
// have to ensure we filter out secrets.
|
|
return logging.FilterString(t)
|
|
case *resource.Asset:
|
|
text := t.Text
|
|
if text != "" {
|
|
// we don't want to include the full text of an asset as we serialize it over as
|
|
// events. They represent user files and are thus are unbounded in size. Instead,
|
|
// we only include the text if it represents a user's serialized program code, as
|
|
// that is something we want the receiver to see to display as part of
|
|
// progress/diffs/etc.
|
|
if t.IsUserProgramCode() {
|
|
// also make sure we filter this in case there are any secrets in the code.
|
|
text = logging.FilterString(resource.MassageIfUserProgramCodeAsset(t, debug).Text)
|
|
} else {
|
|
// We need to have some string here so that we preserve that this is a
|
|
// text-asset
|
|
text = "<stripped>"
|
|
}
|
|
}
|
|
|
|
return &resource.Asset{
|
|
Sig: t.Sig,
|
|
Hash: t.Hash,
|
|
Text: text,
|
|
Path: t.Path,
|
|
URI: t.URI,
|
|
}
|
|
case *resource.Archive:
|
|
return &resource.Archive{
|
|
Sig: t.Sig,
|
|
Hash: t.Hash,
|
|
Path: t.Path,
|
|
URI: t.URI,
|
|
Assets: filterValue(t.Assets).(map[string]interface{}),
|
|
}
|
|
case resource.Computed:
|
|
return resource.Computed{
|
|
Element: filterPropertyValue(t.Element),
|
|
}
|
|
case resource.Output:
|
|
return resource.Output{
|
|
Element: filterPropertyValue(t.Element),
|
|
}
|
|
}
|
|
|
|
// Next, see if it's an array, slice, pointer or struct, and handle each accordingly.
|
|
rv := reflect.ValueOf(v)
|
|
switch rk := rv.Type().Kind(); rk {
|
|
case reflect.Array, reflect.Slice:
|
|
// If an array or slice, just create an array out of it.
|
|
var arr []interface{}
|
|
for i := 0; i < rv.Len(); i++ {
|
|
arr = append(arr, filterValue(rv.Index(i).Interface()))
|
|
}
|
|
return arr
|
|
case reflect.Ptr:
|
|
if rv.IsNil() {
|
|
return nil
|
|
}
|
|
|
|
v1 := filterValue(rv.Elem().Interface())
|
|
return &v1
|
|
case reflect.Map:
|
|
obj := make(map[string]interface{})
|
|
for _, key := range rv.MapKeys() {
|
|
k := key.Interface().(string)
|
|
v := rv.MapIndex(key).Interface()
|
|
obj[k] = filterValue(v)
|
|
}
|
|
return obj
|
|
default:
|
|
contract.Failf("Unrecognized value type: type=%v kind=%v", rv.Type(), rk)
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
return resource.NewPropertyMapFromMapRepl(
|
|
mappable, nil, /*replk*/
|
|
func(v interface{}) (resource.PropertyValue, bool) {
|
|
return resource.NewPropertyValue(filterValue(v)), true
|
|
})
|
|
}
|
|
|
|
func (e *eventEmitter) resourceOperationFailedEvent(
|
|
step deploy.Step, status resource.Status, steps int, debug bool) {
|
|
|
|
contract.Requiref(e != nil, "e", "!= nil")
|
|
|
|
e.Chan <- Event{
|
|
Type: ResourceOperationFailed,
|
|
Payload: ResourceOperationFailedPayload{
|
|
Metadata: makeStepEventMetadata(step, debug),
|
|
Status: status,
|
|
Steps: steps,
|
|
},
|
|
}
|
|
}
|
|
|
|
func (e *eventEmitter) resourceOutputsEvent(
|
|
step deploy.Step, planning bool, debug bool) {
|
|
|
|
contract.Requiref(e != nil, "e", "!= nil")
|
|
|
|
e.Chan <- Event{
|
|
Type: ResourceOutputsEvent,
|
|
Payload: ResourceOutputsEventPayload{
|
|
Metadata: makeStepEventMetadata(step, debug),
|
|
Planning: planning,
|
|
Debug: debug,
|
|
},
|
|
}
|
|
}
|
|
|
|
func (e *eventEmitter) resourcePreEvent(
|
|
step deploy.Step, planning bool, debug bool) {
|
|
|
|
contract.Requiref(e != nil, "e", "!= nil")
|
|
|
|
e.Chan <- Event{
|
|
Type: ResourcePreEvent,
|
|
Payload: ResourcePreEventPayload{
|
|
Metadata: makeStepEventMetadata(step, debug),
|
|
Planning: planning,
|
|
Debug: debug,
|
|
},
|
|
}
|
|
}
|
|
|
|
func (e *eventEmitter) preludeEvent(isPreview bool, cfg config.Map) {
|
|
contract.Requiref(e != nil, "e", "!= nil")
|
|
|
|
configStringMap := make(map[string]string, len(cfg))
|
|
for k, v := range cfg {
|
|
keyString := k.String()
|
|
valueString, err := v.Value(config.NewBlindingDecrypter())
|
|
contract.AssertNoError(err)
|
|
configStringMap[keyString] = valueString
|
|
}
|
|
|
|
e.Chan <- Event{
|
|
Type: PreludeEvent,
|
|
Payload: PreludeEventPayload{
|
|
IsPreview: isPreview,
|
|
Config: configStringMap,
|
|
},
|
|
}
|
|
}
|
|
|
|
func (e *eventEmitter) previewSummaryEvent(resourceChanges ResourceChanges) {
|
|
contract.Requiref(e != nil, "e", "!= nil")
|
|
|
|
e.Chan <- Event{
|
|
Type: SummaryEvent,
|
|
Payload: SummaryEventPayload{
|
|
IsPreview: true,
|
|
MaybeCorrupt: false,
|
|
Duration: 0,
|
|
ResourceChanges: resourceChanges,
|
|
},
|
|
}
|
|
}
|
|
|
|
func (e *eventEmitter) updateSummaryEvent(maybeCorrupt bool,
|
|
duration time.Duration, resourceChanges ResourceChanges) {
|
|
contract.Requiref(e != nil, "e", "!= nil")
|
|
|
|
e.Chan <- Event{
|
|
Type: SummaryEvent,
|
|
Payload: SummaryEventPayload{
|
|
IsPreview: false,
|
|
MaybeCorrupt: maybeCorrupt,
|
|
Duration: duration,
|
|
ResourceChanges: resourceChanges,
|
|
},
|
|
}
|
|
}
|
|
|
|
func diagEvent(e *eventEmitter, d *diag.Diag, prefix, msg string, sev diag.Severity) {
|
|
contract.Requiref(e != nil, "e", "!= nil")
|
|
|
|
e.Chan <- Event{
|
|
Type: DiagEvent,
|
|
Payload: DiagEventPayload{
|
|
URN: d.URN,
|
|
Prefix: logging.FilterString(prefix),
|
|
Message: logging.FilterString(msg),
|
|
Color: colors.Raw,
|
|
Severity: sev,
|
|
StreamID: d.StreamID,
|
|
},
|
|
}
|
|
}
|
|
|
|
func (e *eventEmitter) diagDebugEvent(d *diag.Diag, prefix, msg string) {
|
|
diagEvent(e, d, prefix, msg, diag.Debug)
|
|
}
|
|
|
|
func (e *eventEmitter) diagInfoEvent(d *diag.Diag, prefix, msg string) {
|
|
diagEvent(e, d, prefix, msg, diag.Info)
|
|
}
|
|
|
|
func (e *eventEmitter) diagInfoerrEvent(d *diag.Diag, prefix, msg string) {
|
|
diagEvent(e, d, prefix, msg, diag.Infoerr)
|
|
}
|
|
|
|
func (e *eventEmitter) diagErrorEvent(d *diag.Diag, prefix, msg string) {
|
|
diagEvent(e, d, prefix, msg, diag.Error)
|
|
}
|
|
|
|
func (e *eventEmitter) diagWarningEvent(d *diag.Diag, prefix, msg string) {
|
|
diagEvent(e, d, prefix, msg, diag.Warning)
|
|
}
|