pulumi/pkg/resource/deploy/source.go
Sean Gillespie 9a5f4044fa
Serialize SourceEvents coming from the refresh source (#1725)
* Serialize SourceEvents coming from the refresh source

The engine requires that a source event coming from a source be "ready
to execute" at the moment that it is sent to the engine. Since the
refresh source sent all goal states eagerly through its source iterator,
the engine assumed that it was legal to execute them all in parallel and
did so. This is a problem for the snapshot, since the snapshot expects
to be in an order that is a legal topological ordering of the dependency
DAG.

This PR fixes the issue by sending refresh source events one-at-a-time
through the refresh source iterator, only unblocking to send the next
step as soon as the previous step completes.

* Fix deadlock in refresh test

* Fix an issue where the engine "completed" steps too early

By signalling that a step is done before committing the step's results
to the snapshot, the engine was left with a race where dependent
resources could find themselves completely executed and committed before
a resource that they depend on has been committed.

Fixes pulumi/pulumi#1726

* Fix an issue with Replace steps at the end of a plan

If the last step that was executed successfully was a Replace, we could
end up in a situation where we unintentionally left the snapshot
invalid.

* Add a test

* CR: pass context.Context as first parameter to Iterate

* CR: null->nil
2018-08-08 13:45:48 -07:00

115 lines
4.4 KiB
Go

// Copyright 2016-2018, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package deploy
import (
"context"
"io"
"github.com/pulumi/pulumi/pkg/resource"
"github.com/pulumi/pulumi/pkg/resource/deploy/providers"
"github.com/pulumi/pulumi/pkg/resource/plugin"
"github.com/pulumi/pulumi/pkg/tokens"
)
// A ProviderSource allows a Source to lookup provider plugins.
type ProviderSource interface {
// GetProvider fetches the provider plugin for the given reference.
GetProvider(ref providers.Reference) (plugin.Provider, bool)
}
// A Source can generate a new set of resources that the planner will process accordingly.
type Source interface {
io.Closer
// Project returns the package name of the Pulumi project we are obtaining resources from.
Project() tokens.PackageName
// Info returns a serializable payload that can be used to stamp snapshots for future reconciliation.
Info() interface{}
// IsRefresh indicates whether this source returns events source from existing state (true), and hence can simply
// be assumed to reflect existing state, or whether the events should acted upon (false).
IsRefresh() bool
// Iterate begins iterating the source. Error is non-nil upon failure; otherwise, a valid iterator is returned.
Iterate(ctx context.Context, opts Options, providers ProviderSource) (SourceIterator, error)
}
// A SourceIterator enumerates the list of resources that a source has to offer and tracks associated state.
type SourceIterator interface {
io.Closer
// Next returns the next event from the source.
Next() (SourceEvent, error)
}
// SourceEvent is an event associated with the enumeration of a plan. It is an intent expressed by the source
// program, and it is the responsibility of the engine to make it so.
type SourceEvent interface {
event()
}
// RegisterResourceEvent is a step that asks the engine to provision a resource.
type RegisterResourceEvent interface {
SourceEvent
// Goal returns the goal state for the resource object that was allocated by the program.
Goal() *resource.Goal
// Done indicates that we are done with this step. It must be called to perform cleanup associated with the step.
Done(result *RegisterResult)
}
// RegisterResult is the state of the resource after it has been registered.
type RegisterResult struct {
State *resource.State // the resource state.
Stable bool // if true, the resource state is stable and may be trusted.
Stables []resource.PropertyKey // an optional list of specific resource properties that are stable.
}
// RegisterResourceOutputsEvent is an event that asks the engine to complete the provisioning of a resource.
type RegisterResourceOutputsEvent interface {
SourceEvent
// URN is the resource URN that this completion applies to.
URN() resource.URN
// Outputs returns a property map of output properties to add to a resource before completing.
Outputs() resource.PropertyMap
// Done indicates that we are done with this step. It must be called to perform cleanup associated with the step.
Done()
}
// ReadResourceEvent is an event that asks the engine to read the state of a resource that already exists.
type ReadResourceEvent interface {
SourceEvent
// ID is the requested ID of this read.
ID() resource.ID
// Name is the requested name of this read.
Name() tokens.QName
// Type is type of the resource being read.
Type() tokens.Type
// Provider is a reference to the provider instance to use for this read.
Provider() string
// Parent is the parent resource of the resource being read.
Parent() resource.URN
// Properties is the property bag that will be passed to Read as search parameters.
Properties() resource.PropertyMap
// Dependencies returns the list of URNs upon which this read depends.
Dependencies() []resource.URN
// Done indicates that we are done with this event.
Done(result *ReadResult)
}
type ReadResult struct {
State *resource.State
}