2018-05-22 21:43:36 +02:00
|
|
|
// Copyright 2016-2018, Pulumi Corporation.
|
|
|
|
//
|
|
|
|
// Licensed under the Apache License, Version 2.0 (the "License");
|
|
|
|
// you may not use this file except in compliance with the License.
|
|
|
|
// You may obtain a copy of the License at
|
|
|
|
//
|
|
|
|
// http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
//
|
|
|
|
// Unless required by applicable law or agreed to in writing, software
|
|
|
|
// distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
// See the License for the specific language governing permissions and
|
|
|
|
// limitations under the License.
|
2017-06-10 03:34:37 +02:00
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
package deploy
|
2017-06-10 03:34:37 +02:00
|
|
|
|
|
|
|
import (
|
2019-04-10 22:47:54 +02:00
|
|
|
"fmt"
|
|
|
|
"strings"
|
|
|
|
|
Implement resource protection (#751)
This change implements resource protection, as per pulumi/pulumi#689.
The overall idea is that a resource can be marked as "protect: true",
which will prevent deletion of that resource for any reason whatsoever
(straight deletion, replacement, etc). This is expressed in the
program. To "unprotect" a resource, one must perform an update setting
"protect: false", and then afterwards, they can delete the resource.
For example:
let res = new MyResource("precious", { .. }, { protect: true });
Afterwards, the resource will display in the CLI with a lock icon, and
any attempts to remove it will fail in the usual ways (in planning or,
worst case, during an actual update).
This was done by adding a new ResourceOptions bag parameter to the
base Resource types. This is unfortunately a breaking change, but now
is the right time to take this one. We had been adding new settings
one by one -- like parent and dependsOn -- and this new approach will
set us up to add any number of additional settings down the road,
without needing to worry about breaking anything ever again.
This is related to protected stacks, as described in
pulumi/pulumi-service#399. Most likely this will serve as a foundational
building block that enables the coarser grained policy management.
2017-12-20 23:31:07 +01:00
|
|
|
"github.com/pkg/errors"
|
|
|
|
|
2019-07-12 20:12:01 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/diag"
|
2017-09-22 04:18:21 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/diag/colors"
|
|
|
|
"github.com/pulumi/pulumi/pkg/resource"
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/resource/deploy/providers"
|
2017-09-22 04:18:21 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/resource/plugin"
|
|
|
|
"github.com/pulumi/pulumi/pkg/tokens"
|
|
|
|
"github.com/pulumi/pulumi/pkg/util/contract"
|
2019-08-16 20:04:03 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/util/logging"
|
2017-06-10 03:34:37 +02:00
|
|
|
)
|
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
// StepCompleteFunc is the type of functions returned from Step.Apply. These functions are to be called
|
|
|
|
// when the engine has fully retired a step.
|
|
|
|
type StepCompleteFunc func()
|
|
|
|
|
2017-06-10 03:34:37 +02:00
|
|
|
// Step is a specification for a deployment operation.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
type Step interface {
|
2018-08-08 22:45:48 +02:00
|
|
|
// Apply applies or previews this step. It returns the status of the resource after the step application,
|
|
|
|
// a function to call to signal that this step has fully completed, and an error, if one occurred while applying
|
|
|
|
// the step.
|
|
|
|
//
|
|
|
|
// The returned StepCompleteFunc, if not nil, must be called after committing the results of this step into
|
|
|
|
// the state of the deployment.
|
|
|
|
Apply(preview bool) (resource.Status, StepCompleteFunc, error) // applies or previews this step.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2017-11-17 03:21:41 +01:00
|
|
|
Op() StepOp // the operation performed by this step.
|
2017-08-30 03:24:12 +02:00
|
|
|
URN() resource.URN // the resource URN (for before and after).
|
2017-11-17 03:21:41 +01:00
|
|
|
Type() tokens.Type // the type affected by this step.
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
Provider() string // the provider reference for this step.
|
2017-08-30 03:24:12 +02:00
|
|
|
Old() *resource.State // the state of the resource before performing this step.
|
|
|
|
New() *resource.State // the state of the resource after performing this step.
|
2017-11-17 03:21:41 +01:00
|
|
|
Res() *resource.State // the latest state for the resource that is known (worst case, old).
|
|
|
|
Logical() bool // true if this step represents a logical operation in the program.
|
2018-04-26 02:20:08 +02:00
|
|
|
Plan() *Plan // the owning plan.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// SameStep is a mutating step that does nothing.
|
|
|
|
type SameStep struct {
|
2018-04-26 02:20:08 +02:00
|
|
|
plan *Plan // the current plan.
|
2017-11-29 20:27:32 +01:00
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
old *resource.State // the state of the resource before this step.
|
|
|
|
new *resource.State // the state of the resource after this step.
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
2017-11-17 03:21:41 +01:00
|
|
|
var _ Step = (*SameStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-04-26 02:20:08 +02:00
|
|
|
func NewSameStep(plan *Plan, reg RegisterResourceEvent, old *resource.State, new *resource.State) Step {
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assert(old.URN != "")
|
2017-10-15 12:52:04 +02:00
|
|
|
contract.Assert(old.ID != "" || !old.Custom)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
contract.Assert(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type))
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!old.Delete)
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assert(new.URN != "")
|
|
|
|
contract.Assert(new.ID == "")
|
2019-05-31 02:48:00 +02:00
|
|
|
contract.Assert(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type))
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!new.Delete)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
return &SameStep{
|
2018-04-26 02:20:08 +02:00
|
|
|
plan: plan,
|
2017-11-21 02:38:09 +01:00
|
|
|
reg: reg,
|
2017-08-01 03:26:15 +02:00
|
|
|
old: old,
|
|
|
|
new: new,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2018-04-26 02:20:08 +02:00
|
|
|
func (s *SameStep) Op() StepOp { return OpSame }
|
|
|
|
func (s *SameStep) Plan() *Plan { return s.plan }
|
2019-05-31 02:48:00 +02:00
|
|
|
func (s *SameStep) Type() tokens.Type { return s.new.Type }
|
|
|
|
func (s *SameStep) Provider() string { return s.new.Provider }
|
|
|
|
func (s *SameStep) URN() resource.URN { return s.new.URN }
|
2018-04-26 02:20:08 +02:00
|
|
|
func (s *SameStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *SameStep) New() *resource.State { return s.new }
|
|
|
|
func (s *SameStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *SameStep) Logical() bool { return true }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
func (s *SameStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
2019-05-31 02:48:00 +02:00
|
|
|
// Retain the ID, and outputs:
|
Implement resource protection (#751)
This change implements resource protection, as per pulumi/pulumi#689.
The overall idea is that a resource can be marked as "protect: true",
which will prevent deletion of that resource for any reason whatsoever
(straight deletion, replacement, etc). This is expressed in the
program. To "unprotect" a resource, one must perform an update setting
"protect: false", and then afterwards, they can delete the resource.
For example:
let res = new MyResource("precious", { .. }, { protect: true });
Afterwards, the resource will display in the CLI with a lock icon, and
any attempts to remove it will fail in the usual ways (in planning or,
worst case, during an actual update).
This was done by adding a new ResourceOptions bag parameter to the
base Resource types. This is unfortunately a breaking change, but now
is the right time to take this one. We had been adding new settings
one by one -- like parent and dependsOn -- and this new approach will
set us up to add any number of additional settings down the road,
without needing to worry about breaking anything ever again.
This is related to protected stacks, as described in
pulumi/pulumi-service#399. Most likely this will serve as a foundational
building block that enables the coarser grained policy management.
2017-12-20 23:31:07 +01:00
|
|
|
s.new.ID = s.old.ID
|
|
|
|
s.new.Outputs = s.old.Outputs
|
Propagate inputs to outputs during preview. (#3327)
These changes restore a more-correct version of the behavior that was
disabled with #3014. The original implementation of this behavior was
done in the SDKs, which do not have access to the complete inputs for a
resource (in particular, default values filled in by the provider during
`Check` are not exposed to the SDK). This lack of information meant that
the resolved output values could disagree with the typings present in
a provider SDK. Exacerbating this problem was the fact that unknown
values were dropped entirely, causing `undefined` values to appear in
unexpected places.
By doing this in the engine and allowing unknown values to be
represented in a first-class manner in the SDK, we can attack both of
these issues.
Although this behavior is not _strictly_ consistent with respect to the
resource model--in an update, a resource's output properties will come
from its provider and may differ from its input properties--this
behavior was present in the product for a fairly long time without
significant issues. In the future, we may be able to improve the
accuracy of resource outputs during a preview by allowing the provider
to dry-run CRUD operations and return partially-known values where
possible.
These changes also introduce new APIs in the Node and Python SDKs
that work with unknown values in a first-class fashion:
- A new parameter to the `apply` function that indicates that the
callback should be run even if the result of the apply contains
unknown values
- `containsUnknowns` and `isUnknown`, which return true if a value
either contains nested unknown values or is exactly an unknown value
- The `Unknown` type, which represents unknown values
The primary use case for these APIs is to allow nested, properties with
known values to be accessed via the lifted property accessor even when
the containing property is not fully know. A common example of this
pattern is the `metadata.name` property of a Kubernetes `Namespace`
object: while other properties of the `metadata` bag may be unknown,
`name` is often known. These APIs allow `ns.metadata.name` to return a
known value in this case.
In order to avoid exposing downlevel SDKs to unknown values--a change
which could break user code by exposing it to unexpected values--a
language SDK must indicate whether or not it supports first-class
unknown values as part of each `RegisterResourceRequest`.
These changes also allow us to avoid breaking user code with the new
behavior introduced by the prior commit.
Fixes #3190.
2019-11-11 21:09:34 +01:00
|
|
|
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, complete, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// CreateStep is a mutating step that creates an entirely new resource.
|
|
|
|
type CreateStep struct {
|
2019-07-01 21:34:19 +02:00
|
|
|
plan *Plan // the current plan.
|
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
old *resource.State // the state of the existing resource (only for replacements).
|
|
|
|
new *resource.State // the state of the resource after this step.
|
|
|
|
keys []resource.PropertyKey // the keys causing replacement (only for replacements).
|
|
|
|
diffs []resource.PropertyKey // the keys causing a diff (only for replacements).
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured property diff (only for replacements).
|
|
|
|
replacing bool // true if this is a create due to a replacement.
|
|
|
|
pendingDelete bool // true if this replacement should create a pending delete.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2017-11-17 03:21:41 +01:00
|
|
|
var _ Step = (*CreateStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-04-26 02:20:08 +02:00
|
|
|
func NewCreateStep(plan *Plan, reg RegisterResourceEvent, new *resource.State) Step {
|
2017-11-21 02:38:09 +01:00
|
|
|
contract.Assert(reg != nil)
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assert(new.URN != "")
|
|
|
|
contract.Assert(new.ID == "")
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
contract.Assert(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type))
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!new.Delete)
|
2018-08-03 23:06:00 +02:00
|
|
|
contract.Assert(!new.External)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
return &CreateStep{
|
2018-04-26 02:20:08 +02:00
|
|
|
plan: plan,
|
2017-11-21 02:38:09 +01:00
|
|
|
reg: reg,
|
2017-08-01 03:26:15 +02:00
|
|
|
new: new,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
2019-07-01 21:34:19 +02:00
|
|
|
func NewCreateReplacementStep(plan *Plan, reg RegisterResourceEvent, old, new *resource.State,
|
|
|
|
keys, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool) Step {
|
|
|
|
|
2017-11-21 02:38:09 +01:00
|
|
|
contract.Assert(reg != nil)
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assert(old.URN != "")
|
2017-10-15 12:52:04 +02:00
|
|
|
contract.Assert(old.ID != "" || !old.Custom)
|
2018-04-17 08:04:56 +02:00
|
|
|
contract.Assert(!old.Delete)
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assert(new.URN != "")
|
|
|
|
contract.Assert(new.ID == "")
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
contract.Assert(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type))
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!new.Delete)
|
2018-08-03 23:06:00 +02:00
|
|
|
contract.Assert(!new.External)
|
2017-08-06 19:05:51 +02:00
|
|
|
return &CreateStep{
|
2018-04-26 02:20:08 +02:00
|
|
|
plan: plan,
|
2017-12-10 17:37:22 +01:00
|
|
|
reg: reg,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
keys: keys,
|
2019-03-07 01:41:19 +01:00
|
|
|
diffs: diffs,
|
2019-07-01 21:34:19 +02:00
|
|
|
detailedDiff: detailedDiff,
|
2017-12-10 17:37:22 +01:00
|
|
|
replacing: true,
|
|
|
|
pendingDelete: pendingDelete,
|
2017-08-06 19:05:51 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *CreateStep) Op() StepOp {
|
|
|
|
if s.replacing {
|
|
|
|
return OpCreateReplacement
|
|
|
|
}
|
|
|
|
return OpCreate
|
|
|
|
}
|
2019-07-01 21:34:19 +02:00
|
|
|
func (s *CreateStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *CreateStep) Type() tokens.Type { return s.new.Type }
|
|
|
|
func (s *CreateStep) Provider() string { return s.new.Provider }
|
|
|
|
func (s *CreateStep) URN() resource.URN { return s.new.URN }
|
|
|
|
func (s *CreateStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *CreateStep) New() *resource.State { return s.new }
|
|
|
|
func (s *CreateStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *CreateStep) Keys() []resource.PropertyKey { return s.keys }
|
|
|
|
func (s *CreateStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *CreateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
|
|
|
func (s *CreateStep) Logical() bool { return !s.replacing }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
func (s *CreateStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
2018-05-02 19:36:55 +02:00
|
|
|
var resourceError error
|
|
|
|
resourceStatus := resource.StatusOK
|
2017-11-30 00:05:58 +01:00
|
|
|
if !preview {
|
2018-08-23 02:52:46 +02:00
|
|
|
if s.new.Custom {
|
2017-11-30 00:05:58 +01:00
|
|
|
// Invoke the Create RPC function for this provider:
|
|
|
|
prov, err := getProvider(s)
|
|
|
|
if err != nil {
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, nil, err
|
2017-11-30 00:05:58 +01:00
|
|
|
}
|
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers
* Plumbing the CustomTimeouts through to the engine and adding test to show this
* Change the provider proto to include individual timeouts
* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface
* Change how the CustomTimeouts are sent across RPC
These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest
```
req=&pulumirpc.RegisterResourceRequest{
Type: "aws:s3/bucket:Bucket",
Name: "my-bucket",
Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
Custom: true,
Object: &structpb.Struct{},
Protect: false,
Dependencies: nil,
Provider: "",
PropertyDependencies: {},
DeleteBeforeReplace: false,
Version: "",
IgnoreChanges: nil,
AcceptSecrets: true,
AdditionalSecretOutputs: nil,
Aliases: nil,
CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{
Create: 300,
Update: 400,
Delete: 500,
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
},
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
}
```
* Changing the design to use strings
* CHANGELOG entry to include the CustomTimeouts work
* Changing custom timeouts to be passed around the engine as converted value
We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
|
|
|
|
|
|
|
id, outs, rst, err := prov.Create(s.URN(), s.new.Inputs, s.new.CustomTimeouts.Create)
|
2017-11-30 00:05:58 +01:00
|
|
|
if err != nil {
|
2018-07-28 01:08:11 +02:00
|
|
|
if rst != resource.StatusPartialFailure {
|
2018-08-08 22:45:48 +02:00
|
|
|
return rst, nil, err
|
2018-05-02 19:36:55 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
resourceError = err
|
|
|
|
resourceStatus = rst
|
2018-07-19 07:52:01 +02:00
|
|
|
|
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
|
|
|
}
|
2017-11-30 00:05:58 +01:00
|
|
|
}
|
2018-05-02 19:36:55 +02:00
|
|
|
|
2017-11-30 00:05:58 +01:00
|
|
|
contract.Assert(id != "")
|
|
|
|
|
|
|
|
// Copy any of the default and output properties on the live object state.
|
|
|
|
s.new.ID = id
|
|
|
|
s.new.Outputs = outs
|
Implement components
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 23:18:43 +02:00
|
|
|
}
|
Propagate inputs to outputs during preview. (#3327)
These changes restore a more-correct version of the behavior that was
disabled with #3014. The original implementation of this behavior was
done in the SDKs, which do not have access to the complete inputs for a
resource (in particular, default values filled in by the provider during
`Check` are not exposed to the SDK). This lack of information meant that
the resolved output values could disagree with the typings present in
a provider SDK. Exacerbating this problem was the fact that unknown
values were dropped entirely, causing `undefined` values to appear in
unexpected places.
By doing this in the engine and allowing unknown values to be
represented in a first-class manner in the SDK, we can attack both of
these issues.
Although this behavior is not _strictly_ consistent with respect to the
resource model--in an update, a resource's output properties will come
from its provider and may differ from its input properties--this
behavior was present in the product for a fairly long time without
significant issues. In the future, we may be able to improve the
accuracy of resource outputs during a preview by allowing the provider
to dry-run CRUD operations and return partially-known values where
possible.
These changes also introduce new APIs in the Node and Python SDKs
that work with unknown values in a first-class fashion:
- A new parameter to the `apply` function that indicates that the
callback should be run even if the result of the apply contains
unknown values
- `containsUnknowns` and `isUnknown`, which return true if a value
either contains nested unknown values or is exactly an unknown value
- The `Unknown` type, which represents unknown values
The primary use case for these APIs is to allow nested, properties with
known values to be accessed via the lifted property accessor even when
the containing property is not fully know. A common example of this
pattern is the `metadata.name` property of a Kubernetes `Namespace`
object: while other properties of the `metadata` bag may be unknown,
`name` is often known. These APIs allow `ns.metadata.name` to return a
known value in this case.
In order to avoid exposing downlevel SDKs to unknown values--a change
which could break user code by exposing it to unexpected values--a
language SDK must indicate whether or not it supports first-class
unknown values as part of each `RegisterResourceRequest`.
These changes also allow us to avoid breaking user code with the new
behavior introduced by the prior commit.
Fixes #3190.
2019-11-11 21:09:34 +01:00
|
|
|
} else {
|
|
|
|
s.new.Outputs = s.new.Inputs
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2017-10-05 23:08:35 +02:00
|
|
|
// Mark the old resource as pending deletion if necessary.
|
2017-12-10 17:37:22 +01:00
|
|
|
if s.replacing && s.pendingDelete {
|
2017-10-05 23:08:35 +02:00
|
|
|
s.old.Delete = true
|
|
|
|
}
|
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
|
2018-07-07 00:17:32 +02:00
|
|
|
if resourceError == nil {
|
2018-08-08 22:45:48 +02:00
|
|
|
return resourceStatus, complete, nil
|
2018-07-07 00:17:32 +02:00
|
|
|
}
|
2018-08-08 22:45:48 +02:00
|
|
|
return resourceStatus, complete, resourceError
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2018-08-03 23:06:00 +02:00
|
|
|
// DeleteStep is a mutating step that deletes an existing resource. If `old` is marked "External",
|
|
|
|
// DeleteStep is a no-op.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
type DeleteStep struct {
|
2018-04-26 02:20:08 +02:00
|
|
|
plan *Plan // the current plan.
|
2017-08-01 03:26:15 +02:00
|
|
|
old *resource.State // the state of the existing resource.
|
|
|
|
replacing bool // true if part of a replacement.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2017-11-17 03:21:41 +01:00
|
|
|
var _ Step = (*DeleteStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-04-26 02:20:08 +02:00
|
|
|
func NewDeleteStep(plan *Plan, old *resource.State) Step {
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assert(old.URN != "")
|
2017-10-15 12:52:04 +02:00
|
|
|
contract.Assert(old.ID != "" || !old.Custom)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
contract.Assert(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type))
|
2017-12-10 17:37:22 +01:00
|
|
|
return &DeleteStep{
|
2018-04-26 02:20:08 +02:00
|
|
|
plan: plan,
|
2017-12-10 17:37:22 +01:00
|
|
|
old: old,
|
|
|
|
}
|
|
|
|
}
|
Implement a refresh command
This change implements a `pulumi refresh` command. It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.
It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events. This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state. This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion). The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.
Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint. That
will be coming soon ...
2018-04-10 20:22:39 +02:00
|
|
|
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
func NewDeleteReplacementStep(plan *Plan, old *resource.State, pendingReplace bool) Step {
|
2017-12-10 17:37:22 +01:00
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assert(old.URN != "")
|
|
|
|
contract.Assert(old.ID != "" || !old.Custom)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
contract.Assert(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type))
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
|
|
|
|
// There are two cases in which we create a delete-replacment step:
|
|
|
|
//
|
|
|
|
// 1. When creating the delete steps that occur due to a delete-before-replace
|
|
|
|
// 2. When creating the delete step that occurs due to a delete-after-replace
|
|
|
|
//
|
|
|
|
// In the former case, the persistence layer may require that the resource remain in the
|
|
|
|
// checkpoint file for purposes of checkpoint integrity. We communicate this case by means
|
|
|
|
// of the `PendingReplacement` field on `resource.State`, which we set here.
|
|
|
|
//
|
|
|
|
// In the latter case, the resource must be deleted, but the deletion may not occur if an earlier step fails.
|
|
|
|
// The engine requires that the fact that the old resource must be deleted is persisted in the checkpoint so
|
|
|
|
// that it can issue a deletion of this resource on the next update to this stack.
|
|
|
|
contract.Assert(pendingReplace != old.Delete)
|
|
|
|
old.PendingReplacement = pendingReplace
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
return &DeleteStep{
|
2018-04-26 02:20:08 +02:00
|
|
|
plan: plan,
|
2017-08-01 03:26:15 +02:00
|
|
|
old: old,
|
2017-12-10 17:37:22 +01:00
|
|
|
replacing: true,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-08-06 19:05:51 +02:00
|
|
|
func (s *DeleteStep) Op() StepOp {
|
2019-01-31 22:48:44 +01:00
|
|
|
if s.old.External {
|
|
|
|
if s.replacing {
|
|
|
|
return OpDiscardReplaced
|
|
|
|
}
|
|
|
|
return OpReadDiscard
|
|
|
|
}
|
|
|
|
|
2017-08-06 19:05:51 +02:00
|
|
|
if s.replacing {
|
|
|
|
return OpDeleteReplaced
|
|
|
|
}
|
|
|
|
return OpDelete
|
|
|
|
}
|
2018-04-26 02:20:08 +02:00
|
|
|
func (s *DeleteStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *DeleteStep) Type() tokens.Type { return s.old.Type }
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
func (s *DeleteStep) Provider() string { return s.old.Provider }
|
2018-04-26 02:20:08 +02:00
|
|
|
func (s *DeleteStep) URN() resource.URN { return s.old.URN }
|
|
|
|
func (s *DeleteStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *DeleteStep) New() *resource.State { return nil }
|
|
|
|
func (s *DeleteStep) Res() *resource.State { return s.old }
|
|
|
|
func (s *DeleteStep) Logical() bool { return !s.replacing }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
func (s *DeleteStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
Implement resource protection (#751)
This change implements resource protection, as per pulumi/pulumi#689.
The overall idea is that a resource can be marked as "protect: true",
which will prevent deletion of that resource for any reason whatsoever
(straight deletion, replacement, etc). This is expressed in the
program. To "unprotect" a resource, one must perform an update setting
"protect: false", and then afterwards, they can delete the resource.
For example:
let res = new MyResource("precious", { .. }, { protect: true });
Afterwards, the resource will display in the CLI with a lock icon, and
any attempts to remove it will fail in the usual ways (in planning or,
worst case, during an actual update).
This was done by adding a new ResourceOptions bag parameter to the
base Resource types. This is unfortunately a breaking change, but now
is the right time to take this one. We had been adding new settings
one by one -- like parent and dependsOn -- and this new approach will
set us up to add any number of additional settings down the road,
without needing to worry about breaking anything ever again.
This is related to protected stacks, as described in
pulumi/pulumi-service#399. Most likely this will serve as a foundational
building block that enables the coarser grained policy management.
2017-12-20 23:31:07 +01:00
|
|
|
// Refuse to delete protected resources.
|
|
|
|
if s.old.Protect {
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, nil,
|
Implement resource protection (#751)
This change implements resource protection, as per pulumi/pulumi#689.
The overall idea is that a resource can be marked as "protect: true",
which will prevent deletion of that resource for any reason whatsoever
(straight deletion, replacement, etc). This is expressed in the
program. To "unprotect" a resource, one must perform an update setting
"protect: false", and then afterwards, they can delete the resource.
For example:
let res = new MyResource("precious", { .. }, { protect: true });
Afterwards, the resource will display in the CLI with a lock icon, and
any attempts to remove it will fail in the usual ways (in planning or,
worst case, during an actual update).
This was done by adding a new ResourceOptions bag parameter to the
base Resource types. This is unfortunately a breaking change, but now
is the right time to take this one. We had been adding new settings
one by one -- like parent and dependsOn -- and this new approach will
set us up to add any number of additional settings down the road,
without needing to worry about breaking anything ever again.
This is related to protected stacks, as described in
pulumi/pulumi-service#399. Most likely this will serve as a foundational
building block that enables the coarser grained policy management.
2017-12-20 23:31:07 +01:00
|
|
|
errors.Errorf("refusing to delete protected resource '%s'", s.old.URN)
|
|
|
|
}
|
|
|
|
|
2018-08-03 23:06:00 +02:00
|
|
|
// Deleting an External resource is a no-op, since Pulumi does not own the lifecycle.
|
|
|
|
if !preview && !s.old.External {
|
2018-08-23 02:52:46 +02:00
|
|
|
if s.old.Custom {
|
2017-11-30 00:05:58 +01:00
|
|
|
// Invoke the Delete RPC function for this provider:
|
|
|
|
prov, err := getProvider(s)
|
|
|
|
if err != nil {
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, nil, err
|
2017-11-30 00:05:58 +01:00
|
|
|
}
|
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers
* Plumbing the CustomTimeouts through to the engine and adding test to show this
* Change the provider proto to include individual timeouts
* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface
* Change how the CustomTimeouts are sent across RPC
These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest
```
req=&pulumirpc.RegisterResourceRequest{
Type: "aws:s3/bucket:Bucket",
Name: "my-bucket",
Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
Custom: true,
Object: &structpb.Struct{},
Protect: false,
Dependencies: nil,
Provider: "",
PropertyDependencies: {},
DeleteBeforeReplace: false,
Version: "",
IgnoreChanges: nil,
AcceptSecrets: true,
AdditionalSecretOutputs: nil,
Aliases: nil,
CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{
Create: 300,
Update: 400,
Delete: 500,
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
},
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
}
```
* Changing the design to use strings
* CHANGELOG entry to include the CustomTimeouts work
* Changing custom timeouts to be passed around the engine as converted value
We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
|
|
|
|
|
|
|
if rst, err := prov.Delete(s.URN(), s.old.ID, s.old.Outputs, s.old.CustomTimeouts.Delete); err != nil {
|
2018-08-08 22:45:48 +02:00
|
|
|
return rst, nil, err
|
2017-11-30 00:05:58 +01:00
|
|
|
}
|
Implement components
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 23:18:43 +02:00
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
2017-11-30 00:05:58 +01:00
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, func() {}, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
type RemovePendingReplaceStep struct {
|
|
|
|
plan *Plan // the current plan.
|
|
|
|
old *resource.State // the state of the existing resource.
|
|
|
|
}
|
|
|
|
|
|
|
|
func NewRemovePendingReplaceStep(plan *Plan, old *resource.State) Step {
|
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assert(old.PendingReplacement)
|
|
|
|
return &RemovePendingReplaceStep{
|
|
|
|
plan: plan,
|
|
|
|
old: old,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *RemovePendingReplaceStep) Op() StepOp {
|
|
|
|
return OpRemovePendingReplace
|
|
|
|
}
|
|
|
|
func (s *RemovePendingReplaceStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *RemovePendingReplaceStep) Type() tokens.Type { return s.old.Type }
|
|
|
|
func (s *RemovePendingReplaceStep) Provider() string { return s.old.Provider }
|
|
|
|
func (s *RemovePendingReplaceStep) URN() resource.URN { return s.old.URN }
|
|
|
|
func (s *RemovePendingReplaceStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *RemovePendingReplaceStep) New() *resource.State { return nil }
|
|
|
|
func (s *RemovePendingReplaceStep) Res() *resource.State { return s.old }
|
|
|
|
func (s *RemovePendingReplaceStep) Logical() bool { return false }
|
|
|
|
|
|
|
|
func (s *RemovePendingReplaceStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
|
|
|
return resource.StatusOK, nil, nil
|
|
|
|
}
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
// UpdateStep is a mutating step that updates an existing resource's state.
|
|
|
|
type UpdateStep struct {
|
2019-07-31 18:39:07 +02:00
|
|
|
plan *Plan // the current plan.
|
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
old *resource.State // the state of the existing resource.
|
|
|
|
new *resource.State // the newly computed state of the resource after updating.
|
|
|
|
stables []resource.PropertyKey // an optional list of properties that won't change during this update.
|
|
|
|
diffs []resource.PropertyKey // the keys causing a diff.
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured diff.
|
|
|
|
ignoreChanges []string // a list of property paths to ignore when updating.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2017-11-17 03:21:41 +01:00
|
|
|
var _ Step = (*UpdateStep)(nil)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-04-26 02:20:08 +02:00
|
|
|
func NewUpdateStep(plan *Plan, reg RegisterResourceEvent, old *resource.State,
|
2019-07-31 18:39:07 +02:00
|
|
|
new *resource.State, stables, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff,
|
|
|
|
ignoreChanges []string) Step {
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assert(old.URN != "")
|
2017-10-15 12:52:04 +02:00
|
|
|
contract.Assert(old.ID != "" || !old.Custom)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
contract.Assert(!old.Custom || old.Provider != "" || providers.IsProviderType(old.Type))
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!old.Delete)
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assert(new.URN != "")
|
|
|
|
contract.Assert(new.ID == "")
|
2019-05-31 02:48:00 +02:00
|
|
|
contract.Assert(!new.Custom || new.Provider != "" || providers.IsProviderType(new.Type))
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!new.Delete)
|
2018-08-03 23:06:00 +02:00
|
|
|
contract.Assert(!new.External)
|
|
|
|
contract.Assert(!old.External)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
return &UpdateStep{
|
2019-07-31 18:39:07 +02:00
|
|
|
plan: plan,
|
|
|
|
reg: reg,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
stables: stables,
|
|
|
|
diffs: diffs,
|
|
|
|
detailedDiff: detailedDiff,
|
|
|
|
ignoreChanges: ignoreChanges,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
2019-07-01 21:34:19 +02:00
|
|
|
func (s *UpdateStep) Op() StepOp { return OpUpdate }
|
|
|
|
func (s *UpdateStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *UpdateStep) Type() tokens.Type { return s.new.Type }
|
|
|
|
func (s *UpdateStep) Provider() string { return s.new.Provider }
|
|
|
|
func (s *UpdateStep) URN() resource.URN { return s.new.URN }
|
|
|
|
func (s *UpdateStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *UpdateStep) New() *resource.State { return s.new }
|
|
|
|
func (s *UpdateStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *UpdateStep) Logical() bool { return true }
|
|
|
|
func (s *UpdateStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *UpdateStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
func (s *UpdateStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
2019-05-31 02:48:00 +02:00
|
|
|
// Always propagate the ID, even in previews and refreshes.
|
Implement a refresh command
This change implements a `pulumi refresh` command. It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.
It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events. This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state. This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion). The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.
Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint. That
will be coming soon ...
2018-04-10 20:22:39 +02:00
|
|
|
s.new.ID = s.old.ID
|
|
|
|
|
2018-05-02 19:36:55 +02:00
|
|
|
var resourceError error
|
|
|
|
resourceStatus := resource.StatusOK
|
Implement a refresh command
This change implements a `pulumi refresh` command. It operates a bit
like `pulumi update`, and friends, in that it supports `--preview` and
`--diff`, along with the usual flags, and will update your checkpoint.
It works through substitution of the deploy.Source abstraction, which
generates a sequence of resource registration events. This new
deploy.RefreshSource takes in a prior checkpoint and will walk it,
refreshing the state via the associated resource providers by invoking
Read for each resource encountered, and merging the resulting state with
the prior checkpoint, to yield a new resource.Goal state. This state is
then fed through the engine in the usual ways with a few minor caveats:
namely, although the engine must generate steps for the logical
operations (permitting us to get nice summaries, progress, and diffs),
it mustn't actually carry them out because the state being imported
already reflects reality (a deleted resource has *already* been deleted,
so of course the engine need not perform the deletion). The diffing
logic also needs to know how to treat the case of refresh slightly
differently, because we are going to be diffing outputs and not inputs.
Note that support for managed stacks is not yet complete, since that
requires updates to the service to support a refresh endpoint. That
will be coming soon ...
2018-04-10 20:22:39 +02:00
|
|
|
if !preview {
|
2018-08-23 02:52:46 +02:00
|
|
|
if s.new.Custom {
|
2017-12-01 01:37:44 +01:00
|
|
|
// Invoke the Update RPC function for this provider:
|
|
|
|
prov, err := getProvider(s)
|
|
|
|
if err != nil {
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, nil, err
|
2017-12-01 01:37:44 +01:00
|
|
|
}
|
General prep work for refresh
This change includes a bunch of refactorings I made in prep for
doing refresh (first, the command, see pulumi/pulumi#1081):
* The primary change is to change the way the engine's core update
functionality works with respect to deploy.Source. This is the
way we can plug in new sources of resource information during
planning (and, soon, diffing). The way I intend to model refresh
is by having a new kind of source, deploy.RefreshSource, which
will let us do virtually everything about an update/diff the same
way with refreshes, which avoid otherwise duplicative effort.
This includes changing the planOptions (nee deployOptions) to
take a new SourceFunc callback, which is responsible for creating
a source specific to the kind of plan being requested.
Preview, Update, and Destroy now are primarily differentiated by
the kind of deploy.Source that they return, rather than sprinkling
things like `if Destroying` throughout. This tidies up some logic
and, more importantly, gives us precisely the refresh hook we need.
* Originally, we used the deploy.NullSource for Destroy operations.
This simply returns nothing, which is how Destroy works. For some
reason, we were no longer doing this, and instead had some
`if Destroying` cases sprinkled throughout the deploy.EvalSource.
I think this is a vestige of some old way we did configuration, at
least judging by a comment, which is apparently no longer relevant.
* Move diff and diff-printing logic within the engine into its own
pkg/engine/diff.go file, to prepare for upcoming work.
* I keep noticing benign diffs anytime I regenerate protobufs. I
suspect this is because we're also on different versions. I changed
generate.sh to also dump the version into grpc_version.txt. At
least we can understand where the diffs are coming from, decide
whether to take them (i.e., a newer version), and ensure that as
a team we are monotonically increasing, and not going backwards.
* I also tidied up some tiny things I noticed while in there, like
comments, incorrect types, lint suppressions, and so on.
2018-03-28 16:45:23 +02:00
|
|
|
|
2019-04-22 22:52:36 +02:00
|
|
|
// Update to the combination of the old "all" state, but overwritten with new inputs.
|
2019-07-31 18:39:07 +02:00
|
|
|
outs, rst, upderr := prov.Update(s.URN(), s.old.ID, s.old.Outputs, s.new.Inputs,
|
|
|
|
s.new.CustomTimeouts.Update, s.ignoreChanges)
|
2017-12-01 01:37:44 +01:00
|
|
|
if upderr != nil {
|
2018-07-28 01:08:11 +02:00
|
|
|
if rst != resource.StatusPartialFailure {
|
2018-08-08 22:45:48 +02:00
|
|
|
return rst, nil, upderr
|
2018-05-02 19:36:55 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
resourceError = upderr
|
|
|
|
resourceStatus = rst
|
2018-07-19 07:52:01 +02:00
|
|
|
|
|
|
|
if initErr, isInitErr := upderr.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
|
|
|
}
|
2017-12-01 01:37:44 +01:00
|
|
|
}
|
Implement components
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 23:18:43 +02:00
|
|
|
|
2017-12-01 01:37:44 +01:00
|
|
|
// Now copy any output state back in case the update triggered cascading updates to other properties.
|
|
|
|
s.new.Outputs = outs
|
|
|
|
}
|
Propagate inputs to outputs during preview. (#3327)
These changes restore a more-correct version of the behavior that was
disabled with #3014. The original implementation of this behavior was
done in the SDKs, which do not have access to the complete inputs for a
resource (in particular, default values filled in by the provider during
`Check` are not exposed to the SDK). This lack of information meant that
the resolved output values could disagree with the typings present in
a provider SDK. Exacerbating this problem was the fact that unknown
values were dropped entirely, causing `undefined` values to appear in
unexpected places.
By doing this in the engine and allowing unknown values to be
represented in a first-class manner in the SDK, we can attack both of
these issues.
Although this behavior is not _strictly_ consistent with respect to the
resource model--in an update, a resource's output properties will come
from its provider and may differ from its input properties--this
behavior was present in the product for a fairly long time without
significant issues. In the future, we may be able to improve the
accuracy of resource outputs during a preview by allowing the provider
to dry-run CRUD operations and return partially-known values where
possible.
These changes also introduce new APIs in the Node and Python SDKs
that work with unknown values in a first-class fashion:
- A new parameter to the `apply` function that indicates that the
callback should be run even if the result of the apply contains
unknown values
- `containsUnknowns` and `isUnknown`, which return true if a value
either contains nested unknown values or is exactly an unknown value
- The `Unknown` type, which represents unknown values
The primary use case for these APIs is to allow nested, properties with
known values to be accessed via the lifted property accessor even when
the containing property is not fully know. A common example of this
pattern is the `metadata.name` property of a Kubernetes `Namespace`
object: while other properties of the `metadata` bag may be unknown,
`name` is often known. These APIs allow `ns.metadata.name` to return a
known value in this case.
In order to avoid exposing downlevel SDKs to unknown values--a change
which could break user code by exposing it to unexpected values--a
language SDK must indicate whether or not it supports first-class
unknown values as part of each `RegisterResourceRequest`.
These changes also allow us to avoid breaking user code with the new
behavior introduced by the prior commit.
Fixes #3190.
2019-11-11 21:09:34 +01:00
|
|
|
} else {
|
|
|
|
s.new.Outputs = s.new.Inputs
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
Implement components
This change implements core support for "components" in the Pulumi
Fabric. This work is described further in pulumi/pulumi#340, where
we are still discussing some of the finer points.
In a nutshell, resources no longer imply external providers. It's
entirely possible to have a resource that logically represents
something but without having a physical manifestation that needs to
be tracked and managed by our typical CRUD operations.
For example, the aws/serverless/Function helper is one such type.
It aggregates Lambda-related resources and exposes a nice interface.
All of the Pulumi Cloud Framework resources are also examples.
To indicate that a resource does participate in the usual CRUD resource
provider, it simply derives from ExternalResource instead of Resource.
All resources now have the ability to adopt children. This is purely
a metadata/tagging thing, and will help us roll up displays, provide
attribution to the developer, and even hide aspects of the resource
graph as appropriate (e.g., when they are implementation details).
Our use of this capability is ultra limited right now; in fact, the
only place we display children is in the CLI output. For instance:
+ aws:serverless:Function: (create)
[urn=urn:pulumi:demo::serverless::aws:serverless:Function::mylambda]
=> urn:pulumi:demo::serverless::aws:iam/role:Role::mylambda-iamrole
=> urn:pulumi:demo::serverless::aws:iam/rolePolicyAttachment:RolePolicyAttachment::mylambda-iampolicy-0
=> urn:pulumi:demo::serverless::aws:lambda/function:Function::mylambda
The bit indicating whether a resource is external or not is tracked
in the resulting checkpoint file, along with any of its children.
2017-10-14 23:18:43 +02:00
|
|
|
// Finally, mark this operation as complete.
|
Propagate inputs to outputs during preview. (#3327)
These changes restore a more-correct version of the behavior that was
disabled with #3014. The original implementation of this behavior was
done in the SDKs, which do not have access to the complete inputs for a
resource (in particular, default values filled in by the provider during
`Check` are not exposed to the SDK). This lack of information meant that
the resolved output values could disagree with the typings present in
a provider SDK. Exacerbating this problem was the fact that unknown
values were dropped entirely, causing `undefined` values to appear in
unexpected places.
By doing this in the engine and allowing unknown values to be
represented in a first-class manner in the SDK, we can attack both of
these issues.
Although this behavior is not _strictly_ consistent with respect to the
resource model--in an update, a resource's output properties will come
from its provider and may differ from its input properties--this
behavior was present in the product for a fairly long time without
significant issues. In the future, we may be able to improve the
accuracy of resource outputs during a preview by allowing the provider
to dry-run CRUD operations and return partially-known values where
possible.
These changes also introduce new APIs in the Node and Python SDKs
that work with unknown values in a first-class fashion:
- A new parameter to the `apply` function that indicates that the
callback should be run even if the result of the apply contains
unknown values
- `containsUnknowns` and `isUnknown`, which return true if a value
either contains nested unknown values or is exactly an unknown value
- The `Unknown` type, which represents unknown values
The primary use case for these APIs is to allow nested, properties with
known values to be accessed via the lifted property accessor even when
the containing property is not fully know. A common example of this
pattern is the `metadata.name` property of a Kubernetes `Namespace`
object: while other properties of the `metadata` bag may be unknown,
`name` is often known. These APIs allow `ns.metadata.name` to return a
known value in this case.
In order to avoid exposing downlevel SDKs to unknown values--a change
which could break user code by exposing it to unexpected values--a
language SDK must indicate whether or not it supports first-class
unknown values as part of each `RegisterResourceRequest`.
These changes also allow us to avoid breaking user code with the new
behavior introduced by the prior commit.
Fixes #3190.
2019-11-11 21:09:34 +01:00
|
|
|
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
|
2018-07-07 00:17:32 +02:00
|
|
|
if resourceError == nil {
|
2018-08-08 22:45:48 +02:00
|
|
|
return resourceStatus, complete, nil
|
2018-07-07 00:17:32 +02:00
|
|
|
}
|
2018-08-08 22:45:48 +02:00
|
|
|
return resourceStatus, complete, resourceError
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2017-08-06 19:05:51 +02:00
|
|
|
// ReplaceStep is a logical step indicating a resource will be replaced. This is comprised of three physical steps:
|
|
|
|
// a creation of the new resource, any number of intervening updates of dependents to the new resource, and then
|
|
|
|
// a deletion of the now-replaced old resource. This logical step is primarily here for tools and visualization.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
type ReplaceStep struct {
|
2019-07-01 21:34:19 +02:00
|
|
|
plan *Plan // the current plan.
|
|
|
|
old *resource.State // the state of the existing resource.
|
|
|
|
new *resource.State // the new state snapshot.
|
|
|
|
keys []resource.PropertyKey // the keys causing replacement.
|
|
|
|
diffs []resource.PropertyKey // the keys causing a diff.
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
|
|
|
|
pendingDelete bool // true if a pending deletion should happen.
|
2017-06-11 02:03:58 +02:00
|
|
|
}
|
|
|
|
|
2017-11-17 03:21:41 +01:00
|
|
|
var _ Step = (*ReplaceStep)(nil)
|
|
|
|
|
2018-04-26 02:20:08 +02:00
|
|
|
func NewReplaceStep(plan *Plan, old *resource.State, new *resource.State,
|
2019-07-01 21:34:19 +02:00
|
|
|
keys, diffs []resource.PropertyKey, detailedDiff map[string]plugin.PropertyDiff, pendingDelete bool) Step {
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assert(old.URN != "")
|
2017-10-15 12:52:04 +02:00
|
|
|
contract.Assert(old.ID != "" || !old.Custom)
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!old.Delete)
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assert(new.URN != "")
|
2018-08-03 23:06:00 +02:00
|
|
|
// contract.Assert(new.ID == "")
|
2017-10-05 23:08:35 +02:00
|
|
|
contract.Assert(!new.Delete)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
return &ReplaceStep{
|
2018-04-26 02:20:08 +02:00
|
|
|
plan: plan,
|
2017-12-10 17:37:22 +01:00
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
keys: keys,
|
2019-03-07 01:41:19 +01:00
|
|
|
diffs: diffs,
|
2019-07-01 21:34:19 +02:00
|
|
|
detailedDiff: detailedDiff,
|
2017-12-10 17:37:22 +01:00
|
|
|
pendingDelete: pendingDelete,
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2019-07-01 21:34:19 +02:00
|
|
|
func (s *ReplaceStep) Op() StepOp { return OpReplace }
|
|
|
|
func (s *ReplaceStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *ReplaceStep) Type() tokens.Type { return s.new.Type }
|
|
|
|
func (s *ReplaceStep) Provider() string { return s.new.Provider }
|
|
|
|
func (s *ReplaceStep) URN() resource.URN { return s.new.URN }
|
|
|
|
func (s *ReplaceStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *ReplaceStep) New() *resource.State { return s.new }
|
|
|
|
func (s *ReplaceStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *ReplaceStep) Keys() []resource.PropertyKey { return s.keys }
|
|
|
|
func (s *ReplaceStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *ReplaceStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
|
|
|
func (s *ReplaceStep) Logical() bool { return true }
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
func (s *ReplaceStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
2017-12-10 17:37:22 +01:00
|
|
|
// If this is a pending delete, we should have marked the old resource for deletion in the CreateReplacement step.
|
|
|
|
contract.Assert(!s.pendingDelete || s.old.Delete)
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, func() {}, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
2017-06-10 03:34:37 +02:00
|
|
|
|
2018-08-03 23:06:00 +02:00
|
|
|
// ReadStep is a step indicating that an existing resources will be "read" and projected into the Pulumi object
|
|
|
|
// model. Resources that are read are marked with the "External" bit which indicates to the engine that it does
|
|
|
|
// not own this resource's lifeycle.
|
|
|
|
//
|
|
|
|
// A resource with a given URN can transition freely between an "external" state and a non-external state. If
|
|
|
|
// a URN that was previously marked "External" (i.e. was the target of a ReadStep in a previous plan) is the
|
|
|
|
// target of a RegisterResource in the next plan, a CreateReplacement step will be issued to indicate the transition
|
|
|
|
// from external to owned. If a URN that was previously not marked "External" is the target of a ReadResource in the
|
|
|
|
// next plan, a ReadReplacement step will be issued to indicate the transition from owned to external.
|
|
|
|
type ReadStep struct {
|
|
|
|
plan *Plan // the plan that produced this read
|
|
|
|
event ReadResourceEvent // the event that should be signaled upon completion
|
|
|
|
old *resource.State // the old resource state, if one exists for this urn
|
|
|
|
new *resource.State // the new resource state, to be used to query the provider
|
|
|
|
replacing bool // whether or not the new resource is replacing the old resource
|
|
|
|
}
|
|
|
|
|
|
|
|
// NewReadStep creates a new Read step.
|
|
|
|
func NewReadStep(plan *Plan, event ReadResourceEvent, old *resource.State, new *resource.State) Step {
|
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assertf(new.External, "target of Read step must be marked External")
|
|
|
|
contract.Assertf(new.Custom, "target of Read step must be Custom")
|
|
|
|
|
|
|
|
// If Old was given, it's either an external resource or its ID is equal to the
|
|
|
|
// ID that we are preparing to read.
|
|
|
|
if old != nil {
|
|
|
|
contract.Assert(old.ID == new.ID || old.External)
|
|
|
|
}
|
|
|
|
|
|
|
|
return &ReadStep{
|
|
|
|
plan: plan,
|
|
|
|
event: event,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
replacing: false,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// NewReadReplacementStep creates a new Read step with the `replacing` flag set. When executed,
|
|
|
|
// it will pend deletion of the "old" resource, which must not be an external resource.
|
|
|
|
func NewReadReplacementStep(plan *Plan, event ReadResourceEvent, old *resource.State, new *resource.State) Step {
|
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assertf(new.External, "target of ReadReplacement step must be marked External")
|
|
|
|
contract.Assertf(new.Custom, "target of ReadReplacement step must be Custom")
|
|
|
|
contract.Assert(old != nil)
|
|
|
|
contract.Assertf(!old.External, "old target of ReadReplacement step must not be External")
|
|
|
|
return &ReadStep{
|
|
|
|
plan: plan,
|
|
|
|
event: event,
|
|
|
|
old: old,
|
|
|
|
new: new,
|
|
|
|
replacing: true,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *ReadStep) Op() StepOp {
|
|
|
|
if s.replacing {
|
|
|
|
return OpReadReplacement
|
|
|
|
}
|
|
|
|
|
|
|
|
return OpRead
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *ReadStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *ReadStep) Type() tokens.Type { return s.new.Type }
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
func (s *ReadStep) Provider() string { return s.new.Provider }
|
2018-08-03 23:06:00 +02:00
|
|
|
func (s *ReadStep) URN() resource.URN { return s.new.URN }
|
|
|
|
func (s *ReadStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *ReadStep) New() *resource.State { return s.new }
|
|
|
|
func (s *ReadStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *ReadStep) Logical() bool { return !s.replacing }
|
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
func (s *ReadStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
2018-08-03 23:06:00 +02:00
|
|
|
urn := s.new.URN
|
|
|
|
id := s.new.ID
|
|
|
|
|
2018-08-07 09:40:43 +02:00
|
|
|
var resourceError error
|
|
|
|
resourceStatus := resource.StatusOK
|
2018-08-03 23:06:00 +02:00
|
|
|
// Unlike most steps, Read steps run during previews. The only time
|
|
|
|
// we can't run is if the ID we are given is unknown.
|
2019-08-21 19:09:02 +02:00
|
|
|
if id == plugin.UnknownStringValue {
|
2018-08-08 21:06:20 +02:00
|
|
|
s.new.Outputs = resource.PropertyMap{}
|
|
|
|
} else {
|
2018-08-03 23:06:00 +02:00
|
|
|
prov, err := getProvider(s)
|
|
|
|
if err != nil {
|
2018-08-08 22:45:48 +02:00
|
|
|
return resource.StatusOK, nil, err
|
2018-08-03 23:06:00 +02:00
|
|
|
}
|
|
|
|
|
2019-03-11 21:50:00 +01:00
|
|
|
result, rst, err := prov.Read(urn, id, nil, s.new.Inputs)
|
2018-08-03 23:06:00 +02:00
|
|
|
if err != nil {
|
2018-08-07 09:40:43 +02:00
|
|
|
if rst != resource.StatusPartialFailure {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
resourceError = err
|
|
|
|
resourceStatus = rst
|
|
|
|
|
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
|
|
|
}
|
2018-08-03 23:06:00 +02:00
|
|
|
}
|
|
|
|
|
2019-08-21 19:09:02 +02:00
|
|
|
// If there is no such resource, return an error indicating as such.
|
|
|
|
if result.Outputs == nil {
|
|
|
|
return resource.StatusOK, nil, errors.Errorf("resource '%s' does not exist", id)
|
|
|
|
}
|
2019-03-11 21:50:00 +01:00
|
|
|
s.new.Outputs = result.Outputs
|
2019-08-21 19:09:02 +02:00
|
|
|
|
|
|
|
if result.ID != "" {
|
|
|
|
s.new.ID = result.ID
|
|
|
|
}
|
2018-08-03 23:06:00 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// If we were asked to replace an existing, non-External resource, pend the
|
|
|
|
// deletion here.
|
|
|
|
if s.replacing {
|
|
|
|
s.old.Delete = true
|
|
|
|
}
|
|
|
|
|
2018-08-08 22:45:48 +02:00
|
|
|
complete := func() { s.event.Done(&ReadResult{State: s.new}) }
|
2018-08-07 09:40:43 +02:00
|
|
|
if resourceError == nil {
|
|
|
|
return resourceStatus, complete, nil
|
|
|
|
}
|
|
|
|
return resourceStatus, complete, resourceError
|
2018-08-03 23:06:00 +02:00
|
|
|
}
|
|
|
|
|
2018-08-23 02:52:46 +02:00
|
|
|
// RefreshStep is a step used to track the progress of a refresh operation. A refresh operation updates the an existing
|
|
|
|
// resource by reading its current state from its provider plugin. These steps are not issued by the step generator;
|
|
|
|
// instead, they are issued by the plan executor as the optional first step in plan execution.
|
|
|
|
type RefreshStep struct {
|
|
|
|
plan *Plan // the plan that produced this refresh
|
|
|
|
old *resource.State // the old resource state, if one exists for this urn
|
|
|
|
new *resource.State // the new resource state, to be used to query the provider
|
|
|
|
done chan<- bool // the channel to use to signal completion, if any
|
|
|
|
}
|
|
|
|
|
|
|
|
// NewRefreshStep creates a new Refresh step.
|
|
|
|
func NewRefreshStep(plan *Plan, old *resource.State, done chan<- bool) Step {
|
|
|
|
contract.Assert(old != nil)
|
|
|
|
|
|
|
|
// NOTE: we set the new state to the old state by default so that we don't interpret step failures as deletes.
|
|
|
|
return &RefreshStep{
|
|
|
|
plan: plan,
|
|
|
|
old: old,
|
|
|
|
new: old,
|
|
|
|
done: done,
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *RefreshStep) Op() StepOp { return OpRefresh }
|
|
|
|
func (s *RefreshStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *RefreshStep) Type() tokens.Type { return s.old.Type }
|
|
|
|
func (s *RefreshStep) Provider() string { return s.old.Provider }
|
|
|
|
func (s *RefreshStep) URN() resource.URN { return s.old.URN }
|
|
|
|
func (s *RefreshStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *RefreshStep) New() *resource.State { return s.new }
|
|
|
|
func (s *RefreshStep) Res() *resource.State { return s.old }
|
|
|
|
func (s *RefreshStep) Logical() bool { return false }
|
|
|
|
|
|
|
|
// ResultOp returns the operation that corresponds to the change to this resource after reading its current state, if
|
|
|
|
// any.
|
|
|
|
func (s *RefreshStep) ResultOp() StepOp {
|
|
|
|
if s.new == nil {
|
|
|
|
return OpDelete
|
|
|
|
}
|
|
|
|
if s.new == s.old || s.old.Outputs.Diff(s.new.Outputs) == nil {
|
|
|
|
return OpSame
|
|
|
|
}
|
|
|
|
return OpUpdate
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *RefreshStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
|
|
|
var complete func()
|
|
|
|
if s.done != nil {
|
|
|
|
complete = func() { close(s.done) }
|
|
|
|
}
|
|
|
|
|
2019-08-16 20:04:03 +02:00
|
|
|
resourceID := s.old.ID
|
|
|
|
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
// Component, provider, and pending-replace resources never change with a refresh; just return the current state.
|
|
|
|
if !s.old.Custom || providers.IsProviderType(s.old.Type) || s.old.PendingReplacement {
|
2018-08-23 02:52:46 +02:00
|
|
|
return resource.StatusOK, complete, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
// For a custom resource, fetch the resource's provider and read the resource's current state.
|
|
|
|
prov, err := getProvider(s)
|
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
var initErrors []string
|
2019-08-16 20:04:03 +02:00
|
|
|
refreshed, rst, err := prov.Read(s.old.URN, resourceID, s.old.Inputs, s.old.Outputs)
|
2018-08-23 02:52:46 +02:00
|
|
|
if err != nil {
|
|
|
|
if rst != resource.StatusPartialFailure {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
2018-09-05 23:00:28 +02:00
|
|
|
initErrors = initErr.Reasons
|
2019-04-10 22:47:54 +02:00
|
|
|
|
|
|
|
// Partial failure SHOULD NOT cause refresh to fail. Instead:
|
|
|
|
//
|
|
|
|
// 1. Warn instead that during refresh we noticed the resource has become unhealthy.
|
|
|
|
// 2. Make sure the initialization errors are persisted in the state, so that the next
|
|
|
|
// `pulumi up` will surface them to the user.
|
|
|
|
err = nil
|
|
|
|
msg := fmt.Sprintf("Refreshed resource is in an unhealthy state:\n* %s", strings.Join(initErrors, "\n* "))
|
|
|
|
s.Plan().Diag().Warningf(diag.RawMessage(s.URN(), msg))
|
2018-08-23 02:52:46 +02:00
|
|
|
}
|
|
|
|
}
|
2019-03-11 21:50:00 +01:00
|
|
|
outputs := refreshed.Outputs
|
2018-08-23 02:52:46 +02:00
|
|
|
|
2019-03-11 21:50:00 +01:00
|
|
|
// If the provider specified new inputs for this resource, pick them up now. Otherwise, retain the current inputs.
|
|
|
|
inputs := s.old.Inputs
|
|
|
|
if refreshed.Inputs != nil {
|
|
|
|
inputs = refreshed.Inputs
|
|
|
|
}
|
|
|
|
|
|
|
|
if outputs != nil {
|
2019-08-16 20:04:03 +02:00
|
|
|
// There is a chance that the ID has changed. We want to allow this change to happen
|
|
|
|
// it will have changed already in the outputs, but we need to persist this change
|
|
|
|
// at a state level because the Id
|
|
|
|
if refreshed.ID != "" && refreshed.ID != resourceID {
|
|
|
|
logging.V(7).Infof("Refreshing ID; oldId=%s, newId=%s", resourceID, refreshed.ID)
|
|
|
|
resourceID = refreshed.ID
|
|
|
|
}
|
|
|
|
|
|
|
|
s.new = resource.NewState(s.old.Type, s.old.URN, s.old.Custom, s.old.Delete, resourceID, inputs, outputs,
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
s.old.Parent, s.old.Protect, s.old.External, s.old.Dependencies, initErrors, s.old.Provider,
|
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers
* Plumbing the CustomTimeouts through to the engine and adding test to show this
* Change the provider proto to include individual timeouts
* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface
* Change how the CustomTimeouts are sent across RPC
These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest
```
req=&pulumirpc.RegisterResourceRequest{
Type: "aws:s3/bucket:Bucket",
Name: "my-bucket",
Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
Custom: true,
Object: &structpb.Struct{},
Protect: false,
Dependencies: nil,
Provider: "",
PropertyDependencies: {},
DeleteBeforeReplace: false,
Version: "",
IgnoreChanges: nil,
AcceptSecrets: true,
AdditionalSecretOutputs: nil,
Aliases: nil,
CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{
Create: 300,
Update: 400,
Delete: 500,
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
},
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
}
```
* Changing the design to use strings
* CHANGELOG entry to include the CustomTimeouts work
* Changing custom timeouts to be passed around the engine as converted value
We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
|
|
|
s.old.PropertyDependencies, s.old.PendingReplacement, s.old.AdditionalSecretOutputs, s.old.Aliases,
|
|
|
|
&s.old.CustomTimeouts)
|
2018-08-23 02:52:46 +02:00
|
|
|
} else {
|
|
|
|
s.new = nil
|
|
|
|
}
|
|
|
|
|
|
|
|
return rst, complete, err
|
|
|
|
}
|
|
|
|
|
2019-07-12 20:12:01 +02:00
|
|
|
type ImportStep struct {
|
2019-07-31 18:39:07 +02:00
|
|
|
plan *Plan // the current plan.
|
|
|
|
reg RegisterResourceEvent // the registration intent to convey a URN back to.
|
|
|
|
original *resource.State // the original resource, if this is an import-replace.
|
|
|
|
old *resource.State // the state of the resource fetched from the provider.
|
|
|
|
new *resource.State // the newly computed state of the resource after importing.
|
|
|
|
replacing bool // true if we are replacing a Pulumi-managed resource.
|
|
|
|
diffs []resource.PropertyKey // any keys that differed between the user's program and the actual state.
|
|
|
|
detailedDiff map[string]plugin.PropertyDiff // the structured property diff.
|
|
|
|
ignoreChanges []string // a list of property paths to ignore when updating.
|
2019-07-12 20:12:01 +02:00
|
|
|
}
|
|
|
|
|
2019-07-31 18:39:07 +02:00
|
|
|
func NewImportStep(plan *Plan, reg RegisterResourceEvent, new *resource.State, ignoreChanges []string) Step {
|
2019-07-12 20:12:01 +02:00
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assert(new.URN != "")
|
|
|
|
contract.Assert(new.ID != "")
|
|
|
|
contract.Assert(new.Custom)
|
|
|
|
contract.Assert(!new.Delete)
|
|
|
|
contract.Assert(!new.External)
|
|
|
|
|
|
|
|
return &ImportStep{
|
2019-07-31 18:39:07 +02:00
|
|
|
plan: plan,
|
|
|
|
reg: reg,
|
|
|
|
new: new,
|
|
|
|
ignoreChanges: ignoreChanges,
|
2019-07-12 20:12:01 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-07-31 18:39:07 +02:00
|
|
|
func NewImportReplacementStep(plan *Plan, reg RegisterResourceEvent, original, new *resource.State,
|
|
|
|
ignoreChanges []string) Step {
|
|
|
|
|
2019-07-12 20:12:01 +02:00
|
|
|
contract.Assert(original != nil)
|
|
|
|
contract.Assert(new != nil)
|
|
|
|
contract.Assert(new.URN != "")
|
|
|
|
contract.Assert(new.ID != "")
|
|
|
|
contract.Assert(new.Custom)
|
|
|
|
contract.Assert(!new.Delete)
|
|
|
|
contract.Assert(!new.External)
|
|
|
|
|
|
|
|
return &ImportStep{
|
2019-07-31 18:39:07 +02:00
|
|
|
plan: plan,
|
|
|
|
reg: reg,
|
|
|
|
original: original,
|
|
|
|
new: new,
|
|
|
|
replacing: true,
|
|
|
|
ignoreChanges: ignoreChanges,
|
2019-07-12 20:12:01 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *ImportStep) Op() StepOp {
|
|
|
|
if s.replacing {
|
|
|
|
return OpImportReplacement
|
|
|
|
}
|
|
|
|
return OpImport
|
|
|
|
}
|
|
|
|
|
|
|
|
func (s *ImportStep) Plan() *Plan { return s.plan }
|
|
|
|
func (s *ImportStep) Type() tokens.Type { return s.new.Type }
|
|
|
|
func (s *ImportStep) Provider() string { return s.new.Provider }
|
|
|
|
func (s *ImportStep) URN() resource.URN { return s.new.URN }
|
|
|
|
func (s *ImportStep) Old() *resource.State { return s.old }
|
|
|
|
func (s *ImportStep) New() *resource.State { return s.new }
|
|
|
|
func (s *ImportStep) Res() *resource.State { return s.new }
|
|
|
|
func (s *ImportStep) Logical() bool { return !s.replacing }
|
|
|
|
func (s *ImportStep) Diffs() []resource.PropertyKey { return s.diffs }
|
|
|
|
func (s *ImportStep) DetailedDiff() map[string]plugin.PropertyDiff { return s.detailedDiff }
|
|
|
|
|
|
|
|
func (s *ImportStep) Apply(preview bool) (resource.Status, StepCompleteFunc, error) {
|
|
|
|
complete := func() { s.reg.Done(&RegisterResult{State: s.new}) }
|
|
|
|
|
|
|
|
// Read the current state of the resource to import. If the provider does not hand us back any inputs for the
|
|
|
|
// resource, it probably needs to be updated. If the resource does not exist at all, fail the import.
|
|
|
|
prov, err := getProvider(s)
|
|
|
|
if err != nil {
|
|
|
|
return resource.StatusOK, nil, err
|
|
|
|
}
|
|
|
|
read, rst, err := prov.Read(s.new.URN, s.new.ID, nil, nil)
|
|
|
|
if err != nil {
|
|
|
|
if initErr, isInitErr := err.(*plugin.InitError); isInitErr {
|
|
|
|
s.new.InitErrors = initErr.Reasons
|
|
|
|
} else {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
}
|
|
|
|
if read.Outputs == nil {
|
|
|
|
return rst, nil, errors.Errorf("resource '%v' does not exist", s.new.ID)
|
|
|
|
}
|
|
|
|
if read.Inputs == nil {
|
|
|
|
return resource.StatusOK, nil, errors.Errorf(
|
|
|
|
"provider does not support importing resources; please try updating the '%v' plugin",
|
|
|
|
s.new.URN.Type().Package())
|
|
|
|
}
|
2019-08-24 00:00:24 +02:00
|
|
|
if read.ID != "" {
|
|
|
|
s.new.ID = read.ID
|
|
|
|
}
|
2019-07-12 20:12:01 +02:00
|
|
|
s.new.Outputs = read.Outputs
|
|
|
|
|
|
|
|
// Magic up an old state so the frontend can display a proper diff. This state is the output of the just-executed
|
|
|
|
// `Read` combined with the resource identity and metadata from the desired state. This ensures that the only
|
|
|
|
// differences between the old and new states are between the inputs and outputs.
|
|
|
|
s.old = resource.NewState(s.new.Type, s.new.URN, s.new.Custom, false, s.new.ID, read.Inputs, read.Outputs,
|
|
|
|
s.new.Parent, s.new.Protect, false, s.new.Dependencies, s.new.InitErrors, s.new.Provider,
|
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers
* Plumbing the CustomTimeouts through to the engine and adding test to show this
* Change the provider proto to include individual timeouts
* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface
* Change how the CustomTimeouts are sent across RPC
These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest
```
req=&pulumirpc.RegisterResourceRequest{
Type: "aws:s3/bucket:Bucket",
Name: "my-bucket",
Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
Custom: true,
Object: &structpb.Struct{},
Protect: false,
Dependencies: nil,
Provider: "",
PropertyDependencies: {},
DeleteBeforeReplace: false,
Version: "",
IgnoreChanges: nil,
AcceptSecrets: true,
AdditionalSecretOutputs: nil,
Aliases: nil,
CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{
Create: 300,
Update: 400,
Delete: 500,
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
},
XXX_NoUnkeyedLiteral: struct {}{},
XXX_unrecognized: nil,
XXX_sizecache: 0,
}
```
* Changing the design to use strings
* CHANGELOG entry to include the CustomTimeouts work
* Changing custom timeouts to be passed around the engine as converted value
We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
|
|
|
s.new.PropertyDependencies, false, nil, nil, &s.new.CustomTimeouts)
|
2019-07-12 20:12:01 +02:00
|
|
|
|
|
|
|
// Check the user inputs using the provider inputs for defaults.
|
|
|
|
inputs, failures, err := prov.Check(s.new.URN, s.old.Inputs, s.new.Inputs, preview)
|
|
|
|
if err != nil {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
if issueCheckErrors(s.plan, s.new, s.new.URN, failures) {
|
|
|
|
return rst, nil, errors.New("one or more inputs failed to validate")
|
|
|
|
}
|
|
|
|
s.new.Inputs = inputs
|
|
|
|
|
|
|
|
// Diff the user inputs against the provider inputs. If there are any differences, fail the import.
|
2019-07-31 18:39:07 +02:00
|
|
|
diff, err := diffResource(s.new.URN, s.new.ID, s.old.Inputs, s.old.Outputs, s.new.Inputs, prov, preview,
|
|
|
|
s.ignoreChanges)
|
2019-07-12 20:12:01 +02:00
|
|
|
if err != nil {
|
|
|
|
return rst, nil, err
|
|
|
|
}
|
|
|
|
s.diffs, s.detailedDiff = diff.ChangedKeys, diff.DetailedDiff
|
|
|
|
|
|
|
|
if diff.Changes != plugin.DiffNone {
|
|
|
|
const message = "inputs to import do not match the existing resource"
|
|
|
|
|
|
|
|
if preview {
|
|
|
|
s.plan.ctx.Diag.Warningf(diag.StreamMessage(s.new.URN, message+"; importing this resource will fail", 0))
|
|
|
|
} else {
|
|
|
|
err = errors.New(message)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// If we were asked to replace an existing, non-External resource, pend the deletion here.
|
|
|
|
if err == nil && s.replacing {
|
|
|
|
s.original.Delete = true
|
|
|
|
}
|
|
|
|
|
|
|
|
return rst, complete, err
|
|
|
|
}
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
// StepOp represents the kind of operation performed by a step. It evaluates to its string label.
|
2017-06-10 03:34:37 +02:00
|
|
|
type StepOp string
|
|
|
|
|
|
|
|
const (
|
2019-01-31 22:48:44 +01:00
|
|
|
OpSame StepOp = "same" // nothing to do.
|
|
|
|
OpCreate StepOp = "create" // creating a new resource.
|
|
|
|
OpUpdate StepOp = "update" // updating an existing resource.
|
|
|
|
OpDelete StepOp = "delete" // deleting an existing resource.
|
|
|
|
OpReplace StepOp = "replace" // replacing a resource with a new one.
|
|
|
|
OpCreateReplacement StepOp = "create-replacement" // creating a new resource for a replacement.
|
|
|
|
OpDeleteReplaced StepOp = "delete-replaced" // deleting an existing resource after replacement.
|
|
|
|
OpRead StepOp = "read" // reading an existing resource.
|
|
|
|
OpReadReplacement StepOp = "read-replacement" // reading an existing resource for a replacement.
|
|
|
|
OpRefresh StepOp = "refresh" // refreshing an existing resource.
|
|
|
|
OpReadDiscard StepOp = "discard" // removing a resource that was read.
|
|
|
|
OpDiscardReplaced StepOp = "discard-replaced" // discarding a read resource that was replaced.
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
OpRemovePendingReplace StepOp = "remove-pending-replace" // removing a pending replace resource.
|
2019-07-12 20:12:01 +02:00
|
|
|
OpImport StepOp = "import" // import an existing resource.
|
|
|
|
OpImportReplacement StepOp = "import-replacement" // replace an existing resource with an imported resource.
|
2017-06-10 03:34:37 +02:00
|
|
|
)
|
|
|
|
|
|
|
|
// StepOps contains the full set of step operation types.
|
|
|
|
var StepOps = []StepOp{
|
2017-06-11 17:09:20 +02:00
|
|
|
OpSame,
|
2017-06-10 03:34:37 +02:00
|
|
|
OpCreate,
|
|
|
|
OpUpdate,
|
|
|
|
OpDelete,
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
OpReplace,
|
2017-08-06 19:05:51 +02:00
|
|
|
OpCreateReplacement,
|
|
|
|
OpDeleteReplaced,
|
2018-08-03 23:06:00 +02:00
|
|
|
OpRead,
|
|
|
|
OpReadReplacement,
|
2018-08-23 02:52:46 +02:00
|
|
|
OpRefresh,
|
2019-01-31 22:48:44 +01:00
|
|
|
OpReadDiscard,
|
|
|
|
OpDiscardReplaced,
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
OpRemovePendingReplace,
|
2019-07-12 20:12:01 +02:00
|
|
|
OpImport,
|
|
|
|
OpImportReplacement,
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// Color returns a suggested color for lines of this op type.
|
|
|
|
func (op StepOp) Color() string {
|
|
|
|
switch op {
|
2017-06-11 17:09:20 +02:00
|
|
|
case OpSame:
|
2017-11-17 03:21:41 +01:00
|
|
|
return colors.SpecUnimportant
|
2019-07-12 20:12:01 +02:00
|
|
|
case OpCreate, OpImport:
|
2017-08-06 19:05:51 +02:00
|
|
|
return colors.SpecCreate
|
2017-06-11 02:03:58 +02:00
|
|
|
case OpDelete:
|
2017-08-06 19:05:51 +02:00
|
|
|
return colors.SpecDelete
|
2017-06-10 03:34:37 +02:00
|
|
|
case OpUpdate:
|
2017-08-06 19:05:51 +02:00
|
|
|
return colors.SpecUpdate
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
case OpReplace:
|
2017-08-06 19:05:51 +02:00
|
|
|
return colors.SpecReplace
|
|
|
|
case OpCreateReplacement:
|
|
|
|
return colors.SpecCreateReplacement
|
|
|
|
case OpDeleteReplaced:
|
|
|
|
return colors.SpecDeleteReplaced
|
2018-08-03 23:06:00 +02:00
|
|
|
case OpRead:
|
2019-09-18 18:49:13 +02:00
|
|
|
return colors.SpecRead
|
2019-07-12 20:12:01 +02:00
|
|
|
case OpReadReplacement, OpImportReplacement:
|
2018-08-03 23:06:00 +02:00
|
|
|
return colors.SpecReplace
|
2018-08-23 02:52:46 +02:00
|
|
|
case OpRefresh:
|
|
|
|
return colors.SpecUpdate
|
2019-01-31 22:48:44 +01:00
|
|
|
case OpReadDiscard, OpDiscardReplaced:
|
|
|
|
return colors.SpecDelete
|
2017-06-10 03:34:37 +02:00
|
|
|
default:
|
2018-04-10 21:03:11 +02:00
|
|
|
contract.Failf("Unrecognized resource step op: '%v'", op)
|
2017-06-10 03:34:37 +02:00
|
|
|
return ""
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// Prefix returns a suggested prefix for lines of this op type.
|
|
|
|
func (op StepOp) Prefix() string {
|
2017-11-26 00:54:01 +01:00
|
|
|
return op.Color() + op.RawPrefix()
|
|
|
|
}
|
|
|
|
|
|
|
|
// RawPrefix returns the uncolorized prefix text.
|
|
|
|
func (op StepOp) RawPrefix() string {
|
2017-06-10 03:34:37 +02:00
|
|
|
switch op {
|
2017-08-30 03:24:12 +02:00
|
|
|
case OpSame:
|
2018-09-22 22:07:43 +02:00
|
|
|
return " "
|
2017-06-10 03:34:37 +02:00
|
|
|
case OpCreate:
|
2017-11-26 00:54:01 +01:00
|
|
|
return "+ "
|
2017-06-10 03:34:37 +02:00
|
|
|
case OpDelete:
|
2017-11-26 00:54:01 +01:00
|
|
|
return "- "
|
2017-06-10 03:34:37 +02:00
|
|
|
case OpUpdate:
|
2017-11-26 00:54:01 +01:00
|
|
|
return "~ "
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
case OpReplace:
|
2017-11-26 00:54:01 +01:00
|
|
|
return "+-"
|
2017-08-06 19:05:51 +02:00
|
|
|
case OpCreateReplacement:
|
2017-11-26 00:54:01 +01:00
|
|
|
return "++"
|
2017-08-06 19:05:51 +02:00
|
|
|
case OpDeleteReplaced:
|
2017-11-26 00:54:01 +01:00
|
|
|
return "--"
|
2018-08-03 23:06:00 +02:00
|
|
|
case OpRead:
|
2019-01-31 22:48:44 +01:00
|
|
|
return "> "
|
2018-08-03 23:06:00 +02:00
|
|
|
case OpReadReplacement:
|
2019-01-31 22:48:44 +01:00
|
|
|
return ">>"
|
2018-08-23 02:52:46 +02:00
|
|
|
case OpRefresh:
|
|
|
|
return "~ "
|
2019-01-31 22:48:44 +01:00
|
|
|
case OpReadDiscard:
|
|
|
|
return "< "
|
|
|
|
case OpDiscardReplaced:
|
|
|
|
return "<<"
|
2019-07-12 20:12:01 +02:00
|
|
|
case OpImport:
|
|
|
|
return "= "
|
|
|
|
case OpImportReplacement:
|
|
|
|
return "=>"
|
2017-06-10 03:34:37 +02:00
|
|
|
default:
|
|
|
|
contract.Failf("Unrecognized resource step op: %v", op)
|
|
|
|
return ""
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-09-28 01:58:34 +02:00
|
|
|
func (op StepOp) PastTense() string {
|
|
|
|
switch op {
|
2018-08-03 23:06:00 +02:00
|
|
|
case OpSame, OpCreate, OpDelete, OpReplace, OpCreateReplacement, OpDeleteReplaced, OpUpdate, OpReadReplacement:
|
2017-09-28 01:58:34 +02:00
|
|
|
return string(op) + "d"
|
2018-08-23 02:52:46 +02:00
|
|
|
case OpRefresh:
|
|
|
|
return "refreshed"
|
2018-08-03 23:06:00 +02:00
|
|
|
case OpRead:
|
|
|
|
return "read"
|
2019-01-31 22:48:44 +01:00
|
|
|
case OpReadDiscard, OpDiscardReplaced:
|
|
|
|
return "discarded"
|
2019-07-12 20:12:01 +02:00
|
|
|
case OpImport, OpImportReplacement:
|
|
|
|
return "imported"
|
2017-09-28 01:58:34 +02:00
|
|
|
default:
|
2017-11-29 19:06:51 +01:00
|
|
|
contract.Failf("Unexpected resource step op: %v", op)
|
2017-09-28 01:58:34 +02:00
|
|
|
return ""
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2017-06-10 03:34:37 +02:00
|
|
|
// Suffix returns a suggested suffix for lines of this op type.
|
|
|
|
func (op StepOp) Suffix() string {
|
2019-07-12 20:12:01 +02:00
|
|
|
switch op {
|
|
|
|
case OpCreateReplacement, OpUpdate, OpReplace, OpReadReplacement, OpRefresh, OpImportReplacement:
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
return colors.Reset // updates and replacements colorize individual lines; get has none
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
return ""
|
|
|
|
}
|
2017-11-21 02:38:09 +01:00
|
|
|
|
|
|
|
// getProvider fetches the provider for the given step.
|
|
|
|
func getProvider(s Step) (plugin.Provider, error) {
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
if providers.IsProviderType(s.Type()) {
|
|
|
|
return s.Plan().providers, nil
|
|
|
|
}
|
|
|
|
ref, err := providers.ParseReference(s.Provider())
|
|
|
|
if err != nil {
|
|
|
|
return nil, errors.Errorf("bad provider reference '%v' for resource %v: %v", s.Provider(), s.URN(), err)
|
|
|
|
}
|
|
|
|
provider, ok := s.Plan().GetProvider(ref)
|
|
|
|
if !ok {
|
|
|
|
return nil, errors.Errorf("unknown provider '%v' for resource %v", s.Provider(), s.URN())
|
|
|
|
}
|
|
|
|
return provider, nil
|
2017-11-21 02:38:09 +01:00
|
|
|
}
|