maxmustermann/pulumi
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
pulumi/pkg/engine/lifecycle_test.go

4619 lines
143 KiB
Go
Raw Normal View History

Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Copyright 2016-2018, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
package engine
import (
"context"
Add test for pulumi/pulumi#1770 (#1773) * Add test for pulumi/pulumi#1770 * CR: Add parallel test, do an update before refresh
2018-08-15 00:32:23 +02:00
"fmt"
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
"reflect"
"strconv"
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
"strings"
Fix provider cancellation. (#1914) We signal provider cancellation by hangning a goroutine off of the plan executor's parent context. To ensure clean shutdown, this goroutine also listens on a channel that closes once the plan has finished executing. Unfortunately, we were closing this channel too early, and the close was racing with the cancellation signal. These changes ensure that the channel closes after the plan has fully completed. Fixes #1906. Fixes pulumi/pulumi-kubernetes#185.
2018-09-11 00:18:25 +02:00
"sync"
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
"testing"
"github.com/blang/semver"
Retire pending deletions at start of plan (#1886) * Retire pending deletions at start of plan Instead of letting pending deletions pile up to be retired at the end of a plan, this commit eagerly disposes of any pending deletions that were pending at the end of the previous plan. This is a nice usability win and also reclaims an invariant that at most one resource with a given URN is live and at most one is pending deletion at any point in time. * Rebase against master * Fix a test issue arising from shared snapshots * CR feedback * plan -> replacement * Use ephemeral statuses to communicate deletions
2018-09-11 01:48:14 +02:00
"github.com/mitchellh/copystructure"
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
"github.com/pkg/errors"
"github.com/stretchr/testify/assert"
Validate type tokens before using them (#1904) * Validate type tokens before using them When registering or reading a resource, we take the type token given to us from the language host and assume that it's valid, which resulted in assertion failures in various places in the engine. This commit validates the format of type tokens given to us from the language host and issues an appropriate error if it's not valid. Along the way, this commit also improves the way that fatal exceptions are rendered in the Node language host. * Pre-allocate an exception for ReadResource * Fix integration test * CR Feedback This commit is a lower-impact change that fixes the bugs associated with invalid types on component resources and only checks that a type is valid on custom resources. * CR Take 2: Fix up IsProviderType instead of fixing call sites * Please gometalinter
2018-09-08 00:19:18 +02:00
"google.golang.org/grpc/codes"
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
"github.com/pulumi/pulumi/pkg/diag"
"github.com/pulumi/pulumi/pkg/diag/colors"
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
"github.com/pulumi/pulumi/pkg/resource"
"github.com/pulumi/pulumi/pkg/resource/config"
"github.com/pulumi/pulumi/pkg/resource/deploy"
"github.com/pulumi/pulumi/pkg/resource/deploy/deploytest"
"github.com/pulumi/pulumi/pkg/resource/deploy/providers"
"github.com/pulumi/pulumi/pkg/resource/plugin"
Allow resource IDs to change on reresh steps (#3087) * Allow resource IDs to change on reresh steps This is a requirement for us to be able to move forward with versions of the Terraform Azurerm provider. In v1.32.1, there was a state migration that changed the ID format of the azure table storage resource We used to have a check in place for old ID being equal to new ID. This has been changed now and we allow the change of ID to happen in the RefreshStep * Update pkg/resource/deploy/step.go Co-Authored-By: Pat Gavlin <pat@pulumi.com>
2019-08-16 20:04:03 +02:00
"github.com/pulumi/pulumi/pkg/secrets"
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
"github.com/pulumi/pulumi/pkg/tokens"
"github.com/pulumi/pulumi/pkg/util/cancel"
"github.com/pulumi/pulumi/pkg/util/contract"
"github.com/pulumi/pulumi/pkg/util/logging"
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
"github.com/pulumi/pulumi/pkg/util/result"
Validate type tokens before using them (#1904) * Validate type tokens before using them When registering or reading a resource, we take the type token given to us from the language host and assume that it's valid, which resulted in assertion failures in various places in the engine. This commit validates the format of type tokens given to us from the language host and issues an appropriate error if it's not valid. Along the way, this commit also improves the way that fatal exceptions are rendered in the Node language host. * Pre-allocate an exception for ReadResource * Fix integration test * CR Feedback This commit is a lower-impact change that fixes the bugs associated with invalid types on component resources and only checks that a type is valid on custom resources. * CR Take 2: Fix up IsProviderType instead of fixing call sites * Please gometalinter
2018-09-08 00:19:18 +02:00
"github.com/pulumi/pulumi/pkg/util/rpcutil/rpcerror"
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
"github.com/pulumi/pulumi/pkg/workspace"
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
combinations "github.com/mxschmitt/golang-combinations"
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
)
type JournalEntryKind int
const (
JournalEntryBegin JournalEntryKind = 0
JournalEntrySuccess JournalEntryKind = 1
JournalEntryFailure JournalEntryKind = 2
JournalEntryOutputs JournalEntryKind = 4
)
type JournalEntry struct {
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
Kind JournalEntryKind
Step deploy.Step
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
type Journal struct {
Entries []JournalEntry
events chan JournalEntry
cancel chan bool
done chan bool
}
func (j *Journal) Close() error {
close(j.cancel)
<-j.done
return nil
}
func (j *Journal) BeginMutation(step deploy.Step) (SnapshotMutation, error) {
select {
case j.events <- JournalEntry{Kind: JournalEntryBegin, Step: step}:
return j, nil
case <-j.cancel:
return nil, errors.New("journal closed")
}
}
func (j *Journal) End(step deploy.Step, success bool) error {
kind := JournalEntryFailure
if success {
kind = JournalEntrySuccess
}
select {
case j.events <- JournalEntry{Kind: kind, Step: step}:
return nil
case <-j.cancel:
return errors.New("journal closed")
}
}
func (j *Journal) RegisterResourceOutputs(step deploy.Step) error {
select {
case j.events <- JournalEntry{Kind: JournalEntryOutputs, Step: step}:
return nil
case <-j.cancel:
return errors.New("journal closed")
}
}
func (j *Journal) RecordPlugin(plugin workspace.PluginInfo) error {
return nil
}
func (j *Journal) Snap(base *deploy.Snapshot) *deploy.Snapshot {
// Build up a list of current resources by replaying the journal.
resources, dones := []*resource.State{}, make(map[*resource.State]bool)
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
ops, doneOps := []resource.Operation{}, make(map[*resource.State]bool)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
for _, e := range j.Entries {
logging.V(7).Infof("%v %v (%v)", e.Step.Op(), e.Step.URN(), e.Kind)
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
// Begin journal entries add pending operations to the snapshot. As we see success or failure
// entries, we'll record them in doneOps.
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
switch e.Kind {
case JournalEntryBegin:
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
switch e.Step.Op() {
case deploy.OpCreate, deploy.OpCreateReplacement:
ops = append(ops, resource.NewOperation(e.Step.New(), resource.OperationTypeCreating))
Use "discard" when deleting read resources (#2280) In general, a "delete" in Pulumi is destroying an actual physical resource. In the case of a read resource, however, the delete is merely removing the resource from the stack; the physical resource is not affected. These changes attempt to clarify this situation by using the term "discard" rather than "delete". Fixes #2015.
2019-01-31 22:48:44 +01:00
case deploy.OpDelete, deploy.OpDeleteReplaced, deploy.OpReadDiscard, deploy.OpDiscardReplaced:
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
ops = append(ops, resource.NewOperation(e.Step.Old(), resource.OperationTypeDeleting))
case deploy.OpRead, deploy.OpReadReplacement:
ops = append(ops, resource.NewOperation(e.Step.New(), resource.OperationTypeReading))
case deploy.OpUpdate:
ops = append(ops, resource.NewOperation(e.Step.New(), resource.OperationTypeUpdating))
Add support for importing existing resources. (#2893) A resource can be imported by setting the `import` property in the resource options bag when instantiating a resource. In order to successfully import a resource, its desired configuration (i.e. its inputs) must not differ from its actual configuration (i.e. its state) as calculated by the resource's provider. There are a few interesting state transitions hiding here when importing a resource: 1. No prior resource exists in the checkpoint file. In this case, the resource is simply imported. 2. An external resource exists in the checkpoint file. In this case, the resource is imported and the old external state is discarded. 3. A non-external resource exists in the checkpoint file and its ID is different from the ID to import. In this case, the new resource is imported and the old resource is deleted. 4. A non-external resource exists in the checkpoint file, but the ID is the same as the ID to import. In this case, the import ID is ignored and the resource is treated as it would be in all cases except for changes that would replace the resource. In that case, the step generator issues an error that indicates that the import ID should be removed: were we to move forward with the replace, the new state of the stack would fall under case (3), which is almost certainly not what the user intends. Fixes #1662.
2019-07-12 20:12:01 +02:00
case deploy.OpImport, deploy.OpImportReplacement:
ops = append(ops, resource.NewOperation(e.Step.New(), resource.OperationTypeImporting))
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
}
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
case JournalEntryFailure, JournalEntrySuccess:
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
switch e.Step.Op() {
Use "discard" when deleting read resources (#2280) In general, a "delete" in Pulumi is destroying an actual physical resource. In the case of a read resource, however, the delete is merely removing the resource from the stack; the physical resource is not affected. These changes attempt to clarify this situation by using the term "discard" rather than "delete". Fixes #2015.
2019-01-31 22:48:44 +01:00
// nolint: lll
Add support for importing existing resources. (#2893) A resource can be imported by setting the `import` property in the resource options bag when instantiating a resource. In order to successfully import a resource, its desired configuration (i.e. its inputs) must not differ from its actual configuration (i.e. its state) as calculated by the resource's provider. There are a few interesting state transitions hiding here when importing a resource: 1. No prior resource exists in the checkpoint file. In this case, the resource is simply imported. 2. An external resource exists in the checkpoint file. In this case, the resource is imported and the old external state is discarded. 3. A non-external resource exists in the checkpoint file and its ID is different from the ID to import. In this case, the new resource is imported and the old resource is deleted. 4. A non-external resource exists in the checkpoint file, but the ID is the same as the ID to import. In this case, the import ID is ignored and the resource is treated as it would be in all cases except for changes that would replace the resource. In that case, the step generator issues an error that indicates that the import ID should be removed: were we to move forward with the replace, the new state of the stack would fall under case (3), which is almost certainly not what the user intends. Fixes #1662.
2019-07-12 20:12:01 +02:00
case deploy.OpCreate, deploy.OpCreateReplacement, deploy.OpRead, deploy.OpReadReplacement, deploy.OpUpdate,
deploy.OpImport, deploy.OpImportReplacement:
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
doneOps[e.Step.New()] = true
Use "discard" when deleting read resources (#2280) In general, a "delete" in Pulumi is destroying an actual physical resource. In the case of a read resource, however, the delete is merely removing the resource from the stack; the physical resource is not affected. These changes attempt to clarify this situation by using the term "discard" rather than "delete". Fixes #2015.
2019-01-31 22:48:44 +01:00
case deploy.OpDelete, deploy.OpDeleteReplaced, deploy.OpReadDiscard, deploy.OpDiscardReplaced:
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
doneOps[e.Step.Old()] = true
}
}
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
// Now mark resources done as necessary.
if e.Kind == JournalEntrySuccess {
switch e.Step.Op() {
case deploy.OpSame, deploy.OpUpdate:
resources = append(resources, e.Step.New())
Emit reads for external resources when refreshing (#1749) * Emit reads for external resources when refreshing Fixes pulumi/pulumi#1744. This commit educates the refresh source about external resources. If a refresh source encounters a resource with the External bit set, it'll send a Read event to the engine and the engine will process it accordingly. * CR: save last event channel instead of last event, style fixes
2018-08-09 21:45:39 +02:00
dones[e.Step.Old()] = true
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
case deploy.OpCreate, deploy.OpCreateReplacement:
resources = append(resources, e.Step.New())
if old := e.Step.Old(); old != nil && old.PendingReplacement {
dones[old] = true
}
Use "discard" when deleting read resources (#2280) In general, a "delete" in Pulumi is destroying an actual physical resource. In the case of a read resource, however, the delete is merely removing the resource from the stack; the physical resource is not affected. These changes attempt to clarify this situation by using the term "discard" rather than "delete". Fixes #2015.
2019-01-31 22:48:44 +01:00
case deploy.OpDelete, deploy.OpDeleteReplaced, deploy.OpReadDiscard, deploy.OpDiscardReplaced:
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
if old := e.Step.Old(); !old.PendingReplacement {
dones[old] = true
}
case deploy.OpReplace:
// do nothing.
case deploy.OpRead, deploy.OpReadReplacement:
resources = append(resources, e.Step.New())
if e.Step.Old() != nil {
dones[e.Step.Old()] = true
}
case deploy.OpRemovePendingReplace:
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
dones[e.Step.Old()] = true
Add support for importing existing resources. (#2893) A resource can be imported by setting the `import` property in the resource options bag when instantiating a resource. In order to successfully import a resource, its desired configuration (i.e. its inputs) must not differ from its actual configuration (i.e. its state) as calculated by the resource's provider. There are a few interesting state transitions hiding here when importing a resource: 1. No prior resource exists in the checkpoint file. In this case, the resource is simply imported. 2. An external resource exists in the checkpoint file. In this case, the resource is imported and the old external state is discarded. 3. A non-external resource exists in the checkpoint file and its ID is different from the ID to import. In this case, the new resource is imported and the old resource is deleted. 4. A non-external resource exists in the checkpoint file, but the ID is the same as the ID to import. In this case, the import ID is ignored and the resource is treated as it would be in all cases except for changes that would replace the resource. In that case, the step generator issues an error that indicates that the import ID should be removed: were we to move forward with the replace, the new state of the stack would fall under case (3), which is almost certainly not what the user intends. Fixes #1662.
2019-07-12 20:12:01 +02:00
case deploy.OpImport, deploy.OpImportReplacement:
resources = append(resources, e.Step.New())
dones[e.Step.New()] = true
Emit reads for external resources when refreshing (#1749) * Emit reads for external resources when refreshing Fixes pulumi/pulumi#1744. This commit educates the refresh source about external resources. If a refresh source encounters a resource with the External bit set, it'll send a Read event to the engine and the engine will process it accordingly. * CR: save last event channel instead of last event, style fixes
2018-08-09 21:45:39 +02:00
}
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
}
// Append any resources from the base snapshot that were not produced by the current snapshot.
// See backend.SnapshotManager.snap for why this works.
if base != nil {
for _, res := range base.Resources {
if !dones[res] {
resources = append(resources, res)
}
}
}
Add a list of in-flight operations to the deployment (#1759) * Add a list of in-flight operations to the deployment This commit augments 'DeploymentV2' with a list of operations that are currently in flight. This information is used by the engine to keep track of whether or not a particular deployment is in a valid state. The SnapshotManager is responsible for inserting and removing operations from the in-flight operation list. When the engine registers an intent to perform an operation, SnapshotManager inserts an Operation into this list and saves it to the snapshot. When an operation completes, the SnapshotManager removes it from the snapshot. From this, the engine can infer that if it ever sees a deployment with pending operations, the Pulumi CLI must have crashed or otherwise abnormally terminated before seeing whether or not an operation completed successfully. To remedy this state, this commit also adds code to 'pulumi stack import' that clears all pending operations from a deployment, as well as code to plan generation that will reject any deployments that have pending operations present. At the CLI level, if we see that we are in a state where pending operations were in-flight when the engine died, we'll issue a human-friendly error message that indicates which resources are in a bad state and how to recover their stack. * CR: Multi-line string literals, renaming in-flight -> pending * CR: Add enum to apitype for operation type, also name status -> type for clarity * Fix the yaml type * Fix missed renames * Add implementation for lifecycle_test.go * Rebase against master
2018-08-11 06:39:59 +02:00
// Append any pending operations.
var operations []resource.Operation
for _, op := range ops {
if !doneOps[op.Resource] {
operations = append(operations, op)
}
}
Retain the SecretsManager that was used to deserialize a deployment We have many cases where we want to do the following: deployment -> snapshot -> process snapshot -> deployment We now retain information in the snapshot about the secrets manager that was used to construct it, so in these round trip cases, we can re-use the existing manager.
2019-04-24 21:13:00 +02:00
// If we have a base snapshot, copy over its secrets manager.
var secretsManager secrets.Manager
if base != nil {
secretsManager = base.SecretsManager
}
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
manifest := deploy.Manifest{}
manifest.Magic = manifest.NewMagic()
Retain the SecretsManager that was used to deserialize a deployment We have many cases where we want to do the following: deployment -> snapshot -> process snapshot -> deployment We now retain information in the snapshot about the secrets manager that was used to construct it, so in these round trip cases, we can re-use the existing manager.
2019-04-24 21:13:00 +02:00
return deploy.NewSnapshot(manifest, secretsManager, resources, operations)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
func (j *Journal) SuccessfulSteps() []deploy.Step {
var steps []deploy.Step
for _, entry := range j.Entries {
if entry.Kind == JournalEntrySuccess {
steps = append(steps, entry.Step)
}
}
return steps
}
type StepSummary struct {
Op deploy.StepOp
URN resource.URN
}
func AssertSameSteps(t *testing.T, expected []StepSummary, actual []deploy.Step) bool {
assert.Equal(t, len(expected), len(actual))
for _, exp := range expected {
act := actual[0]
actual = actual[1:]
if !assert.Equal(t, exp.Op, act.Op()) || !assert.Equal(t, exp.URN, act.URN()) {
return false
}
}
return true
}
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
func newJournal() *Journal {
j := &Journal{
events: make(chan JournalEntry),
cancel: make(chan bool),
done: make(chan bool),
}
go func() {
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
for {
select {
case <-j.cancel:
close(j.done)
return
case e := <-j.events:
j.Entries = append(j.Entries, e)
}
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
}()
return j
}
type updateInfo struct {
project workspace.Project
target deploy.Target
}
func (u *updateInfo) GetRoot() string {
return ""
}
func (u *updateInfo) GetProject() *workspace.Project {
return &u.project
}
func (u *updateInfo) GetTarget() *deploy.Target {
return &u.target
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
type TestOp func(UpdateInfo, *Context, UpdateOptions, bool) (ResourceChanges, result.Result)
type ValidateFunc func(project workspace.Project, target deploy.Target, j *Journal,
events []Event, res result.Result) result.Result
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
func (op TestOp) Run(project workspace.Project, target deploy.Target, opts UpdateOptions,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
dryRun bool, backendClient deploy.BackendClient, validate ValidateFunc) (*deploy.Snapshot, result.Result) {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
return op.RunWithContext(context.Background(), project, target, opts, dryRun, backendClient, validate)
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
func (op TestOp) RunWithContext(
callerCtx context.Context, project workspace.Project,
target deploy.Target, opts UpdateOptions, dryRun bool,
backendClient deploy.BackendClient, validate ValidateFunc) (*deploy.Snapshot, result.Result) {
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Create an appropriate update info and context.
info := &updateInfo{project: project, target: target}
Fix provider cancellation. (#1914) We signal provider cancellation by hangning a goroutine off of the plan executor's parent context. To ensure clean shutdown, this goroutine also listens on a channel that closes once the plan has finished executing. Unfortunately, we were closing this channel too early, and the close was racing with the cancellation signal. These changes ensure that the channel closes after the plan has fully completed. Fixes #1906. Fixes pulumi/pulumi-kubernetes#185.
2018-09-11 00:18:25 +02:00
cancelCtx, cancelSrc := cancel.NewContext(context.Background())
done := make(chan bool)
defer close(done)
go func() {
select {
case <-callerCtx.Done():
cancelSrc.Cancel()
case <-done:
}
}()
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
events := make(chan Event)
journal := newJournal()
ctx := &Context{
Cancel: cancelCtx,
Events: events,
SnapshotManager: journal,
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
BackendClient: backendClient,
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
// Begin draining events.
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
var firedEvents []Event
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
go func() {
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
for e := range events {
firedEvents = append(firedEvents, e)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
}()
// Run the step and its validator.
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
_, res := op(info, ctx, opts, dryRun)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
contract.IgnoreClose(journal)
if dryRun {
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return nil, res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
if validate != nil {
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
res = validate(project, target, journal, firedEvents, res)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
snap := journal.Snap(target.Snapshot)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
if res == nil && snap != nil {
res = result.WrapIfNonNil(snap.VerifyIntegrity())
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return snap, res
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
type TestStep struct {
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
Op TestOp
ExpectFailure bool
SkipPreview bool
Validate ValidateFunc
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
type TestPlan struct {
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
Project string
Stack string
Runtime string
Config config.Map
Decrypter config.Decrypter
BackendClient deploy.BackendClient
Options UpdateOptions
Steps []TestStep
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
Adopt golangci-lint and address issues We run the same suite of changes that we did on gometalinter. This ended up catching a few new issues, some of which were addressed and some of which were baselined.
2018-11-05 22:36:35 +01:00
//nolint: goconst
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
func (p *TestPlan) getNames() (stack tokens.QName, project tokens.PackageName, runtime string) {
project = tokens.PackageName(p.Project)
if project == "" {
project = "test"
}
runtime = p.Runtime
if runtime == "" {
runtime = "test"
}
stack = tokens.QName(p.Stack)
if stack == "" {
stack = "test"
}
return stack, project, runtime
}
func (p *TestPlan) NewURN(typ tokens.Type, name string, parent resource.URN) resource.URN {
stack, project, _ := p.getNames()
var pt tokens.Type
if parent != "" {
pt = parent.Type()
}
return resource.NewURN(stack, project, pt, typ, tokens.QName(name))
}
func (p *TestPlan) NewProviderURN(pkg tokens.Package, name string, parent resource.URN) resource.URN {
return p.NewURN(providers.MakeProviderType(pkg), name, parent)
}
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
func (p *TestPlan) GetProject() workspace.Project {
_, projectName, runtime := p.getNames()
return workspace.Project{
Tidy up some data structures (#2135) In preparation for some workspace restructuring, I decided to scratch a few itches of my own in the code: * Change project's RuntimeInfo field to just Runtime, to match the serialized name in JSON/YAML. * Eliminate the no-longer-used Context and NoDefaultIgnores fields on project, and all of the associated legacy PPC-related code. * Eliminate the no-longer-used IgnoreFile constant. * Remove a bunch of "// nolint: lll" annotations, and simply format the structures with comments on dedicated lines, to avoid overly lengthy lines and lint suppressions. * Mark Dependencies and InitErrors as `omitempty` in the JSON serialization directives for CheckpointV2 files. This was done for the YAML directives, but (presumably accidentally) omitted for JSON.
2018-11-01 16:28:11 +01:00
Name: projectName,
Runtime: workspace.NewProjectRuntimeInfo(runtime, nil),
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
}
}
func (p *TestPlan) GetTarget(snapshot *deploy.Snapshot) deploy.Target {
stack, _, _ := p.getNames()
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
cfg := p.Config
if cfg == nil {
cfg = config.Map{}
}
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
return deploy.Target{
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
Name: stack,
Config: cfg,
Decrypter: p.Decrypter,
Snapshot: snapshot,
}
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
func assertIsErrorOrBailResult(t *testing.T, res result.Result) {
assert.NotNil(t, res)
}
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
func (p *TestPlan) Run(t *testing.T, snapshot *deploy.Snapshot) *deploy.Snapshot {
Retire pending deletions at start of plan (#1886) * Retire pending deletions at start of plan Instead of letting pending deletions pile up to be retired at the end of a plan, this commit eagerly disposes of any pending deletions that were pending at the end of the previous plan. This is a nice usability win and also reclaims an invariant that at most one resource with a given URN is live and at most one is pending deletion at any point in time. * Rebase against master * Fix a test issue arising from shared snapshots * CR feedback * plan -> replacement * Use ephemeral statuses to communicate deletions
2018-09-11 01:48:14 +02:00
project := p.GetProject()
snap := snapshot
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
for _, step := range p.Steps {
Retire pending deletions at start of plan (#1886) * Retire pending deletions at start of plan Instead of letting pending deletions pile up to be retired at the end of a plan, this commit eagerly disposes of any pending deletions that were pending at the end of the previous plan. This is a nice usability win and also reclaims an invariant that at most one resource with a given URN is live and at most one is pending deletion at any point in time. * Rebase against master * Fix a test issue arising from shared snapshots * CR feedback * plan -> replacement * Use ephemeral statuses to communicate deletions
2018-09-11 01:48:14 +02:00
// note: it's really important that the preview and update operate on different snapshots. the engine can and
// does mutate the snapshot in-place, even in previews, and sharing a snapshot between preview and update can
// cause state changes from the preview to persist even when doing an update.
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
if !step.SkipPreview {
Retire pending deletions at start of plan (#1886) * Retire pending deletions at start of plan Instead of letting pending deletions pile up to be retired at the end of a plan, this commit eagerly disposes of any pending deletions that were pending at the end of the previous plan. This is a nice usability win and also reclaims an invariant that at most one resource with a given URN is live and at most one is pending deletion at any point in time. * Rebase against master * Fix a test issue arising from shared snapshots * CR feedback * plan -> replacement * Use ephemeral statuses to communicate deletions
2018-09-11 01:48:14 +02:00
previewSnap := CloneSnapshot(t, snap)
previewTarget := p.GetTarget(previewSnap)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
_, res := step.Op.Run(project, previewTarget, p.Options, true, p.BackendClient, step.Validate)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
if step.ExpectFailure {
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
assertIsErrorOrBailResult(t, res)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
continue
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
assert.Nil(t, res)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
var res result.Result
Retire pending deletions at start of plan (#1886) * Retire pending deletions at start of plan Instead of letting pending deletions pile up to be retired at the end of a plan, this commit eagerly disposes of any pending deletions that were pending at the end of the previous plan. This is a nice usability win and also reclaims an invariant that at most one resource with a given URN is live and at most one is pending deletion at any point in time. * Rebase against master * Fix a test issue arising from shared snapshots * CR feedback * plan -> replacement * Use ephemeral statuses to communicate deletions
2018-09-11 01:48:14 +02:00
target := p.GetTarget(snap)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
snap, res = step.Op.Run(project, target, p.Options, false, p.BackendClient, step.Validate)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
if step.ExpectFailure {
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
assertIsErrorOrBailResult(t, res)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
continue
}
Delete specific target (#3244)
2019-09-20 04:28:14 +02:00
if res != nil {
if res.IsBail() {
t.Logf("Got unexpected bail result")
t.FailNow()
} else {
t.Logf("Got unexpected error result: %v", res.Error())
t.FailNow()
}
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
assert.Nil(t, res)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
Retire pending deletions at start of plan (#1886) * Retire pending deletions at start of plan Instead of letting pending deletions pile up to be retired at the end of a plan, this commit eagerly disposes of any pending deletions that were pending at the end of the previous plan. This is a nice usability win and also reclaims an invariant that at most one resource with a given URN is live and at most one is pending deletion at any point in time. * Rebase against master * Fix a test issue arising from shared snapshots * CR feedback * plan -> replacement * Use ephemeral statuses to communicate deletions
2018-09-11 01:48:14 +02:00
return snap
}
// CloneSnapshot makes a deep copy of the given snapshot and returns a pointer to the clone.
func CloneSnapshot(t *testing.T, snap *deploy.Snapshot) *deploy.Snapshot {
t.Helper()
if snap != nil {
copiedSnap := copystructure.Must(copystructure.Copy(*snap)).(deploy.Snapshot)
assert.True(t, reflect.DeepEqual(*snap, copiedSnap))
return &copiedSnap
}
return snap
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
func MakeBasicLifecycleSteps(t *testing.T, resCount int) []TestStep {
return []TestStep{
// Initial update
{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Should see only creates.
for _, entry := range j.Entries {
assert.Equal(t, deploy.OpCreate, entry.Step.Op())
}
assert.Len(t, j.Snap(target.Snapshot).Resources, resCount)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
},
// No-op refresh
{
Op: Refresh,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
// Should see only refresh-sames.
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
for _, entry := range j.Entries {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
assert.Equal(t, deploy.OpSame, entry.Step.(*deploy.RefreshStep).ResultOp())
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
assert.Len(t, j.Snap(target.Snapshot).Resources, resCount)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
},
// No-op update
{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Should see only sames.
for _, entry := range j.Entries {
assert.Equal(t, deploy.OpSame, entry.Step.Op())
}
assert.Len(t, j.Snap(target.Snapshot).Resources, resCount)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
},
// No-op refresh
{
Op: Refresh,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
// Should see only referesh-sames.
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
for _, entry := range j.Entries {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
assert.Equal(t, deploy.OpSame, entry.Step.(*deploy.RefreshStep).ResultOp())
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
assert.Len(t, j.Snap(target.Snapshot).Resources, resCount)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
},
// Destroy
{
Op: Destroy,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Should see only deletes.
for _, entry := range j.Entries {
Use "discard" when deleting read resources (#2280) In general, a "delete" in Pulumi is destroying an actual physical resource. In the case of a read resource, however, the delete is merely removing the resource from the stack; the physical resource is not affected. These changes attempt to clarify this situation by using the term "discard" rather than "delete". Fixes #2015.
2019-01-31 22:48:44 +01:00
switch entry.Step.Op() {
case deploy.OpDelete, deploy.OpReadDiscard:
// ok
default:
assert.Fail(t, "expected OpDelete or OpReadDiscard")
}
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
assert.Len(t, j.Snap(target.Snapshot).Resources, 0)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
},
// No-op refresh
{
Op: Refresh,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.Len(t, j.Entries, 0)
assert.Len(t, j.Snap(target.Snapshot).Resources, 0)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
},
}
}
func TestEmptyProgramLifecycle(t *testing.T) {
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, _ *deploytest.ResourceMonitor) error {
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: MakeBasicLifecycleSteps(t, 0),
}
p.Run(t, nil)
}
func TestSingleResourceDefaultProviderLifecycle(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: MakeBasicLifecycleSteps(t, 2),
}
p.Run(t, nil)
}
func TestSingleResourceExplicitProviderLifecycle(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
provURN, provID, _, err := monitor.RegisterResource(providers.MakeProviderType("pkgA"), "provA", true)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
if provID == "" {
provID = providers.UnknownID
}
provRef, err := providers.NewReference(provURN, provID)
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err = monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Provider: provRef.String(),
})
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: MakeBasicLifecycleSteps(t, 2),
}
p.Run(t, nil)
}
func TestSingleResourceDefaultProviderUpgrade(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
}
provURN := p.NewProviderURN("pkgA", "default", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Create an old snapshot with an existing copy of the single resource and no providers.
old := &deploy.Snapshot{
Resources: []*resource.State{{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "0",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
}},
}
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
isRefresh := false
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
validate := func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Should see only sames: the default provider should be injected into the old state before the update
// runs.
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN, resURN:
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
expect := deploy.OpSame
if isRefresh {
expect = deploy.OpRefresh
}
assert.Equal(t, expect, entry.Step.Op())
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
default:
t.Fatalf("unexpected resource %v", urn)
}
}
assert.Len(t, j.Snap(target.Snapshot).Resources, 2)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
}
// Run a single update step using the base snapshot.
p.Steps = []TestStep{{Op: Update, Validate: validate}}
p.Run(t, old)
// Run a single refresh step using the base snapshot.
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
isRefresh = true
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p.Steps = []TestStep{{Op: Refresh, Validate: validate}}
p.Run(t, old)
// Run a single destroy step using the base snapshot.
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
isRefresh = false
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p.Steps = []TestStep{{
Op: Destroy,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Should see two deletes: the default provider should be injected into the old state before the update
// runs.
deleted := make(map[resource.URN]bool)
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN, resURN:
deleted[urn] = true
assert.Equal(t, deploy.OpDelete, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
assert.Len(t, deleted, 2)
assert.Len(t, j.Snap(target.Snapshot).Resources, 0)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
}}
p.Run(t, old)
// Run a partial lifecycle using the base snapshot, skipping the initial update step.
p.Steps = MakeBasicLifecycleSteps(t, 2)[1:]
p.Run(t, old)
}
func TestSingleResourceDefaultProviderReplace(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
Flow `allowUnknows` for Diff/Check Config We pass this information for Diff and Check on specific resources, so we can correctly block unknows from flowing to plugins during applies.
2019-05-23 19:54:18 +02:00
DiffConfigF: func(urn resource.URN, olds, news resource.PropertyMap,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
ignoreChanges []string) (plugin.DiffResult, error) {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Always require replacement.
keys := []resource.PropertyKey{}
for k := range news {
keys = append(keys, k)
}
return plugin.DiffResult{ReplaceKeys: keys}, nil
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Config: config.Map{
config.MustMakeKey("pkgA", "foo"): config.NewValue("bar"),
},
}
// Build a basic lifecycle.
steps := MakeBasicLifecycleSteps(t, 2)
// Run the lifecycle through its no-op update+refresh.
p.Steps = steps[:4]
snap := p.Run(t, nil)
// Change the config and run an update. We expect everything to require replacement.
p.Config[config.MustMakeKey("pkgA", "foo")] = config.NewValue("baz")
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
provURN := p.NewProviderURN("pkgA", "default", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Look for replace steps on the provider and the resource.
replacedProvider, replacedResource := false, false
for _, entry := range j.Entries {
if entry.Kind != JournalEntrySuccess || entry.Step.Op() != deploy.OpDeleteReplaced {
continue
}
switch urn := entry.Step.URN(); urn {
case provURN:
replacedProvider = true
case resURN:
replacedResource = true
default:
t.Fatalf("unexpected resource %v", urn)
}
}
assert.True(t, replacedProvider)
assert.True(t, replacedResource)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
}}
Retire pending deletions at start of plan (#1886) * Retire pending deletions at start of plan Instead of letting pending deletions pile up to be retired at the end of a plan, this commit eagerly disposes of any pending deletions that were pending at the end of the previous plan. This is a nice usability win and also reclaims an invariant that at most one resource with a given URN is live and at most one is pending deletion at any point in time. * Rebase against master * Fix a test issue arising from shared snapshots * CR feedback * plan -> replacement * Use ephemeral statuses to communicate deletions
2018-09-11 01:48:14 +02:00
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
snap = p.Run(t, snap)
// Resume the lifecycle with another no-op update.
p.Steps = steps[2:]
p.Run(t, snap)
}
func TestSingleResourceExplicitProviderReplace(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
Flow `allowUnknows` for Diff/Check Config We pass this information for Diff and Check on specific resources, so we can correctly block unknows from flowing to plugins during applies.
2019-05-23 19:54:18 +02:00
DiffConfigF: func(urn resource.URN, olds, news resource.PropertyMap,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
ignoreChanges []string) (plugin.DiffResult, error) {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Always require replacement.
keys := []resource.PropertyKey{}
for k := range news {
keys = append(keys, k)
}
return plugin.DiffResult{ReplaceKeys: keys}, nil
},
}, nil
}),
}
providerInputs := resource.PropertyMap{
resource.PropertyKey("foo"): resource.NewStringProperty("bar"),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
provURN, provID, _, err := monitor.RegisterResource(providers.MakeProviderType("pkgA"), "provA", true,
deploytest.ResourceOptions{Inputs: providerInputs})
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
if provID == "" {
provID = providers.UnknownID
}
provRef, err := providers.NewReference(provURN, provID)
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err = monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Provider: provRef.String(),
})
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
}
// Build a basic lifecycle.
steps := MakeBasicLifecycleSteps(t, 2)
// Run the lifecycle through its no-op update+refresh.
p.Steps = steps[:4]
snap := p.Run(t, nil)
// Change the config and run an update. We expect everything to require replacement.
providerInputs[resource.PropertyKey("foo")] = resource.NewStringProperty("baz")
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
provURN := p.NewProviderURN("pkgA", "provA", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Look for replace steps on the provider and the resource.
replacedProvider, replacedResource := false, false
for _, entry := range j.Entries {
if entry.Kind != JournalEntrySuccess || entry.Step.Op() != deploy.OpDeleteReplaced {
continue
}
switch urn := entry.Step.URN(); urn {
case provURN:
replacedProvider = true
case resURN:
replacedResource = true
default:
t.Fatalf("unexpected resource %v", urn)
}
}
assert.True(t, replacedProvider)
assert.True(t, replacedResource)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
}}
snap = p.Run(t, snap)
// Resume the lifecycle with another no-op update.
p.Steps = steps[2:]
p.Run(t, snap)
}
func TestSingleResourceExplicitProviderDeleteBeforeReplace(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
Flow `allowUnknows` for Diff/Check Config We pass this information for Diff and Check on specific resources, so we can correctly block unknows from flowing to plugins during applies.
2019-05-23 19:54:18 +02:00
DiffConfigF: func(urn resource.URN, olds, news resource.PropertyMap,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
ignoreChanges []string) (plugin.DiffResult, error) {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
// Always require replacement.
keys := []resource.PropertyKey{}
for k := range news {
keys = append(keys, k)
}
return plugin.DiffResult{ReplaceKeys: keys, DeleteBeforeReplace: true}, nil
},
}, nil
}),
}
providerInputs := resource.PropertyMap{
resource.PropertyKey("foo"): resource.NewStringProperty("bar"),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
provURN, provID, _, err := monitor.RegisterResource(providers.MakeProviderType("pkgA"), "provA", true,
deploytest.ResourceOptions{Inputs: providerInputs})
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
if provID == "" {
provID = providers.UnknownID
}
provRef, err := providers.NewReference(provURN, provID)
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err = monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Provider: provRef.String(),
})
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
}
// Build a basic lifecycle.
steps := MakeBasicLifecycleSteps(t, 2)
// Run the lifecycle through its no-op update+refresh.
p.Steps = steps[:4]
snap := p.Run(t, nil)
// Change the config and run an update. We expect everything to require replacement.
providerInputs[resource.PropertyKey("foo")] = resource.NewStringProperty("baz")
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
provURN := p.NewProviderURN("pkgA", "provA", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Look for replace steps on the provider and the resource.
createdProvider, createdResource := false, false
deletedProvider, deletedResource := false, false
for _, entry := range j.Entries {
if entry.Kind != JournalEntrySuccess {
continue
}
switch urn := entry.Step.URN(); urn {
case provURN:
if entry.Step.Op() == deploy.OpDeleteReplaced {
assert.False(t, createdProvider)
assert.False(t, createdResource)
assert.True(t, deletedResource)
deletedProvider = true
} else if entry.Step.Op() == deploy.OpCreateReplacement {
assert.True(t, deletedProvider)
assert.True(t, deletedResource)
assert.False(t, createdResource)
createdProvider = true
}
case resURN:
if entry.Step.Op() == deploy.OpDeleteReplaced {
assert.False(t, deletedProvider)
assert.False(t, deletedResource)
deletedResource = true
} else if entry.Step.Op() == deploy.OpCreateReplacement {
assert.True(t, deletedProvider)
assert.True(t, deletedResource)
assert.True(t, createdProvider)
createdResource = true
}
default:
t.Fatalf("unexpected resource %v", urn)
}
}
assert.True(t, deletedProvider)
assert.True(t, deletedResource)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement first-class providers. (#1695) ### First-Class Providers These changes implement support for first-class providers. First-class providers are provider plugins that are exposed as resources via the Pulumi programming model so that they may be explicitly and multiply instantiated. Each instance of a provider resource may be configured differently, and configuration parameters may be source from the outputs of other resources. ### Provider Plugin Changes In order to accommodate the need to verify and diff provider configuration and configure providers without complete configuration information, these changes adjust the high-level provider plugin interface. Two new methods for validating a provider's configuration and diffing changes to the same have been added (`CheckConfig` and `DiffConfig`, respectively), and the type of the configuration bag accepted by `Configure` has been changed to a `PropertyMap`. These changes have not yet been reflected in the provider plugin gRPC interface. We will do this in a set of follow-up changes. Until then, these methods are implemented by adapters: - `CheckConfig` validates that all configuration parameters are string or unknown properties. This is necessary because existing plugins only accept string-typed configuration values. - `DiffConfig` either returns "never replace" if all configuration values are known or "must replace" if any configuration value is unknown. The justification for this behavior is given [here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106) - `Configure` converts the config bag to a legacy config map and configures the provider plugin if all config values are known. If any config value is unknown, the underlying plugin is not configured and the provider may only perform `Check`, `Read`, and `Invoke`, all of which return empty results. We justify this behavior becuase it is only possible during a preview and provides the best experience we can manage with the existing gRPC interface. ### Resource Model Changes Providers are now exposed as resources that participate in a stack's dependency graph. Like other resources, they are explicitly created, may have multiple instances, and may have dependencies on other resources. Providers are referred to using provider references, which are a combination of the provider's URN and its ID. This design addresses the need during a preview to refer to providers that have not yet been physically created and therefore have no ID. All custom resources that are not themselves providers must specify a single provider via a provider reference. The named provider will be used to manage that resource's CRUD operations. If a resource's provider reference changes, the resource must be replaced. Though its URN is not present in the resource's dependency list, the provider should be treated as a dependency of the resource when topologically sorting the dependency graph. Finally, `Invoke` operations must now specify a provider to use for the invocation via a provider reference. ### Engine Changes First-class providers support requires a few changes to the engine: - The engine must have some way to map from provider references to provider plugins. It must be possible to add providers from a stack's checkpoint to this map and to register new/updated providers during the execution of a plan in response to CRUD operations on provider resources. - In order to support updating existing stacks using existing Pulumi programs that may not explicitly instantiate providers, the engine must be able to manage the "default" providers for each package referenced by a checkpoint or Pulumi program. The configuration for a "default" provider is taken from the stack's configuration data. The former need is addressed by adding a provider registry type that is responsible for managing all of the plugins required by a plan. In addition to loading plugins froma checkpoint and providing the ability to map from a provider reference to a provider plugin, this type serves as the provider plugin for providers themselves (i.e. it is the "provider provider"). The latter need is solved via two relatively self-contained changes to plan setup and the eval source. During plan setup, the old checkpoint is scanned for custom resources that do not have a provider reference in order to compute the set of packages that require a default provider. Once this set has been computed, the required default provider definitions are conjured and prepended to the checkpoint's resource list. Each resource that requires a default provider is then updated to refer to the default provider for its package. While an eval source is running, each custom resource registration, resource read, and invoke that does not name a provider is trapped before being returned by the source iterator. If no default provider for the appropriate package has been registered, the eval source synthesizes an appropriate registration, waits for it to complete, and records the registered provider's reference. This reference is injected into the original request, which is then processed as usual. If a default provider was already registered, the recorded reference is used and no new registration occurs. ### SDK Changes These changes only expose first-class providers from the Node.JS SDK. - A new abstract class, `ProviderResource`, can be subclassed and used to instantiate first-class providers. - A new field in `ResourceOptions`, `provider`, can be used to supply a particular provider instance to manage a `CustomResource`'s CRUD operations. - A new type, `InvokeOptions`, can be used to specify options that control the behavior of a call to `pulumi.runtime.invoke`. This type includes a `provider` field that is analogous to `ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
},
}}
snap = p.Run(t, snap)
// Resume the lifecycle with another no-op update.
p.Steps = steps[2:]
p.Run(t, snap)
}
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
Handle unconfigured plugins in Diff. (#2238) After #2088, we began calling `Diff` on providers that are not configured due to unknown configuration values. This hit an assertion intended to detect exactly this scenario, which was previously unexpected. These changes adjust `Diff` to indicate that a Diff is unavailable and return an error message that describes why. The step generator then interprets the diff as indicating a normal update and issues the error message to the diagnostic stream. Fixes #2223.
2018-11-22 01:53:29 +01:00
func TestSingleResourceDiffUnavailable(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
DiffF: func(urn resource.URN, id resource.ID,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
Handle unconfigured plugins in Diff. (#2238) After #2088, we began calling `Diff` on providers that are not configured due to unknown configuration values. This hit an assertion intended to detect exactly this scenario, which was previously unexpected. These changes adjust `Diff` to indicate that a Diff is unavailable and return an error message that describes why. The step generator then interprets the diff as indicating a normal update and issues the error message to the diagnostic stream. Fixes #2223.
2018-11-22 01:53:29 +01:00
return plugin.DiffResult{}, plugin.DiffUnavailable("diff unavailable")
},
}, nil
}),
}
inputs := resource.PropertyMap{}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Inputs: inputs,
})
Handle unconfigured plugins in Diff. (#2238) After #2088, we began calling `Diff` on providers that are not configured due to unknown configuration values. This hit an assertion intended to detect exactly this scenario, which was previously unexpected. These changes adjust `Diff` to indicate that a Diff is unavailable and return an error message that describes why. The step generator then interprets the diff as indicating a normal update and issues the error message to the diagnostic stream. Fixes #2223.
2018-11-22 01:53:29 +01:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
}
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Run the initial update.
project := p.GetProject()
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
snap, res := TestOp(Update).Run(project, p.GetTarget(nil), p.Options, false, p.BackendClient, nil)
assert.Nil(t, res)
Handle unconfigured plugins in Diff. (#2238) After #2088, we began calling `Diff` on providers that are not configured due to unknown configuration values. This hit an assertion intended to detect exactly this scenario, which was previously unexpected. These changes adjust `Diff` to indicate that a Diff is unavailable and return an error message that describes why. The step generator then interprets the diff as indicating a normal update and issues the error message to the diagnostic stream. Fixes #2223.
2018-11-22 01:53:29 +01:00
// Now change the inputs to our resource and run a preview.
inputs = resource.PropertyMap{"foo": resource.NewStringProperty("bar")}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, true, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, _ *Journal,
events []Event, res result.Result) result.Result {
Handle unconfigured plugins in Diff. (#2238) After #2088, we began calling `Diff` on providers that are not configured due to unknown configuration values. This hit an assertion intended to detect exactly this scenario, which was previously unexpected. These changes adjust `Diff` to indicate that a Diff is unavailable and return an error message that describes why. The step generator then interprets the diff as indicating a normal update and issues the error message to the diagnostic stream. Fixes #2223.
2018-11-22 01:53:29 +01:00
found := false
for _, e := range events {
if e.Type == DiagEvent {
p := e.Payload.(DiagEventPayload)
if p.URN == resURN && p.Severity == diag.Warning && p.Message == "diff unavailable" {
found = true
break
}
}
}
assert.True(t, found)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Handle unconfigured plugins in Diff. (#2238) After #2088, we began calling `Diff` on providers that are not configured due to unknown configuration values. This hit an assertion intended to detect exactly this scenario, which was previously unexpected. These changes adjust `Diff` to indicate that a Diff is unavailable and return an error message that describes why. The step generator then interprets the diff as indicating a normal update and issues the error message to the diagnostic stream. Fixes #2223.
2018-11-22 01:53:29 +01:00
})
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
assert.Nil(t, res)
Handle unconfigured plugins in Diff. (#2238) After #2088, we began calling `Diff` on providers that are not configured due to unknown configuration values. This hit an assertion intended to detect exactly this scenario, which was previously unexpected. These changes adjust `Diff` to indicate that a Diff is unavailable and return an error message that describes why. The step generator then interprets the diff as indicating a normal update and issues the error message to the diagnostic stream. Fixes #2223.
2018-11-22 01:53:29 +01:00
}
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
func TestDestroyWithPendingDelete(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, _ *deploytest.ResourceMonitor) error {
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
}
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Create an old snapshot with two copies of a resource that share a URN: one that is pending deletion and one
// that is not.
old := &deploy.Snapshot{
Resources: []*resource.State{
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "1",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
},
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "0",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
Delete: true,
},
},
}
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(_ workspace.Project, _ deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Adopt golangci-lint and address issues We run the same suite of changes that we did on gometalinter. This ended up catching a few new issues, some of which were addressed and some of which were baselined.
2018-11-05 22:36:35 +01:00
// Verify that we see a DeleteReplacement for the resource with ID 0 and a Delete for the resource with
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
// ID 1.
deletedID0, deletedID1 := false, false
for _, entry := range j.Entries {
// Ignore non-terminal steps and steps that affect the injected default provider.
if entry.Kind != JournalEntrySuccess || entry.Step.URN() != resURN ||
(entry.Step.Op() != deploy.OpDelete && entry.Step.Op() != deploy.OpDeleteReplaced) {
continue
}
switch id := entry.Step.Old().ID; id {
case "0":
assert.False(t, deletedID0)
deletedID0 = true
case "1":
assert.False(t, deletedID1)
deletedID1 = true
default:
assert.Fail(t, "unexpected resource ID %v", string(id))
}
}
assert.True(t, deletedID0)
assert.True(t, deletedID1)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
},
}}
p.Run(t, old)
}
func TestUpdateWithPendingDelete(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, nil, loaders...)
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
}
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Create an old snapshot with two copies of a resource that share a URN: one that is pending deletion and one
// that is not.
old := &deploy.Snapshot{
Resources: []*resource.State{
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "1",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
},
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "0",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
Delete: true,
},
},
}
p.Steps = []TestStep{{
Op: Destroy,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(_ workspace.Project, _ deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Adopt golangci-lint and address issues We run the same suite of changes that we did on gometalinter. This ended up catching a few new issues, some of which were addressed and some of which were baselined.
2018-11-05 22:36:35 +01:00
// Verify that we see a DeleteReplacement for the resource with ID 0 and a Delete for the resource with
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
// ID 1.
deletedID0, deletedID1 := false, false
for _, entry := range j.Entries {
// Ignore non-terminal steps and steps that affect the injected default provider.
if entry.Kind != JournalEntrySuccess || entry.Step.URN() != resURN ||
(entry.Step.Op() != deploy.OpDelete && entry.Step.Op() != deploy.OpDeleteReplaced) {
continue
}
switch id := entry.Step.Old().ID; id {
case "0":
assert.False(t, deletedID0)
deletedID0 = true
case "1":
assert.False(t, deletedID1)
deletedID1 = true
default:
assert.Fail(t, "unexpected resource ID %v", string(id))
}
}
assert.True(t, deletedID0)
assert.True(t, deletedID1)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Fix deletes with duplicate URNs. (#1716) When calculating deletes, we will only issue a single delete step for a particular URN. This is incorrect in the presence of pending deletes that share URNs with a live resource if the pending deletes follow the live resource in the checkpoint: instead of issuing a delete for every resource with a particular URN, we will only issue deletes for the pending deletes. Before first-class providers, this was mostly benigin: any remaining resources could be deleted by re-running the destroy. With the first-class provider changes, however, the provider for the undeleted resources will be deleted, leaving the checkpoint in an invalid state. These changes fix this issue by allowing the step generator to issue multiple deletes for a single URN and add a test for this scenario.
2018-08-07 20:01:08 +02:00
},
}}
p.Run(t, old)
}
Serialize SourceEvents coming from the refresh source (#1725) * Serialize SourceEvents coming from the refresh source The engine requires that a source event coming from a source be "ready to execute" at the moment that it is sent to the engine. Since the refresh source sent all goal states eagerly through its source iterator, the engine assumed that it was legal to execute them all in parallel and did so. This is a problem for the snapshot, since the snapshot expects to be in an order that is a legal topological ordering of the dependency DAG. This PR fixes the issue by sending refresh source events one-at-a-time through the refresh source iterator, only unblocking to send the next step as soon as the previous step completes. * Fix deadlock in refresh test * Fix an issue where the engine "completed" steps too early By signalling that a step is done before committing the step's results to the snapshot, the engine was left with a race where dependent resources could find themselves completely executed and committed before a resource that they depend on has been committed. Fixes pulumi/pulumi#1726 * Fix an issue with Replace steps at the end of a plan If the last step that was executed successfully was a Replace, we could end up in a situation where we unintentionally left the snapshot invalid. * Add a test * CR: pass context.Context as first parameter to Iterate * CR: null->nil
2018-08-08 22:45:48 +02:00
func TestParallelRefresh(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
// Create a program that registers four resources, each of which depends on the resource that immediately precedes
// it.
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
resA, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Serialize SourceEvents coming from the refresh source (#1725) * Serialize SourceEvents coming from the refresh source The engine requires that a source event coming from a source be "ready to execute" at the moment that it is sent to the engine. Since the refresh source sent all goal states eagerly through its source iterator, the engine assumed that it was legal to execute them all in parallel and did so. This is a problem for the snapshot, since the snapshot expects to be in an order that is a legal topological ordering of the dependency DAG. This PR fixes the issue by sending refresh source events one-at-a-time through the refresh source iterator, only unblocking to send the next step as soon as the previous step completes. * Fix deadlock in refresh test * Fix an issue where the engine "completed" steps too early By signalling that a step is done before committing the step's results to the snapshot, the engine was left with a race where dependent resources could find themselves completely executed and committed before a resource that they depend on has been committed. Fixes pulumi/pulumi#1726 * Fix an issue with Replace steps at the end of a plan If the last step that was executed successfully was a Replace, we could end up in a situation where we unintentionally left the snapshot invalid. * Add a test * CR: pass context.Context as first parameter to Iterate * CR: null->nil
2018-08-08 22:45:48 +02:00
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
resB, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resB", true, deploytest.ResourceOptions{
Dependencies: []resource.URN{resA},
})
Serialize SourceEvents coming from the refresh source (#1725) * Serialize SourceEvents coming from the refresh source The engine requires that a source event coming from a source be "ready to execute" at the moment that it is sent to the engine. Since the refresh source sent all goal states eagerly through its source iterator, the engine assumed that it was legal to execute them all in parallel and did so. This is a problem for the snapshot, since the snapshot expects to be in an order that is a legal topological ordering of the dependency DAG. This PR fixes the issue by sending refresh source events one-at-a-time through the refresh source iterator, only unblocking to send the next step as soon as the previous step completes. * Fix deadlock in refresh test * Fix an issue where the engine "completed" steps too early By signalling that a step is done before committing the step's results to the snapshot, the engine was left with a race where dependent resources could find themselves completely executed and committed before a resource that they depend on has been committed. Fixes pulumi/pulumi#1726 * Fix an issue with Replace steps at the end of a plan If the last step that was executed successfully was a Replace, we could end up in a situation where we unintentionally left the snapshot invalid. * Add a test * CR: pass context.Context as first parameter to Iterate * CR: null->nil
2018-08-08 22:45:48 +02:00
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
resC, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resC", true, deploytest.ResourceOptions{
Dependencies: []resource.URN{resB},
})
Serialize SourceEvents coming from the refresh source (#1725) * Serialize SourceEvents coming from the refresh source The engine requires that a source event coming from a source be "ready to execute" at the moment that it is sent to the engine. Since the refresh source sent all goal states eagerly through its source iterator, the engine assumed that it was legal to execute them all in parallel and did so. This is a problem for the snapshot, since the snapshot expects to be in an order that is a legal topological ordering of the dependency DAG. This PR fixes the issue by sending refresh source events one-at-a-time through the refresh source iterator, only unblocking to send the next step as soon as the previous step completes. * Fix deadlock in refresh test * Fix an issue where the engine "completed" steps too early By signalling that a step is done before committing the step's results to the snapshot, the engine was left with a race where dependent resources could find themselves completely executed and committed before a resource that they depend on has been committed. Fixes pulumi/pulumi#1726 * Fix an issue with Replace steps at the end of a plan If the last step that was executed successfully was a Replace, we could end up in a situation where we unintentionally left the snapshot invalid. * Add a test * CR: pass context.Context as first parameter to Iterate * CR: null->nil
2018-08-08 22:45:48 +02:00
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err = monitor.RegisterResource("pkgA:m:typA", "resD", true, deploytest.ResourceOptions{
Dependencies: []resource.URN{resC},
})
Serialize SourceEvents coming from the refresh source (#1725) * Serialize SourceEvents coming from the refresh source The engine requires that a source event coming from a source be "ready to execute" at the moment that it is sent to the engine. Since the refresh source sent all goal states eagerly through its source iterator, the engine assumed that it was legal to execute them all in parallel and did so. This is a problem for the snapshot, since the snapshot expects to be in an order that is a legal topological ordering of the dependency DAG. This PR fixes the issue by sending refresh source events one-at-a-time through the refresh source iterator, only unblocking to send the next step as soon as the previous step completes. * Fix deadlock in refresh test * Fix an issue where the engine "completed" steps too early By signalling that a step is done before committing the step's results to the snapshot, the engine was left with a race where dependent resources could find themselves completely executed and committed before a resource that they depend on has been committed. Fixes pulumi/pulumi#1726 * Fix an issue with Replace steps at the end of a plan If the last step that was executed successfully was a Replace, we could end up in a situation where we unintentionally left the snapshot invalid. * Add a test * CR: pass context.Context as first parameter to Iterate * CR: null->nil
2018-08-08 22:45:48 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Serialize SourceEvents coming from the refresh source (#1725) * Serialize SourceEvents coming from the refresh source The engine requires that a source event coming from a source be "ready to execute" at the moment that it is sent to the engine. Since the refresh source sent all goal states eagerly through its source iterator, the engine assumed that it was legal to execute them all in parallel and did so. This is a problem for the snapshot, since the snapshot expects to be in an order that is a legal topological ordering of the dependency DAG. This PR fixes the issue by sending refresh source events one-at-a-time through the refresh source iterator, only unblocking to send the next step as soon as the previous step completes. * Fix deadlock in refresh test * Fix an issue where the engine "completed" steps too early By signalling that a step is done before committing the step's results to the snapshot, the engine was left with a race where dependent resources could find themselves completely executed and committed before a resource that they depend on has been committed. Fixes pulumi/pulumi#1726 * Fix an issue with Replace steps at the end of a plan If the last step that was executed successfully was a Replace, we could end up in a situation where we unintentionally left the snapshot invalid. * Add a test * CR: pass context.Context as first parameter to Iterate * CR: null->nil
2018-08-08 22:45:48 +02:00
p := &TestPlan{
Options: UpdateOptions{Parallel: 4, host: host},
}
p.Steps = []TestStep{{Op: Update}}
snap := p.Run(t, nil)
assert.Len(t, snap.Resources, 5)
assert.Equal(t, string(snap.Resources[0].URN.Name()), "default") // provider
assert.Equal(t, string(snap.Resources[1].URN.Name()), "resA")
assert.Equal(t, string(snap.Resources[2].URN.Name()), "resB")
assert.Equal(t, string(snap.Resources[3].URN.Name()), "resC")
assert.Equal(t, string(snap.Resources[4].URN.Name()), "resD")
p.Steps = []TestStep{{Op: Refresh}}
snap = p.Run(t, snap)
assert.Len(t, snap.Resources, 5)
assert.Equal(t, string(snap.Resources[0].URN.Name()), "default") // provider
assert.Equal(t, string(snap.Resources[1].URN.Name()), "resA")
assert.Equal(t, string(snap.Resources[2].URN.Name()), "resB")
assert.Equal(t, string(snap.Resources[3].URN.Name()), "resC")
assert.Equal(t, string(snap.Resources[4].URN.Name()), "resD")
}
Emit reads for external resources when refreshing (#1749) * Emit reads for external resources when refreshing Fixes pulumi/pulumi#1744. This commit educates the refresh source about external resources. If a refresh source encounters a resource with the External bit set, it'll send a Read event to the engine and the engine will process it accordingly. * CR: save last event channel instead of last event, style fixes
2018-08-09 21:45:39 +02:00
func TestExternalRefresh(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
// Our program reads a resource and exits.
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Load specific provider versions if requested (#2648) * Load specific provider versions if requested As part of pulumi/pulumi#2389, we need the ability for language hosts to tell the engine that a particular resource registration, read, or invoke needs to use a particular version of a resource provider. This was not previously possible before; the engine prior to this commit loaded plugins from a default provider map, which was inferred for every resource provider based on the contents of a user's package.json, and was itself prone to bugs. This PR adds the engine support needed for language hosts to request a particular version of a provider. If this occurs, the source evaluator specifically records the intent to load a provider with a given version and produces a "default" provider registration that requests exactly that version. This allows the source evaluator to produce multiple default providers for a signle package, which was previously not possible. This is accomplished by having the source evaluator deal in the "ProviderRequest" type, which is a tuple of version and package. A request to load a provider whose version matches the package of a previously loaded provider will re-use the existing default provider. If the version was not previously loaded, a new default provider is injected. * CR Feedback: raise error if semver is invalid * CR: call String() if you want a hash key * Update pkg/resource/deploy/providers/provider.go Co-Authored-By: swgillespie <sean@pulumi.com>
2019-04-17 20:25:02 +02:00
_, _, err := monitor.ReadResource("pkgA:m:typA", "resA", "resA-some-id", "", resource.PropertyMap{}, "", "")
Emit reads for external resources when refreshing (#1749) * Emit reads for external resources when refreshing Fixes pulumi/pulumi#1744. This commit educates the refresh source about external resources. If a refresh source encounters a resource with the External bit set, it'll send a Read event to the engine and the engine will process it accordingly. * CR: save last event channel instead of last event, style fixes
2018-08-09 21:45:39 +02:00
if !assert.NoError(t, err) {
t.FailNow()
}
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Emit reads for external resources when refreshing (#1749) * Emit reads for external resources when refreshing Fixes pulumi/pulumi#1744. This commit educates the refresh source about external resources. If a refresh source encounters a resource with the External bit set, it'll send a Read event to the engine and the engine will process it accordingly. * CR: save last event channel instead of last event, style fixes
2018-08-09 21:45:39 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{Op: Update}},
}
// The read should place "resA" in the snapshot with the "External" bit set.
snap := p.Run(t, nil)
assert.Len(t, snap.Resources, 2)
assert.Equal(t, string(snap.Resources[0].URN.Name()), "default") // provider
assert.Equal(t, string(snap.Resources[1].URN.Name()), "resA")
assert.True(t, snap.Resources[1].External)
p = &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{Op: Refresh}},
}
snap = p.Run(t, snap)
// A refresh should leave "resA" as it is in the snapshot. The External bit should still be set.
assert.Len(t, snap.Resources, 2)
assert.Equal(t, string(snap.Resources[0].URN.Name()), "default") // provider
assert.Equal(t, string(snap.Resources[1].URN.Name()), "resA")
assert.True(t, snap.Resources[1].External)
}
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
func TestRefreshInitFailure(t *testing.T) {
Fail refreshes with init errors. (#1882) And ensure that refreshes continue on errors. Fixes #1881.
2018-09-05 23:00:28 +02:00
p := &TestPlan{}
provURN := p.NewProviderURN("pkgA", "default", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
res2URN := p.NewURN("pkgA:m:typA", "resB", "")
res2Outputs := resource.PropertyMap{"foo": resource.NewStringProperty("bar")}
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
//
// Refresh will persist any initialization errors that are returned by `Read`. This provider
// will error out or not based on the value of `refreshShouldFail`.
//
refreshShouldFail := false
//
// Set up test environment to use `readFailProvider` as the underlying resource provider.
//
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
urn resource.URN, id resource.ID, inputs, state resource.PropertyMap,
) (plugin.ReadResult, resource.Status, error) {
Fail refreshes with init errors. (#1882) And ensure that refreshes continue on errors. Fixes #1881.
2018-09-05 23:00:28 +02:00
if refreshShouldFail && urn == resURN {
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
err := &plugin.InitError{
Reasons: []string{"Refresh reports continued to fail to initialize"},
}
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
return plugin.ReadResult{Outputs: resource.PropertyMap{}}, resource.StatusPartialFailure, err
Fail refreshes with init errors. (#1882) And ensure that refreshes continue on errors. Fixes #1881.
2018-09-05 23:00:28 +02:00
} else if urn == res2URN {
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
return plugin.ReadResult{Outputs: res2Outputs}, resource.StatusOK, nil
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
}
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
return plugin.ReadResult{Outputs: resource.PropertyMap{}}, resource.StatusOK, nil
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
assert.NoError(t, err)
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
Fail refreshes with init errors. (#1882) And ensure that refreshes continue on errors. Fixes #1881.
2018-09-05 23:00:28 +02:00
p.Options.host = host
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
//
// Create an old snapshot with a single initialization failure.
//
old := &deploy.Snapshot{
Fail refreshes with init errors. (#1882) And ensure that refreshes continue on errors. Fixes #1881.
2018-09-05 23:00:28 +02:00
Resources: []*resource.State{
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "0",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
InitErrors: []string{"Resource failed to initialize"},
},
{
Type: res2URN.Type(),
URN: res2URN,
Custom: true,
ID: "1",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
},
},
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
}
//
// Refresh DOES NOT fail, causing the initialization error to disappear.
//
p.Steps = []TestStep{{Op: Refresh}}
snap := p.Run(t, old)
for _, resource := range snap.Resources {
switch urn := resource.URN; urn {
case provURN:
// break
case resURN:
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
assert.Empty(t, resource.InitErrors)
Fail refreshes with init errors. (#1882) And ensure that refreshes continue on errors. Fixes #1881.
2018-09-05 23:00:28 +02:00
case res2URN:
assert.Equal(t, res2Outputs, resource.Outputs)
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
default:
t.Fatalf("unexpected resource %v", urn)
}
}
//
Warn instead of error when `refresh`'d resource is unhealthy Fixes #2633. Currently when a user runs `refresh` and a resource is in a state of error, the `refresh` will fail and the resource state will not be persisted. This can make it vastly harder to incrementally fix infrastructure. The issue mentioned above explains more of the historical context, as well as some specific failure modes. This commit resolves this issue by causing refresh to *not* report an error in this case, and instead to simply log a warning that the `refresh` has recognized that the resource is in an unhealthy state during state sync.
2019-04-10 22:47:54 +02:00
// Refresh again, see the resource is in a partial state of failure, but the refresh operation
// DOES NOT fail. The initialization error is still persisted.
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
//
refreshShouldFail = true
Warn instead of error when `refresh`'d resource is unhealthy Fixes #2633. Currently when a user runs `refresh` and a resource is in a state of error, the `refresh` will fail and the resource state will not be persisted. This can make it vastly harder to incrementally fix infrastructure. The issue mentioned above explains more of the historical context, as well as some specific failure modes. This commit resolves this issue by causing refresh to *not* report an error in this case, and instead to simply log a warning that the `refresh` has recognized that the resource is in an unhealthy state during state sync.
2019-04-10 22:47:54 +02:00
p.Steps = []TestStep{{Op: Refresh, SkipPreview: true}}
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
snap = p.Run(t, old)
for _, resource := range snap.Resources {
switch urn := resource.URN; urn {
case provURN:
// break
case resURN:
assert.Equal(t, []string{"Refresh reports continued to fail to initialize"}, resource.InitErrors)
Fail refreshes with init errors. (#1882) And ensure that refreshes continue on errors. Fixes #1881.
2018-09-05 23:00:28 +02:00
case res2URN:
assert.Equal(t, res2Outputs, resource.Outputs)
Implement partial `Read` Some time ago, we introduced the concept of the initialization error to Pulumi (i.e., an error where the resource was successfully created but failed to fully initialize). This was originally implemented in `Create` and `Update` methods of the resource provider interface; when we detected an initialization failure, we'd pack the live version of the object into the error, and return that to the engine. Omitted from this initial implementation was a similar semantics for `Read`. There are many implications of this, but one of them is that a `pulumi refresh` will erase any initialization errors that had previously been observed, even if the initialization errors still exist in the resource. This commit will introduce the initialization error semantics to `Read`, fixing this issue.
2018-08-07 09:40:43 +02:00
default:
t.Fatalf("unexpected resource %v", urn)
}
}
}
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
// Test that ensures that we log diagnostics for resources that receive an error from Check. (Note that this
// is distinct from receiving non-error failures from Check.)
func TestCheckFailureRecord(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
CheckF: func(urn resource.URN,
olds, news resource.PropertyMap) (resource.PropertyMap, []plugin.CheckFailure, error) {
return nil, nil, errors.New("oh no, check had an error")
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
assert.Error(t, err)
return err
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{
Op: Update,
ExpectFailure: true,
SkipPreview: true,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
sawFailure := false
for _, evt := range evts {
if evt.Type == DiagEvent {
e := evt.Payload.(DiagEventPayload)
msg := colors.Never.Colorize(e.Message)
sawFailure = msg == "oh no, check had an error\n" && e.Severity == diag.Error
}
}
assert.True(t, sawFailure)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
},
}},
}
p.Run(t, nil)
}
// Test that checks that we emit diagnostics for properties that check says are invalid.
func TestCheckFailureInvalidPropertyRecord(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
CheckF: func(urn resource.URN,
olds, news resource.PropertyMap) (resource.PropertyMap, []plugin.CheckFailure, error) {
return nil, []plugin.CheckFailure{{
Property: "someprop",
Reason: "field is not valid",
}}, nil
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
assert.Error(t, err)
return err
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{
Op: Update,
ExpectFailure: true,
SkipPreview: true,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
sawFailure := false
for _, evt := range evts {
if evt.Type == DiagEvent {
e := evt.Payload.(DiagEventPayload)
msg := colors.Never.Colorize(e.Message)
sawFailure = strings.Contains(msg, "field is not valid") && e.Severity == diag.Error
if sawFailure {
break
}
}
}
assert.True(t, sawFailure)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Emit diagnostics for errors coming from step generation (#1766) The plan executor assumed that the step generator was responsible for logging its own diagnostics, which it sort-of is but also doesn't log a majority of the diagnositcs that come out of it. This commit logs all errors coming out of step generation so that we don't unintentionally drop errors.
2018-08-13 22:41:01 +02:00
},
}},
}
p.Run(t, nil)
}
Add test for pulumi/pulumi#1770 (#1773) * Add test for pulumi/pulumi#1770 * CR: Add parallel test, do an update before refresh
2018-08-15 00:32:23 +02:00
// Test that tests that Refresh can detect that resources have been deleted and removes them
// from the snapshot.
func TestRefreshWithDelete(t *testing.T) {
for _, parallelFactor := range []int{1, 4} {
t.Run(fmt.Sprintf("parallel-%d", parallelFactor), func(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
urn resource.URN, id resource.ID, inputs, state resource.PropertyMap,
) (plugin.ReadResult, resource.Status, error) {
Add test for pulumi/pulumi#1770 (#1773) * Add test for pulumi/pulumi#1770 * CR: Add parallel test, do an update before refresh
2018-08-15 00:32:23 +02:00
// This thing doesn't exist. Returning nil from Read should trigger
// the engine to delete it from the snapshot.
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
return plugin.ReadResult{}, resource.StatusOK, nil
Add test for pulumi/pulumi#1770 (#1773) * Add test for pulumi/pulumi#1770 * CR: Add parallel test, do an update before refresh
2018-08-15 00:32:23 +02:00
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Add test for pulumi/pulumi#1770 (#1773) * Add test for pulumi/pulumi#1770 * CR: Add parallel test, do an update before refresh
2018-08-15 00:32:23 +02:00
assert.NoError(t, err)
return err
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Add test for pulumi/pulumi#1770 (#1773) * Add test for pulumi/pulumi#1770 * CR: Add parallel test, do an update before refresh
2018-08-15 00:32:23 +02:00
p := &TestPlan{Options: UpdateOptions{host: host, Parallel: parallelFactor}}
p.Steps = []TestStep{{Op: Update}}
snap := p.Run(t, nil)
p.Steps = []TestStep{{Op: Refresh}}
snap = p.Run(t, snap)
// Refresh succeeds and records that the resource in the snapshot doesn't exist anymore
provURN := p.NewProviderURN("pkgA", "default", "")
assert.Len(t, snap.Resources, 1)
assert.Equal(t, provURN, snap.Resources[0].URN)
})
}
}
Log errors coming from the language host (#1780) * Log errors coming from the language host Similar to pulumi/pulumi#1762, fixes pulumi/pulumi#1775. The language host can fail without issuing any diagnostics and it is very unclear what happens if the engine does not log the error. * CR feedback
2018-08-21 01:41:07 +02:00
Delete specific target (#3244)
2019-09-20 04:28:14 +02:00
func pickURN(t *testing.T, urns []resource.URN, names []string, target string) resource.URN {
assert.Equal(t, len(urns), len(names))
assert.Contains(t, names, target)
for i, name := range names {
if name == target {
return urns[i]
}
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
}
Delete specific target (#3244)
2019-09-20 04:28:14 +02:00
t.Fatalf("Could not find target: %v in %v", target, names)
return ""
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
}
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
// Tests that dependencies are correctly rewritten when refresh removes deleted resources.
func TestRefreshDeleteDependencies(t *testing.T) {
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
names := []string{"resA", "resB", "resC"}
// Try refreshing a stack with every combination of the three above resources as a target to
// refresh.
subsets := combinations.All(names)
// combinations.All doesn't return the empty set. So explicitly test that case (i.e. test no
// targets specified)
validateRefreshDeleteCombination(t, names, []string{})
for _, subset := range subsets {
validateRefreshDeleteCombination(t, names, subset)
}
}
func validateRefreshDeleteCombination(t *testing.T, names []string, targets []string) {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
p := &TestPlan{}
const resType = "pkgA:m:typA"
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
urnA := p.NewURN(resType, names[0], "")
urnB := p.NewURN(resType, names[1], "")
urnC := p.NewURN(resType, names[2], "")
urns := []resource.URN{urnA, urnB, urnC}
refreshTargets := []resource.URN{}
t.Logf("Refreshing targets: %v", targets)
for _, target := range targets {
Delete specific target (#3244)
2019-09-20 04:28:14 +02:00
refreshTargets = append(refreshTargets, pickURN(t, urns, names, target))
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
}
p.Options.RefreshTargets = refreshTargets
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
newResource := func(urn resource.URN, id resource.ID, delete bool, dependencies ...resource.URN) *resource.State {
return &resource.State{
Type: urn.Type(),
URN: urn,
Custom: true,
Delete: delete,
ID: id,
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
Dependencies: dependencies,
}
}
oldResources := []*resource.State{
newResource(urnA, "0", false),
newResource(urnB, "1", false, urnA),
newResource(urnC, "2", false, urnA, urnB),
newResource(urnA, "3", true),
newResource(urnA, "4", true),
newResource(urnC, "5", true, urnA, urnB),
}
old := &deploy.Snapshot{
Resources: oldResources,
}
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(urn resource.URN, id resource.ID,
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
inputs, state resource.PropertyMap) (plugin.ReadResult, resource.Status, error) {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
switch id {
case "0", "4":
// We want to delete resources A::0 and A::4.
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
return plugin.ReadResult{}, resource.StatusOK, nil
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
default:
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
return plugin.ReadResult{Inputs: inputs, Outputs: state}, resource.StatusOK, nil
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
}
},
}, nil
}),
}
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
p.Options.host = deploytest.NewPluginHost(nil, nil, nil, loaders...)
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
p.Steps = []TestStep{
{
Op: Refresh,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
// Should see only refreshes.
for _, entry := range j.Entries {
if len(refreshTargets) > 0 {
// should only see changes to urns we explicitly asked to change
assert.Containsf(t, refreshTargets, entry.Step.URN(),
"Refreshed a resource that wasn't a target: %v", entry.Step.URN())
}
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
}
return res
},
},
}
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
snap := p.Run(t, old)
provURN := p.NewProviderURN("pkgA", "default", "")
for _, r := range snap.Resources {
switch urn := r.URN; urn {
case provURN:
continue
case urnA, urnB, urnC:
// break
default:
t.Fatalf("unexpected resource %v", urn)
}
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
if len(refreshTargets) == 0 || containsURN(refreshTargets, urnA) {
// 'A' was deleted, so we should see the impact downstream.
switch r.ID {
case "1":
// A::0 was deleted, so B's dependency list should be empty.
assert.Equal(t, urnB, r.URN)
assert.Empty(t, r.Dependencies)
case "2":
// A::0 was deleted, so C's dependency list should only contain B.
assert.Equal(t, urnC, r.URN)
assert.Equal(t, []resource.URN{urnB}, r.Dependencies)
case "3":
// A::3 should not have changed.
assert.Equal(t, oldResources[3], r)
case "5":
// A::4 was deleted but A::3 was still refernceable by C, so C should not have changed.
assert.Equal(t, oldResources[5], r)
default:
t.Fatalf("Unexpected changed resource when refreshing %v: %v::%v", refreshTargets, r.URN, r.ID)
}
} else {
// A was not deleted. So nothing should be impacted.
id, err := strconv.Atoi(r.ID.String())
assert.NoError(t, err)
assert.Equal(t, oldResources[id], r)
}
}
}
func containsURN(urns []resource.URN, urn resource.URN) bool {
for _, val := range urns {
if val == urn {
return true
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
}
}
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
return false
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
}
// Tests basic refresh functionality.
func TestRefreshBasics(t *testing.T) {
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
names := []string{"resA", "resB", "resC"}
// Try refreshing a stack with every combination of the three above resources as a target to
// refresh.
subsets := combinations.All(names)
// combinations.All doesn't return the empty set. So explicitly test that case (i.e. test no
// targets specified)
validateRefreshBasicsCombination(t, names, []string{})
for _, subset := range subsets {
validateRefreshBasicsCombination(t, names, subset)
}
}
func validateRefreshBasicsCombination(t *testing.T, names []string, targets []string) {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
p := &TestPlan{}
const resType = "pkgA:m:typA"
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
urnA := p.NewURN(resType, names[0], "")
urnB := p.NewURN(resType, names[1], "")
urnC := p.NewURN(resType, names[2], "")
urns := []resource.URN{urnA, urnB, urnC}
refreshTargets := []resource.URN{}
for _, target := range targets {
Delete specific target (#3244)
2019-09-20 04:28:14 +02:00
refreshTargets = append(p.Options.RefreshTargets, pickURN(t, urns, names, target))
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
}
p.Options.RefreshTargets = refreshTargets
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
newResource := func(urn resource.URN, id resource.ID, delete bool, dependencies ...resource.URN) *resource.State {
return &resource.State{
Type: urn.Type(),
URN: urn,
Custom: true,
Delete: delete,
ID: id,
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
Dependencies: dependencies,
}
}
oldResources := []*resource.State{
newResource(urnA, "0", false),
newResource(urnB, "1", false, urnA),
newResource(urnC, "2", false, urnA, urnB),
newResource(urnA, "3", true),
newResource(urnA, "4", true),
newResource(urnC, "5", true, urnA, urnB),
}
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
newStates := map[resource.ID]plugin.ReadResult{
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
// A::0 and A::3 will have no changes.
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
"0": {Outputs: resource.PropertyMap{}, Inputs: resource.PropertyMap{}},
"3": {Outputs: resource.PropertyMap{}, Inputs: resource.PropertyMap{}},
// B::1 and A::4 will have changes. The latter will also have input changes.
"1": {Outputs: resource.PropertyMap{"foo": resource.NewStringProperty("bar")}, Inputs: resource.PropertyMap{}},
"4": {
Outputs: resource.PropertyMap{"baz": resource.NewStringProperty("qux")},
Inputs: resource.PropertyMap{"oof": resource.NewStringProperty("zab")},
},
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
// C::2 and C::5 will be deleted.
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
"2": {},
"5": {},
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
}
old := &deploy.Snapshot{
Resources: oldResources,
}
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(urn resource.URN, id resource.ID,
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
inputs, state resource.PropertyMap) (plugin.ReadResult, resource.Status, error) {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
new, hasNewState := newStates[id]
assert.True(t, hasNewState)
return new, resource.StatusOK, nil
},
}, nil
}),
}
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
p.Options.host = deploytest.NewPluginHost(nil, nil, nil, loaders...)
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
p.Steps = []TestStep{{
Op: Refresh,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
// Should see only refreshes.
for _, entry := range j.Entries {
Add support for refreshing specific targets. (#3225)
2019-09-18 03:14:10 +02:00
if len(refreshTargets) > 0 {
// should only see changes to urns we explicitly asked to change
assert.Containsf(t, refreshTargets, entry.Step.URN(),
"Refreshed a resource that wasn't a target: %v", entry.Step.URN())
}
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
resultOp := entry.Step.(*deploy.RefreshStep).ResultOp()
old := entry.Step.Old()
if !old.Custom || providers.IsProviderType(old.Type) {
// Component and provider resources should never change.
assert.Equal(t, deploy.OpSame, resultOp)
continue
}
expected, new := newStates[old.ID], entry.Step.New()
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
if expected.Outputs == nil {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
// If the resource was deleted, we want the result op to be an OpDelete.
assert.Nil(t, new)
assert.Equal(t, deploy.OpDelete, resultOp)
} else {
// If there were changes to the outputs, we want the result op to be an OpUpdate. Otherwise we want
// an OpSame.
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
if reflect.DeepEqual(old.Outputs, expected.Outputs) {
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
assert.Equal(t, deploy.OpSame, resultOp)
} else {
assert.Equal(t, deploy.OpUpdate, resultOp)
}
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
// Only the inputs and outputs should have changed (if anything changed).
old.Inputs = expected.Inputs
old.Outputs = expected.Outputs
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
assert.Equal(t, old, new)
}
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
},
}}
snap := p.Run(t, old)
provURN := p.NewProviderURN("pkgA", "default", "")
for _, r := range snap.Resources {
switch urn := r.URN; urn {
case provURN:
continue
case urnA, urnB, urnC:
// break
default:
t.Fatalf("unexpected resource %v", urn)
}
// The only resources left in the checkpoint should be those that were not deleted by the refresh.
expected := newStates[r.ID]
assert.NotNil(t, expected)
idx, err := strconv.ParseInt(string(r.ID), 0, 0)
assert.NoError(t, err)
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
// The new resources should be equal to the old resources + the new inputs and outputs.
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
old := oldResources[int(idx)]
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
old.Inputs = expected.Inputs
old.Outputs = expected.Outputs
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
assert.Equal(t, old, r)
}
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
}
// Tests that an interrupted refresh leaves behind an expected state.
func TestCanceledRefresh(t *testing.T) {
p := &TestPlan{}
const resType = "pkgA:m:typA"
urnA := p.NewURN(resType, "resA", "")
urnB := p.NewURN(resType, "resB", "")
urnC := p.NewURN(resType, "resC", "")
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
newResource := func(urn resource.URN, id resource.ID, delete bool, dependencies ...resource.URN) *resource.State {
return &resource.State{
Type: urn.Type(),
URN: urn,
Custom: true,
Delete: delete,
ID: id,
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
Dependencies: dependencies,
}
}
oldResources := []*resource.State{
newResource(urnA, "0", false),
newResource(urnB, "1", false),
newResource(urnC, "2", false),
}
newStates := map[resource.ID]resource.PropertyMap{
// A::0 and B::1 will have changes; D::3 will be deleted.
"0": {"foo": resource.NewStringProperty("bar")},
"1": {"baz": resource.NewStringProperty("qux")},
"2": nil,
}
old := &deploy.Snapshot{
Resources: oldResources,
}
// Set up a cancelable context for the refresh operation.
ctx, cancel := context.WithCancel(context.Background())
// Serialize all refreshes s.t. we can cancel after the first is issued.
Fix TestCanceledRefresh (#2331) This test had been intermittently failing due to a race condition. Its implementation of `plugin.Provider.Read` was intended to ensure that the cancellation of a refresh operation occurred. As written, it was only able to ensure that the cancellation was requested. These changes ensure that cancellation has been acknowledged by the engine by implementing providing an implementation for `plugin.Provider.Cancel` that closes a channel on which the implementation of `Read` waits.
2019-01-04 04:40:07 +01:00
refreshes, cancelled := make(chan resource.ID), make(chan bool)
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
go func() {
<-refreshes
cancel()
}()
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(urn resource.URN, id resource.ID,
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
inputs, state resource.PropertyMap) (plugin.ReadResult, resource.Status, error) {
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
Fix TestCanceledRefresh (#2331) This test had been intermittently failing due to a race condition. Its implementation of `plugin.Provider.Read` was intended to ensure that the cancellation of a refresh operation occurred. As written, it was only able to ensure that the cancellation was requested. These changes ensure that cancellation has been acknowledged by the engine by implementing providing an implementation for `plugin.Provider.Cancel` that closes a channel on which the implementation of `Read` waits.
2019-01-04 04:40:07 +01:00
refreshes <- id
<-cancelled
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
new, hasNewState := newStates[id]
assert.True(t, hasNewState)
Refresh inputs (#2531) These changes take advantage of the newly-added support for returning inputs from Read to update a resource's inputs as part of a refresh. As a consequence, the Pulumi engine will now properly detect drift between the actual state of a resource and the desired state described by the program and generate appropriate update or replace steps. As part of these changes, a resource's old inputs are now passed to the provider when performing a refresh. The provider can take advantage of this to maintain the accuracy of any additional data or metadata in the resource's inputs that may need to be updated during the refresh. This is required for the complete implementation of https://github.com/pulumi/pulumi-terraform/pull/349. Without access to the old inputs for a resource, TF-based providers would lose all information about default population during a refresh.
2019-03-11 21:50:00 +01:00
return plugin.ReadResult{Outputs: new}, resource.StatusOK, nil
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
},
Fix TestCanceledRefresh (#2331) This test had been intermittently failing due to a race condition. Its implementation of `plugin.Provider.Read` was intended to ensure that the cancellation of a refresh operation occurred. As written, it was only able to ensure that the cancellation was requested. These changes ensure that cancellation has been acknowledged by the engine by implementing providing an implementation for `plugin.Provider.Cancel` that closes a channel on which the implementation of `Read` waits.
2019-01-04 04:40:07 +01:00
CancelF: func() error {
close(cancelled)
return nil
},
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
}, nil
}),
}
refreshed := make(map[resource.ID]bool)
op := TestOp(Refresh)
options := UpdateOptions{
Parallel: 1,
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host: deploytest.NewPluginHost(nil, nil, nil, loaders...),
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
}
project, target := p.GetProject(), p.GetTarget(old)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
validate := func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
for _, entry := range j.Entries {
assert.Equal(t, deploy.OpRefresh, entry.Step.Op())
resultOp := entry.Step.(*deploy.RefreshStep).ResultOp()
old := entry.Step.Old()
if !old.Custom || providers.IsProviderType(old.Type) {
// Component and provider resources should never change.
assert.Equal(t, deploy.OpSame, resultOp)
continue
}
refreshed[old.ID] = true
expected, new := newStates[old.ID], entry.Step.New()
if expected == nil {
// If the resource was deleted, we want the result op to be an OpDelete.
assert.Nil(t, new)
assert.Equal(t, deploy.OpDelete, resultOp)
} else {
// If there were changes to the outputs, we want the result op to be an OpUpdate. Otherwise we want
// an OpSame.
if reflect.DeepEqual(old.Outputs, expected) {
assert.Equal(t, deploy.OpSame, resultOp)
} else {
assert.Equal(t, deploy.OpUpdate, resultOp)
}
// Only the outputs should have changed (if anything changed).
old.Outputs = expected
assert.Equal(t, old, new)
}
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
snap, res := op.RunWithContext(ctx, project, target, options, false, nil, validate)
assertIsErrorOrBailResult(t, res)
Fix TestCanceledRefresh (#2331) This test had been intermittently failing due to a race condition. Its implementation of `plugin.Provider.Read` was intended to ensure that the cancellation of a refresh operation occurred. As written, it was only able to ensure that the cancellation was requested. These changes ensure that cancellation has been acknowledged by the engine by implementing providing an implementation for `plugin.Provider.Cancel` that closes a channel on which the implementation of `Read` waits.
2019-01-04 04:40:07 +01:00
assert.Equal(t, 1, len(refreshed))
Fix some potential bugs in the refresher. (#1845) - Create all refresh steps before issuing any. This is important as the state update loop expects all steps to exist. - Check for cancellation later in the refresher. This also fixes races in the SnapshotManager and the test journal that could cause panics during cancellation.
2018-08-30 06:00:05 +02:00
provURN := p.NewProviderURN("pkgA", "default", "")
for _, r := range snap.Resources {
switch urn := r.URN; urn {
case provURN:
continue
case urnA, urnB, urnC:
// break
default:
t.Fatalf("unexpected resource %v", urn)
}
idx, err := strconv.ParseInt(string(r.ID), 0, 0)
assert.NoError(t, err)
if refreshed[r.ID] {
// The refreshed resource should have its new state.
expected := newStates[r.ID]
if expected == nil {
assert.Fail(t, "refreshed resource was not deleted")
} else {
old := oldResources[int(idx)]
old.Outputs = expected
assert.Equal(t, old, r)
}
} else {
// Any resources that were not refreshed should retain their original state.
old := oldResources[int(idx)]
assert.Equal(t, old, r)
}
}
Reimplement refresh. (#1814) Replace the Source-based implementation of refresh with a phase that runs as the first part of plan execution and rewrites the snapshot in-memory. In order to fit neatly within the existing framework for resource operations, these changes introduce a new kind of step, RefreshStep, to represent refreshes. RefreshSteps operate similar to ReadSteps but do not imply that the resource being read is not managed by Pulumi. In addition to the refresh reimplementation, these changes incorporate those from #1394 to run refresh in the integration test framework. Fixes #1598. Fixes pulumi/pulumi-terraform#165. Contributes to #1449.
2018-08-23 02:52:46 +02:00
}
Log errors coming from the language host (#1780) * Log errors coming from the language host Similar to pulumi/pulumi#1762, fixes pulumi/pulumi#1775. The language host can fail without issuing any diagnostics and it is very unclear what happens if the engine does not log the error. * CR feedback
2018-08-21 01:41:07 +02:00
// Tests that errors returned directly from the language host get logged by the engine.
func TestLanguageHostDiagnostics(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
errorText := "oh no"
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, _ *deploytest.ResourceMonitor) error {
// Exiting immediately with an error simulates a language exiting immediately with a non-zero exit code.
return errors.New(errorText)
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Log errors coming from the language host (#1780) * Log errors coming from the language host Similar to pulumi/pulumi#1762, fixes pulumi/pulumi#1775. The language host can fail without issuing any diagnostics and it is very unclear what happens if the engine does not log the error. * CR feedback
2018-08-21 01:41:07 +02:00
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{
Op: Update,
ExpectFailure: true,
SkipPreview: true,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assertIsErrorOrBailResult(t, res)
Log errors coming from the language host (#1780) * Log errors coming from the language host Similar to pulumi/pulumi#1762, fixes pulumi/pulumi#1775. The language host can fail without issuing any diagnostics and it is very unclear what happens if the engine does not log the error. * CR feedback
2018-08-21 01:41:07 +02:00
sawExitCode := false
for _, evt := range evts {
if evt.Type == DiagEvent {
e := evt.Payload.(DiagEventPayload)
msg := colors.Never.Colorize(e.Message)
sawExitCode = strings.Contains(msg, errorText) && e.Severity == diag.Error
if sawExitCode {
break
}
}
}
assert.True(t, sawExitCode)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Log errors coming from the language host (#1780) * Log errors coming from the language host Similar to pulumi/pulumi#1762, fixes pulumi/pulumi#1775. The language host can fail without issuing any diagnostics and it is very unclear what happens if the engine does not log the error. * CR feedback
2018-08-21 01:41:07 +02:00
},
}},
}
p.Run(t, nil)
}
Show a better error message when decrypting fails (#1815) * Show a better error message when decrypting fails It is most often the case that failing to decrypt a secret implies that the secret was transferred from one stack to another via copying the configuration. This commit introduces a better error message for this case and instructs users to explicitly re-encrypt their encrypted keys in the context of the new stack. * Spelling * CR: Grammar fixes
2018-08-23 00:32:54 +02:00
type brokenDecrypter struct {
ErrorMessage string
}
func (b brokenDecrypter) DecryptValue(ciphertext string) (string, error) {
return "", fmt.Errorf(b.ErrorMessage)
}
// Tests that the engine presents a reasonable error message when a decrypter fails to decrypt a config value.
func TestBrokenDecrypter(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, _ *deploytest.ResourceMonitor) error {
return nil
})
Implement status sinks This commit reverts most of #1853 and replaces it with functionally identical logic, using the notion of status message-specific sinks. In other words, where the original commit implemented ephemeral status messages by adding an `isStatus` parameter to most of the logging methdos in pulumi/pulumi, this implements ephemeral status messages as a parallel logging sink, which emits _only_ ephemeral status messages. The original commit message in that PR was: > Allow log events to be marked "status" events > > This commit will introduce a field, IsStatus to LogRequest. A "status" > logging event will be displayed in the Info column of the main > display, but will not be printed out at the end, when resource > operations complete. > > For example, for complex resource initialization, we'd like to display > a series of intermediate results: [1/4] Service object created, for > example. We'd like these to appear in the Info column, but not at the > end, where they are not helpful to the user.
2018-08-31 22:12:40 +02:00
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
Show a better error message when decrypting fails (#1815) * Show a better error message when decrypting fails It is most often the case that failing to decrypt a secret implies that the secret was transferred from one stack to another via copying the configuration. This commit introduces a better error message for this case and instructs users to explicitly re-encrypt their encrypted keys in the context of the new stack. * Spelling * CR: Grammar fixes
2018-08-23 00:32:54 +02:00
key := config.MustMakeKey("foo", "bar")
msg := "decryption failed"
configMap := make(config.Map)
configMap[key] = config.NewSecureValue("hunter2")
p := &TestPlan{
Options: UpdateOptions{host: host},
Decrypter: brokenDecrypter{ErrorMessage: msg},
Config: configMap,
Steps: []TestStep{{
Op: Update,
ExpectFailure: true,
SkipPreview: true,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assertIsErrorOrBailResult(t, res)
decryptErr := res.Error().(DecryptError)
Show a better error message when decrypting fails (#1815) * Show a better error message when decrypting fails It is most often the case that failing to decrypt a secret implies that the secret was transferred from one stack to another via copying the configuration. This commit introduces a better error message for this case and instructs users to explicitly re-encrypt their encrypted keys in the context of the new stack. * Spelling * CR: Grammar fixes
2018-08-23 00:32:54 +02:00
assert.Equal(t, key, decryptErr.Key)
assert.Contains(t, decryptErr.Err.Error(), msg)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Show a better error message when decrypting fails (#1815) * Show a better error message when decrypting fails It is most often the case that failing to decrypt a secret implies that the secret was transferred from one stack to another via copying the configuration. This commit introduces a better error message for this case and instructs users to explicitly re-encrypt their encrypted keys in the context of the new stack. * Spelling * CR: Grammar fixes
2018-08-23 00:32:54 +02:00
},
}},
}
p.Run(t, nil)
}
Validate type tokens before using them (#1904) * Validate type tokens before using them When registering or reading a resource, we take the type token given to us from the language host and assume that it's valid, which resulted in assertion failures in various places in the engine. This commit validates the format of type tokens given to us from the language host and issues an appropriate error if it's not valid. Along the way, this commit also improves the way that fatal exceptions are rendered in the Node language host. * Pre-allocate an exception for ReadResource * Fix integration test * CR Feedback This commit is a lower-impact change that fixes the bugs associated with invalid types on component resources and only checks that a type is valid on custom resources. * CR Take 2: Fix up IsProviderType instead of fixing call sites * Please gometalinter
2018-09-08 00:19:18 +02:00
func TestBadResourceType(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, mon *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := mon.RegisterResource("very:bad", "resA", true)
Validate type tokens before using them (#1904) * Validate type tokens before using them When registering or reading a resource, we take the type token given to us from the language host and assume that it's valid, which resulted in assertion failures in various places in the engine. This commit validates the format of type tokens given to us from the language host and issues an appropriate error if it's not valid. Along the way, this commit also improves the way that fatal exceptions are rendered in the Node language host. * Pre-allocate an exception for ReadResource * Fix integration test * CR Feedback This commit is a lower-impact change that fixes the bugs associated with invalid types on component resources and only checks that a type is valid on custom resources. * CR Take 2: Fix up IsProviderType instead of fixing call sites * Please gometalinter
2018-09-08 00:19:18 +02:00
assert.Error(t, err)
rpcerr, ok := rpcerror.FromError(err)
assert.True(t, ok)
assert.Equal(t, codes.InvalidArgument, rpcerr.Code())
assert.Contains(t, rpcerr.Message(), "Type 'very:bad' is not a valid type token")
Load specific provider versions if requested (#2648) * Load specific provider versions if requested As part of pulumi/pulumi#2389, we need the ability for language hosts to tell the engine that a particular resource registration, read, or invoke needs to use a particular version of a resource provider. This was not previously possible before; the engine prior to this commit loaded plugins from a default provider map, which was inferred for every resource provider based on the contents of a user's package.json, and was itself prone to bugs. This PR adds the engine support needed for language hosts to request a particular version of a provider. If this occurs, the source evaluator specifically records the intent to load a provider with a given version and produces a "default" provider registration that requests exactly that version. This allows the source evaluator to produce multiple default providers for a signle package, which was previously not possible. This is accomplished by having the source evaluator deal in the "ProviderRequest" type, which is a tuple of version and package. A request to load a provider whose version matches the package of a previously loaded provider will re-use the existing default provider. If the version was not previously loaded, a new default provider is injected. * CR Feedback: raise error if semver is invalid * CR: call String() if you want a hash key * Update pkg/resource/deploy/providers/provider.go Co-Authored-By: swgillespie <sean@pulumi.com>
2019-04-17 20:25:02 +02:00
_, _, err = mon.ReadResource("very:bad", "someResource", "someId", "", resource.PropertyMap{}, "", "")
Validate type tokens before using them (#1904) * Validate type tokens before using them When registering or reading a resource, we take the type token given to us from the language host and assume that it's valid, which resulted in assertion failures in various places in the engine. This commit validates the format of type tokens given to us from the language host and issues an appropriate error if it's not valid. Along the way, this commit also improves the way that fatal exceptions are rendered in the Node language host. * Pre-allocate an exception for ReadResource * Fix integration test * CR Feedback This commit is a lower-impact change that fixes the bugs associated with invalid types on component resources and only checks that a type is valid on custom resources. * CR Take 2: Fix up IsProviderType instead of fixing call sites * Please gometalinter
2018-09-08 00:19:18 +02:00
assert.Error(t, err)
rpcerr, ok = rpcerror.FromError(err)
assert.True(t, ok)
assert.Equal(t, codes.InvalidArgument, rpcerr.Code())
assert.Contains(t, rpcerr.Message(), "Type 'very:bad' is not a valid type token")
// Component resources may have any format type.
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, noErr := mon.RegisterResource("a:component", "resB", false)
Validate type tokens before using them (#1904) * Validate type tokens before using them When registering or reading a resource, we take the type token given to us from the language host and assume that it's valid, which resulted in assertion failures in various places in the engine. This commit validates the format of type tokens given to us from the language host and issues an appropriate error if it's not valid. Along the way, this commit also improves the way that fatal exceptions are rendered in the Node language host. * Pre-allocate an exception for ReadResource * Fix integration test * CR Feedback This commit is a lower-impact change that fixes the bugs associated with invalid types on component resources and only checks that a type is valid on custom resources. * CR Take 2: Fix up IsProviderType instead of fixing call sites * Please gometalinter
2018-09-08 00:19:18 +02:00
assert.NoError(t, noErr)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, noErr = mon.RegisterResource("singlename", "resC", false)
Validate type tokens before using them (#1904) * Validate type tokens before using them When registering or reading a resource, we take the type token given to us from the language host and assume that it's valid, which resulted in assertion failures in various places in the engine. This commit validates the format of type tokens given to us from the language host and issues an appropriate error if it's not valid. Along the way, this commit also improves the way that fatal exceptions are rendered in the Node language host. * Pre-allocate an exception for ReadResource * Fix integration test * CR Feedback This commit is a lower-impact change that fixes the bugs associated with invalid types on component resources and only checks that a type is valid on custom resources. * CR Take 2: Fix up IsProviderType instead of fixing call sites * Please gometalinter
2018-09-08 00:19:18 +02:00
assert.NoError(t, noErr)
return err
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{
Op: Update,
ExpectFailure: true,
SkipPreview: true,
}},
}
p.Run(t, nil)
}
Fix provider cancellation. (#1914) We signal provider cancellation by hangning a goroutine off of the plan executor's parent context. To ensure clean shutdown, this goroutine also listens on a channel that closes once the plan has finished executing. Unfortunately, we were closing this channel too early, and the close was racing with the cancellation signal. These changes ensure that the channel closes after the plan has fully completed. Fixes #1906. Fixes pulumi/pulumi-kubernetes#185.
2018-09-11 00:18:25 +02:00
// Tests that provider cancellation occurs as expected.
func TestProviderCancellation(t *testing.T) {
const resourceCount = 4
// Set up a cancelable context for the refresh operation.
ctx, cancel := context.WithCancel(context.Background())
// Wait for our resource ops, then cancel.
var ops sync.WaitGroup
ops.Add(resourceCount)
go func() {
ops.Wait()
cancel()
}()
// Set up an independent cancelable context for the provider's operations.
provCtx, provCancel := context.WithCancel(context.Background())
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
CreateF: func(urn resource.URN,
Addition of Custom Timeouts (#2885) * Plumbing the custom timeouts from the engine to the providers * Plumbing the CustomTimeouts through to the engine and adding test to show this * Change the provider proto to include individual timeouts * Plumbing the CustomTimeouts from the engine through to the Provider RPC interface * Change how the CustomTimeouts are sent across RPC These errors were spotted in testing. We can now see that the timeout information is arriving in the RegisterResourceRequest ``` req=&pulumirpc.RegisterResourceRequest{ Type: "aws:s3/bucket:Bucket", Name: "my-bucket", Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev", Custom: true, Object: &structpb.Struct{}, Protect: false, Dependencies: nil, Provider: "", PropertyDependencies: {}, DeleteBeforeReplace: false, Version: "", IgnoreChanges: nil, AcceptSecrets: true, AdditionalSecretOutputs: nil, Aliases: nil, CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{ Create: 300, Update: 400, Delete: 500, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, }, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, } ``` * Changing the design to use strings * CHANGELOG entry to include the CustomTimeouts work * Changing custom timeouts to be passed around the engine as converted value We don't want to pass around strings - the user can provide it but we want to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
inputs resource.PropertyMap, timeout float64) (resource.ID, resource.PropertyMap,
resource.Status, error) {
Fix provider cancellation. (#1914) We signal provider cancellation by hangning a goroutine off of the plan executor's parent context. To ensure clean shutdown, this goroutine also listens on a channel that closes once the plan has finished executing. Unfortunately, we were closing this channel too early, and the close was racing with the cancellation signal. These changes ensure that the channel closes after the plan has fully completed. Fixes #1906. Fixes pulumi/pulumi-kubernetes#185.
2018-09-11 00:18:25 +02:00
// Inform the waiter that we've entered a provider op and wait for cancellation.
ops.Done()
<-provCtx.Done()
return resource.ID(urn.Name()), resource.PropertyMap{}, resource.StatusOK, nil
},
CancelF: func() error {
provCancel()
return nil
},
}, nil
}),
}
done := make(chan bool)
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
errors := make([]error, resourceCount)
var resources sync.WaitGroup
resources.Add(resourceCount)
for i := 0; i < resourceCount; i++ {
go func(idx int) {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, errors[idx] = monitor.RegisterResource("pkgA:m:typA", fmt.Sprintf("res%d", idx), true)
Fix provider cancellation. (#1914) We signal provider cancellation by hangning a goroutine off of the plan executor's parent context. To ensure clean shutdown, this goroutine also listens on a channel that closes once the plan has finished executing. Unfortunately, we were closing this channel too early, and the close was racing with the cancellation signal. These changes ensure that the channel closes after the plan has fully completed. Fixes #1906. Fixes pulumi/pulumi-kubernetes#185.
2018-09-11 00:18:25 +02:00
resources.Done()
}(i)
}
resources.Wait()
for _, err := range errors {
assert.NoError(t, err)
}
close(done)
return nil
})
p := &TestPlan{}
op := TestOp(Update)
options := UpdateOptions{
Parallel: resourceCount,
host: deploytest.NewPluginHost(nil, nil, program, loaders...),
}
project, target := p.GetProject(), p.GetTarget(nil)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
_, res := op.RunWithContext(ctx, project, target, options, false, nil, nil)
assertIsErrorOrBailResult(t, res)
Fix provider cancellation. (#1914) We signal provider cancellation by hangning a goroutine off of the plan executor's parent context. To ensure clean shutdown, this goroutine also listens on a channel that closes once the plan has finished executing. Unfortunately, we were closing this channel too early, and the close was racing with the cancellation signal. These changes ensure that the channel closes after the plan has fully completed. Fixes #1906. Fixes pulumi/pulumi-kubernetes#185.
2018-09-11 00:18:25 +02:00
// Wait for the program to finish.
<-done
}
Allow previews when operations are pending. (#1999) The preview will proceed as if the operations had not been issued (i.e. we will not speculate on a new state for the stack). This is consistent with our behavior prior to the changes that added pending operations to the checkpoint.
2018-10-01 18:48:48 +02:00
// Tests that a preview works for a stack with pending operations.
func TestPreviewWithPendingOperations(t *testing.T) {
p := &TestPlan{}
const resType = "pkgA:m:typA"
urnA := p.NewURN(resType, "resA", "")
newResource := func(urn resource.URN, id resource.ID, delete bool, dependencies ...resource.URN) *resource.State {
return &resource.State{
Type: urn.Type(),
URN: urn,
Custom: true,
Delete: delete,
ID: id,
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
Dependencies: dependencies,
}
}
old := &deploy.Snapshot{
PendingOperations: []resource.Operation{{
Resource: newResource(urnA, "0", false),
Type: resource.OperationTypeUpdating,
}},
Resources: []*resource.State{
newResource(urnA, "0", false),
},
}
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Allow previews when operations are pending. (#1999) The preview will proceed as if the operations had not been issued (i.e. we will not speculate on a new state for the stack). This is consistent with our behavior prior to the changes that added pending operations to the checkpoint.
2018-10-01 18:48:48 +02:00
assert.NoError(t, err)
return nil
})
op := TestOp(Update)
options := UpdateOptions{host: deploytest.NewPluginHost(nil, nil, program, loaders...)}
project, target := p.GetProject(), p.GetTarget(old)
// A preview should succeed despite the pending operations.
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
_, res := op.Run(project, target, options, true, nil, nil)
assert.Nil(t, res)
Allow previews when operations are pending. (#1999) The preview will proceed as if the operations had not been issued (i.e. we will not speculate on a new state for the stack). This is consistent with our behavior prior to the changes that added pending operations to the checkpoint.
2018-10-01 18:48:48 +02:00
// But an update should fail.
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
_, res = op.Run(project, target, options, false, nil, nil)
assertIsErrorOrBailResult(t, res)
assert.EqualError(t, res.Error(), deploy.PlanPendingOperationsError{}.Error())
Allow previews when operations are pending. (#1999) The preview will proceed as if the operations had not been issued (i.e. we will not speculate on a new state for the stack). This is consistent with our behavior prior to the changes that added pending operations to the checkpoint.
2018-10-01 18:48:48 +02:00
}
Save old inputs on partial update failures (#2035) Whenever an update fails partially, it gives the engine back some state bag of outputs that should be persisted to the snapshot. When saving this state, we shouldn't save the inputs that triggered the update that failed, since the resource that exists will never have been updated successfully with those inputs. Instead of saving the new inputs on partial failed updates, this commit saves the old inputs and the new outputs. The new outputs will likely need to be refreshed to be useful, but the old inputs will be correct from the perpsective of the Pulumi program that generated the last successful update. Fixes pulumi/pulumi#2011
2018-10-09 20:19:31 +02:00
// Tests that a failed partial update causes the engine to persist the resource's old inputs and new outputs.
func TestUpdatePartialFailure(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
DiffF: func(urn resource.URN, id resource.ID, olds, news resource.PropertyMap,
ignoreChanges []string) (plugin.DiffResult, error) {
Save old inputs on partial update failures (#2035) Whenever an update fails partially, it gives the engine back some state bag of outputs that should be persisted to the snapshot. When saving this state, we shouldn't save the inputs that triggered the update that failed, since the resource that exists will never have been updated successfully with those inputs. Instead of saving the new inputs on partial failed updates, this commit saves the old inputs and the new outputs. The new outputs will likely need to be refreshed to be useful, but the old inputs will be correct from the perpsective of the Pulumi program that generated the last successful update. Fixes pulumi/pulumi#2011
2018-10-09 20:19:31 +02:00
return plugin.DiffResult{
Changes: plugin.DiffSome,
}, nil
},
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
UpdateF: func(urn resource.URN, id resource.ID, olds, news resource.PropertyMap,
timeout float64, ignoreChanges []string) (resource.PropertyMap, resource.Status, error) {
Save old inputs on partial update failures (#2035) Whenever an update fails partially, it gives the engine back some state bag of outputs that should be persisted to the snapshot. When saving this state, we shouldn't save the inputs that triggered the update that failed, since the resource that exists will never have been updated successfully with those inputs. Instead of saving the new inputs on partial failed updates, this commit saves the old inputs and the new outputs. The new outputs will likely need to be refreshed to be useful, but the old inputs will be correct from the perpsective of the Pulumi program that generated the last successful update. Fixes pulumi/pulumi#2011
2018-10-09 20:19:31 +02:00
outputs := resource.NewPropertyMapFromMap(map[string]interface{}{
"output_prop": 42,
})
return outputs, resource.StatusPartialFailure, errors.New("update failed to apply")
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, mon *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := mon.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Inputs: resource.NewPropertyMapFromMap(map[string]interface{}{
Save old inputs on partial update failures (#2035) Whenever an update fails partially, it gives the engine back some state bag of outputs that should be persisted to the snapshot. When saving this state, we shouldn't save the inputs that triggered the update that failed, since the resource that exists will never have been updated successfully with those inputs. Instead of saving the new inputs on partial failed updates, this commit saves the old inputs and the new outputs. The new outputs will likely need to be refreshed to be useful, but the old inputs will be correct from the perpsective of the Pulumi program that generated the last successful update. Fixes pulumi/pulumi#2011
2018-10-09 20:19:31 +02:00
"input_prop": "new inputs",
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
}),
})
Save old inputs on partial update failures (#2035) Whenever an update fails partially, it gives the engine back some state bag of outputs that should be persisted to the snapshot. When saving this state, we shouldn't save the inputs that triggered the update that failed, since the resource that exists will never have been updated successfully with those inputs. Instead of saving the new inputs on partial failed updates, this commit saves the old inputs and the new outputs. The new outputs will likely need to be refreshed to be useful, but the old inputs will be correct from the perpsective of the Pulumi program that generated the last successful update. Fixes pulumi/pulumi#2011
2018-10-09 20:19:31 +02:00
return err
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{Options: UpdateOptions{host: host}}
resURN := p.NewURN("pkgA:m:typA", "resA", "")
p.Steps = []TestStep{{
Op: Update,
ExpectFailure: true,
SkipPreview: true,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assertIsErrorOrBailResult(t, res)
Save old inputs on partial update failures (#2035) Whenever an update fails partially, it gives the engine back some state bag of outputs that should be persisted to the snapshot. When saving this state, we shouldn't save the inputs that triggered the update that failed, since the resource that exists will never have been updated successfully with those inputs. Instead of saving the new inputs on partial failed updates, this commit saves the old inputs and the new outputs. The new outputs will likely need to be refreshed to be useful, but the old inputs will be correct from the perpsective of the Pulumi program that generated the last successful update. Fixes pulumi/pulumi#2011
2018-10-09 20:19:31 +02:00
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case resURN:
assert.Equal(t, deploy.OpUpdate, entry.Step.Op())
switch entry.Kind {
case JournalEntryBegin:
continue
case JournalEntrySuccess:
inputs := entry.Step.New().Inputs
outputs := entry.Step.New().Outputs
assert.Len(t, inputs, 1)
assert.Len(t, outputs, 1)
assert.Equal(t,
resource.NewStringProperty("old inputs"), inputs[resource.PropertyKey("input_prop")])
assert.Equal(t,
resource.NewNumberProperty(42), outputs[resource.PropertyKey("output_prop")])
default:
t.Fatalf("unexpected journal operation: %d", entry.Kind)
}
}
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Save old inputs on partial update failures (#2035) Whenever an update fails partially, it gives the engine back some state bag of outputs that should be persisted to the snapshot. When saving this state, we shouldn't save the inputs that triggered the update that failed, since the resource that exists will never have been updated successfully with those inputs. Instead of saving the new inputs on partial failed updates, this commit saves the old inputs and the new outputs. The new outputs will likely need to be refreshed to be useful, but the old inputs will be correct from the perpsective of the Pulumi program that generated the last successful update. Fixes pulumi/pulumi#2011
2018-10-09 20:19:31 +02:00
},
}}
old := &deploy.Snapshot{
Resources: []*resource.State{
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "1",
Inputs: resource.NewPropertyMapFromMap(map[string]interface{}{
"input_prop": "old inputs",
}),
Outputs: resource.NewPropertyMapFromMap(map[string]interface{}{
"output_prop": 1,
}),
},
},
}
p.Run(t, old)
}
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
// Tests that the StackReference resource works as intended,
func TestStackReference(t *testing.T) {
loaders := []*deploytest.ProviderLoader{}
// Test that the normal lifecycle works correctly.
program := deploytest.NewLanguageRuntime(func(info plugin.RunInfo, mon *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, state, err := mon.RegisterResource("pulumi:pulumi:StackReference", "other", true, deploytest.ResourceOptions{
Inputs: resource.NewPropertyMapFromMap(map[string]interface{}{
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
"name": "other",
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
}),
})
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
assert.NoError(t, err)
if !info.DryRun {
assert.Equal(t, "bar", state["outputs"].ObjectValue()["foo"].StringValue())
}
return nil
})
p := &TestPlan{
BackendClient: &deploytest.BackendClient{
GetStackOutputsF: func(ctx context.Context, name string) (resource.PropertyMap, error) {
switch name {
case "other":
return resource.NewPropertyMapFromMap(map[string]interface{}{
"foo": "bar",
}), nil
default:
return nil, errors.Errorf("unknown stack \"%s\"", name)
}
},
},
Options: UpdateOptions{host: deploytest.NewPluginHost(nil, nil, program, loaders...)},
Steps: MakeBasicLifecycleSteps(t, 2),
}
p.Run(t, nil)
// Test that changes to `name` cause replacement.
resURN := p.NewURN("pulumi:pulumi:StackReference", "other", "")
old := &deploy.Snapshot{
Resources: []*resource.State{
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: "1",
Inputs: resource.NewPropertyMapFromMap(map[string]interface{}{
"name": "other2",
}),
Outputs: resource.NewPropertyMapFromMap(map[string]interface{}{
"name": "other2",
"outputs": resource.PropertyMap{},
}),
},
},
}
p.Steps = []TestStep{{
Op: Update,
SkipPreview: true,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case resURN:
switch entry.Step.Op() {
case deploy.OpCreateReplacement, deploy.OpDeleteReplaced, deploy.OpReplace:
// OK
default:
t.Fatalf("unexpected journal operation: %v", entry.Step.Op())
}
}
}
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
},
}}
p.Run(t, old)
// Test that unknown stacks are handled appropriately.
program = deploytest.NewLanguageRuntime(func(info plugin.RunInfo, mon *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := mon.RegisterResource("pulumi:pulumi:StackReference", "other", true, deploytest.ResourceOptions{
Inputs: resource.NewPropertyMapFromMap(map[string]interface{}{
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
"name": "rehto",
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
}),
})
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
assert.Error(t, err)
return err
})
p.Options = UpdateOptions{host: deploytest.NewPluginHost(nil, nil, program, loaders...)}
p.Steps = []TestStep{{
Op: Update,
ExpectFailure: true,
SkipPreview: true,
}}
p.Run(t, nil)
// Test that unknown properties cause errors.
program = deploytest.NewLanguageRuntime(func(info plugin.RunInfo, mon *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := mon.RegisterResource("pulumi:pulumi:StackReference", "other", true, deploytest.ResourceOptions{
Inputs: resource.NewPropertyMapFromMap(map[string]interface{}{
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
"name": "other",
"foo": "bar",
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
}),
})
Add an API for importing stack outputs (#2180) These changes add a new resource to the Pulumi SDK, `pulumi.StackReference`, that represents a reference to another stack. This resource has an output property, `outputs`, that contains the complete set of outputs for the referenced stack. The Pulumi account performing the deployment that creates a `StackReference` must have access to the referenced stack or the call will fail. This resource is implemented by a builtin provider managed by the engine. This provider will be used for any custom resources and invokes inside the `pulumi:pulumi` module. Currently this provider supports only the `pulumi:pulumi:StackReference` resource. Fixes #109.
2018-11-14 22:33:35 +01:00
assert.Error(t, err)
return err
})
p.Options = UpdateOptions{host: deploytest.NewPluginHost(nil, nil, program, loaders...)}
p.Run(t, nil)
}
Close plugin context on plan creation failure (#2304) This ensures that the gRPC server is properly shut down. This fixes an issue in which a resource plugin that is still configuring could report log messages to the plugin host, which would in turn attempt to send diagnostic packets over a closed channel, causing a panic. Fixes #2170.
2018-12-18 22:25:52 +01:00
type channelWriter struct {
channel chan []byte
}
func (cw *channelWriter) Write(d []byte) (int, error) {
cw.channel <- d
return len(d), nil
}
// Tests that a failed plugin load correctly shuts down the host.
func TestLoadFailureShutdown(t *testing.T) {
// Note that the setup here is a bit baroque, and is intended to replicate the CLI architecture that lead to
// issue #2170. That issue--a panic on a closed channel--was caused by the intersection of several design choices:
//
// - The provider registry loads and configures the set of providers necessary for the resources currently in the
// checkpoint it is processing at plan creation time. Registry creation fails promptly if a provider plugin
// fails to load (e.g. because is binary is missing).
// - Provider configuration in the CLI's host happens asynchronously. This is meant to allow the engine to remain
// responsive while plugins configure.
// - Providers may call back into the CLI's host for logging. Callbacks are processed as long as the CLI's plugin
// context is open.
// - Log events from the CLI's host are delivered to the CLI's diagnostic streams via channels. The CLI closes
// these channels once the engine operation it initiated completes.
//
// These choices gave rise to the following situation:
// 1. The provider registry loads a provider for package A and kicks off its configuration.
// 2. The provider registry attempts to load a provider for package B. The load fails, and the provider registry
// creation call fails promptly.
// 3. The engine operation requested by the CLI fails promptly because provider registry creation failed.
// 4. The CLI shuts down its diagnostic channels.
// 5. The provider for package A calls back in to the host to log a message. The host then attempts to deliver
// the message to the CLI's diagnostic channels, causing a panic.
//
// The fix was to properly close the plugin host during step (3) s.t. the host was no longer accepting callbacks
// and would not attempt to send messages to the CLI's diagnostic channels.
//
// As such, this test attempts to replicate the CLI architecture by using one provider that configures
// asynchronously and attempts to call back into the engine and a second provider that fails to load.
release, done := make(chan bool), make(chan bool)
sinkWriter := &channelWriter{channel: make(chan []byte)}
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoaderWithHost("pkgA", semver.MustParse("1.0.0"),
func(host plugin.Host) (plugin.Provider, error) {
return &deploytest.Provider{
ConfigureF: func(news resource.PropertyMap) error {
go func() {
<-release
host.Log(diag.Info, "", "configuring pkgA provider...", 0)
close(done)
}()
return nil
},
}, nil
}),
deploytest.NewProviderLoader("pkgB", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return nil, errors.New("pkgB load failure")
}),
}
p := &TestPlan{}
provAURN := p.NewProviderURN("pkgA", "default", "")
provBURN := p.NewProviderURN("pkgB", "default", "")
old := &deploy.Snapshot{
Resources: []*resource.State{
{
Type: provAURN.Type(),
URN: provAURN,
Custom: true,
ID: "0",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
},
{
Type: provBURN.Type(),
URN: provBURN,
Custom: true,
ID: "1",
Inputs: resource.PropertyMap{},
Outputs: resource.PropertyMap{},
},
},
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
return nil
})
op := TestOp(Update)
sink := diag.DefaultSink(sinkWriter, sinkWriter, diag.FormatOptions{Color: colors.Raw})
options := UpdateOptions{host: deploytest.NewPluginHost(sink, sink, program, loaders...)}
project, target := p.GetProject(), p.GetTarget(old)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
_, res := op.Run(project, target, options, true, nil, nil)
assertIsErrorOrBailResult(t, res)
Close plugin context on plan creation failure (#2304) This ensures that the gRPC server is properly shut down. This fixes an issue in which a resource plugin that is still configuring could report log messages to the plugin host, which would in turn attempt to send diagnostic packets over a closed channel, causing a panic. Fixes #2170.
2018-12-18 22:25:52 +01:00
close(sinkWriter.channel)
close(release)
<-done
}
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
func TestDeleteBeforeReplace(t *testing.T) {
// A
// _________|_________
// B C D
// ___|___ ___|___
// E F G H I J
// |__|
// K L
//
// For a given resource R in (A, C, D):
// - R will be the provider for its first dependent
// - A change to R will require that its second dependent be replaced
// - A change to R will not require that its third dependent be replaced
//
// In addition, K will have a requires-replacement property that depends on both F and G, and
// L will have a normal property that depends on both F and G.
//
// With that in mind, the following resources should require replacement: A, B, C, E, F, and K
p := &TestPlan{}
const resType = "pkgA:m:typA"
type propertyDependencies map[resource.PropertyKey][]resource.URN
urnA := p.NewProviderURN("pkgA", "A", "")
urnB := p.NewURN(resType, "B", "")
urnC := p.NewProviderURN("pkgA", "C", "")
urnD := p.NewProviderURN("pkgA", "D", "")
urnE := p.NewURN(resType, "E", "")
urnF := p.NewURN(resType, "F", "")
urnG := p.NewURN(resType, "G", "")
urnH := p.NewURN(resType, "H", "")
urnI := p.NewURN(resType, "I", "")
urnJ := p.NewURN(resType, "J", "")
urnK := p.NewURN(resType, "K", "")
urnL := p.NewURN(resType, "L", "")
newResource := func(urn resource.URN, id resource.ID, provider string, dependencies []resource.URN,
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDeps propertyDependencies, outputs resource.PropertyMap) *resource.State {
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
inputs := resource.PropertyMap{}
for k := range propertyDeps {
inputs[k] = resource.NewStringProperty("foo")
}
return &resource.State{
Type: urn.Type(),
URN: urn,
Custom: true,
Delete: false,
ID: id,
Inputs: inputs,
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
Outputs: outputs,
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
Dependencies: dependencies,
Provider: provider,
PropertyDependencies: propertyDeps,
}
}
old := &deploy.Snapshot{
Resources: []*resource.State{
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
newResource(urnA, "0", "", nil, nil, resource.PropertyMap{"A": resource.NewStringProperty("foo")}),
newResource(urnB, "1", string(urnA)+"::0", nil, nil, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnC, "2", "",
[]resource.URN{urnA},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"A": []resource.URN{urnA}},
resource.PropertyMap{"A": resource.NewStringProperty("bar")}),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnD, "3", "",
[]resource.URN{urnA},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"B": []resource.URN{urnA}}, nil),
newResource(urnE, "4", string(urnC)+"::2", nil, nil, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnF, "5", "",
[]resource.URN{urnC},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"A": []resource.URN{urnC}}, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnG, "6", "",
[]resource.URN{urnC},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"B": []resource.URN{urnC}}, nil),
newResource(urnH, "4", string(urnD)+"::3", nil, nil, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnI, "5", "",
[]resource.URN{urnD},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"A": []resource.URN{urnD}}, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnJ, "6", "",
[]resource.URN{urnD},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"B": []resource.URN{urnD}}, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnK, "7", "",
[]resource.URN{urnF, urnG},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"A": []resource.URN{urnF, urnG}}, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
newResource(urnL, "8", "",
[]resource.URN{urnF, urnG},
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
propertyDependencies{"B": []resource.URN{urnF, urnG}}, nil),
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
},
}
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
Flow `allowUnknows` for Diff/Check Config We pass this information for Diff and Check on specific resources, so we can correctly block unknows from flowing to plugins during applies.
2019-05-23 19:54:18 +02:00
DiffConfigF: func(urn resource.URN, olds, news resource.PropertyMap,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
ignoreChanges []string) (plugin.DiffResult, error) {
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
if !olds["A"].DeepEquals(news["A"]) {
return plugin.DiffResult{
ReplaceKeys: []resource.PropertyKey{"A"},
DeleteBeforeReplace: true,
}, nil
}
return plugin.DiffResult{}, nil
},
DiffF: func(urn resource.URN, id resource.ID,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
if !olds["A"].DeepEquals(news["A"]) {
return plugin.DiffResult{ReplaceKeys: []resource.PropertyKey{"A"}}, nil
}
return plugin.DiffResult{}, nil
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
register := func(urn resource.URN, provider string, inputs resource.PropertyMap) resource.ID {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, id, _, err := monitor.RegisterResource(urn.Type(), string(urn.Name()), true, deploytest.ResourceOptions{
Provider: provider,
Inputs: inputs,
})
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
assert.NoError(t, err)
return id
}
idA := register(urnA, "", resource.PropertyMap{"A": resource.NewStringProperty("bar")})
register(urnB, string(urnA)+"::"+string(idA), nil)
idC := register(urnC, "", nil)
idD := register(urnD, "", nil)
register(urnE, string(urnC)+"::"+string(idC), nil)
register(urnF, "", nil)
register(urnG, "", nil)
register(urnH, string(urnD)+"::"+string(idD), nil)
register(urnI, "", nil)
register(urnJ, "", nil)
register(urnK, "", nil)
register(urnL, "", nil)
return nil
})
p.Options.host = deploytest.NewPluginHost(nil, nil, program, loaders...)
p.Steps = []TestStep{{
Op: Update,
ExpectFailure: false,
SkipPreview: true,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
replaced := make(map[resource.URN]bool)
for _, entry := range j.Entries {
if entry.Step.Op() == deploy.OpReplace {
replaced[entry.Step.URN()] = true
}
}
assert.Equal(t, map[resource.URN]bool{
urnA: true,
urnB: true,
urnC: true,
urnE: true,
urnF: true,
urnK: true,
}, replaced)
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
},
}}
p.Run(t, old)
}
func TestPropertyDependenciesAdapter(t *testing.T) {
// Ensure that the eval source properly shims in property dependencies if none were reported (and does not if
// any were reported).
type propertyDependencies map[resource.PropertyKey][]resource.URN
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
const resType = "pkgA:m:typA"
var urnA, urnB, urnC, urnD resource.URN
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
register := func(name string, inputs resource.PropertyMap, inputDeps propertyDependencies,
dependencies []resource.URN) resource.URN {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
urn, _, _, err := monitor.RegisterResource(resType, name, true, deploytest.ResourceOptions{
Inputs: inputs,
Dependencies: dependencies,
PropertyDeps: inputDeps,
})
Implement more precise delete-before-replace semantics. (#2369) This implements the new algorithm for deciding which resources must be deleted due to a delete-before-replace operation. We need to compute the set of resources that may be replaced by a change to the resource under consideration. We do this by taking the complete set of transitive dependents on the resource under consideration and removing any resources that would not be replaced by changes to their dependencies. We determine whether or not a resource may be replaced by substituting unknowns for input properties that may change due to deletion of the resources their value depends on and calling the resource provider's Diff method. This is perhaps clearer when described by example. Consider the following dependency graph: A __|__ B C | _|_ D E F In this graph, all of B, C, D, E, and F transitively depend on A. It may be the case, however, that changes to the specific properties of any of those resources R that would occur if a resource on the path to A were deleted and recreated may not cause R to be replaced. For example, the edge from B to A may be a simple dependsOn edge such that a change to B does not actually influence any of B's input properties. In that case, neither B nor D would need to be deleted before A could be deleted. In order to make the above algorithm a reality, the resource monitor interface has been updated to include a map that associates an input property key with the list of resources that input property depends on. Older clients of the resource monitor will leave this map empty, in which case all input properties will be treated as depending on all dependencies of the resource. This is probably overly conservative, but it is less conservative than what we currently implement, and is certainly correct.
2019-01-28 18:46:30 +01:00
assert.NoError(t, err)
return urn
}
urnA = register("A", nil, nil, nil)
urnB = register("B", nil, nil, nil)
urnC = register("C", resource.PropertyMap{
"A": resource.NewStringProperty("foo"),
"B": resource.NewStringProperty("bar"),
}, nil, []resource.URN{urnA, urnB})
urnD = register("D", resource.PropertyMap{
"A": resource.NewStringProperty("foo"),
"B": resource.NewStringProperty("bar"),
}, propertyDependencies{
"A": []resource.URN{urnB},
"B": []resource.URN{urnA, urnC},
}, []resource.URN{urnA, urnB, urnC})
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{Op: Update}},
}
snap := p.Run(t, nil)
for _, res := range snap.Resources {
switch res.URN {
case urnA, urnB:
assert.Empty(t, res.Dependencies)
assert.Empty(t, res.PropertyDependencies)
case urnC:
assert.Equal(t, []resource.URN{urnA, urnB}, res.Dependencies)
assert.EqualValues(t, propertyDependencies{
"A": res.Dependencies,
"B": res.Dependencies,
}, res.PropertyDependencies)
case urnD:
assert.Equal(t, []resource.URN{urnA, urnB, urnC}, res.Dependencies)
assert.EqualValues(t, propertyDependencies{
"A": []resource.URN{urnB},
"B": []resource.URN{urnA, urnC},
}, res.PropertyDependencies)
}
}
}
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
func TestExplicitDeleteBeforeReplace(t *testing.T) {
p := &TestPlan{}
Allow users to explicitly disable delete-before-replace. (#3118) With these changes, a user may explicitly set `deleteBeforeReplace` to `false` in order to disable DBR behavior for a particular resource. This is the SDK + CLI escape hatch for cases where the changes in https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable behavior.
2019-08-21 00:51:02 +02:00
dbrDiff := false
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
DiffF: func(urn resource.URN, id resource.ID,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
if !olds["A"].DeepEquals(news["A"]) {
Allow users to explicitly disable delete-before-replace. (#3118) With these changes, a user may explicitly set `deleteBeforeReplace` to `false` in order to disable DBR behavior for a particular resource. This is the SDK + CLI escape hatch for cases where the changes in https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable behavior.
2019-08-21 00:51:02 +02:00
return plugin.DiffResult{
ReplaceKeys: []resource.PropertyKey{"A"},
DeleteBeforeReplace: dbrDiff,
}, nil
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
}
return plugin.DiffResult{}, nil
},
}, nil
}),
}
const resType = "pkgA:index:typ"
inputsA := resource.NewPropertyMapFromMap(map[string]interface{}{"A": "foo"})
Allow users to explicitly disable delete-before-replace. (#3118) With these changes, a user may explicitly set `deleteBeforeReplace` to `false` in order to disable DBR behavior for a particular resource. This is the SDK + CLI escape hatch for cases where the changes in https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable behavior.
2019-08-21 00:51:02 +02:00
dbrValue, dbrA := true, (*bool)(nil)
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
inputsB := resource.NewPropertyMapFromMap(map[string]interface{}{"A": "foo"})
var provURN, urnA, urnB resource.URN
var provID resource.ID
var err error
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
provURN, provID, _, err = monitor.RegisterResource(providers.MakeProviderType("pkgA"), "provA", true)
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
assert.NoError(t, err)
if provID == "" {
provID = providers.UnknownID
}
provRef, err := providers.NewReference(provURN, provID)
assert.NoError(t, err)
provA := provRef.String()
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
urnA, _, _, err = monitor.RegisterResource(resType, "resA", true, deploytest.ResourceOptions{
Provider: provA,
Inputs: inputsA,
DeleteBeforeReplace: dbrA,
})
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
assert.NoError(t, err)
inputDepsB := map[resource.PropertyKey][]resource.URN{"A": {urnA}}
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
urnB, _, _, err = monitor.RegisterResource(resType, "resB", true, deploytest.ResourceOptions{
Provider: provA,
Inputs: inputsB,
Dependencies: []resource.URN{urnA},
PropertyDeps: inputDepsB,
})
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
assert.NoError(t, err)
return nil
})
p.Options.host = deploytest.NewPluginHost(nil, nil, program, loaders...)
p.Steps = []TestStep{{Op: Update}}
snap := p.Run(t, nil)
// Change the value of resA.A. Only resA should be replaced, and the replacement should be create-before-delete.
inputsA["A"] = resource.NewStringProperty("bar")
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
AssertSameSteps(t, []StepSummary{
{Op: deploy.OpSame, URN: provURN},
{Op: deploy.OpCreateReplacement, URN: urnA},
{Op: deploy.OpReplace, URN: urnA},
{Op: deploy.OpSame, URN: urnB},
{Op: deploy.OpDeleteReplaced, URN: urnA},
}, j.SuccessfulSteps())
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
},
}}
snap = p.Run(t, snap)
// Change the registration of resA such that it requires delete-before-replace and change the value of resA.A. Both
// resA and resB should be replaced, and the replacements should be delete-before-replace.
Allow users to explicitly disable delete-before-replace. (#3118) With these changes, a user may explicitly set `deleteBeforeReplace` to `false` in order to disable DBR behavior for a particular resource. This is the SDK + CLI escape hatch for cases where the changes in https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable behavior.
2019-08-21 00:51:02 +02:00
dbrA, inputsA["A"] = &dbrValue, resource.NewStringProperty("baz")
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
AssertSameSteps(t, []StepSummary{
{Op: deploy.OpSame, URN: provURN},
{Op: deploy.OpDeleteReplaced, URN: urnB},
{Op: deploy.OpDeleteReplaced, URN: urnA},
{Op: deploy.OpReplace, URN: urnA},
{Op: deploy.OpCreateReplacement, URN: urnA},
{Op: deploy.OpReplace, URN: urnB},
{Op: deploy.OpCreateReplacement, URN: urnB},
}, j.SuccessfulSteps())
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
},
}}
snap = p.Run(t, snap)
// Change the value of resB.A. Only resB should be replaced, and the replacement should be create-before-delete.
inputsB["A"] = resource.NewStringProperty("qux")
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
AssertSameSteps(t, []StepSummary{
{Op: deploy.OpSame, URN: provURN},
{Op: deploy.OpSame, URN: urnA},
{Op: deploy.OpCreateReplacement, URN: urnB},
{Op: deploy.OpReplace, URN: urnB},
{Op: deploy.OpDeleteReplaced, URN: urnB},
}, j.SuccessfulSteps())
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
},
}}
snap = p.Run(t, snap)
// Change the registration of resA such that it no longer requires delete-before-replace and change the value of
Allow users to explicitly disable delete-before-replace. (#3118) With these changes, a user may explicitly set `deleteBeforeReplace` to `false` in order to disable DBR behavior for a particular resource. This is the SDK + CLI escape hatch for cases where the changes in https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable behavior.
2019-08-21 00:51:02 +02:00
// resA.A. Only resA should be replaced, and the replacement should be create-before-delete.
dbrA, inputsA["A"] = nil, resource.NewStringProperty("zam")
p.Steps = []TestStep{{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
AssertSameSteps(t, []StepSummary{
{Op: deploy.OpSame, URN: provURN},
{Op: deploy.OpCreateReplacement, URN: urnA},
{Op: deploy.OpReplace, URN: urnA},
{Op: deploy.OpSame, URN: urnB},
{Op: deploy.OpDeleteReplaced, URN: urnA},
}, j.SuccessfulSteps())
return res
},
}}
snap = p.Run(t, snap)
// Change the diff of resA such that it requires delete-before-replace and change the value of resA.A. Both
// resA and resB should be replaced, and the replacements should be delete-before-replace.
dbrDiff, inputsA["A"] = true, resource.NewStringProperty("foo")
p.Steps = []TestStep{{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
AssertSameSteps(t, []StepSummary{
{Op: deploy.OpSame, URN: provURN},
{Op: deploy.OpDeleteReplaced, URN: urnB},
{Op: deploy.OpDeleteReplaced, URN: urnA},
{Op: deploy.OpReplace, URN: urnA},
{Op: deploy.OpCreateReplacement, URN: urnA},
{Op: deploy.OpReplace, URN: urnB},
{Op: deploy.OpCreateReplacement, URN: urnB},
}, j.SuccessfulSteps())
return res
},
}}
snap = p.Run(t, snap)
// Change the registration of resA such that it disables delete-before-replace and change the value of
// resA.A. Only resA should be replaced, and the replacement should be create-before-delete.
dbrA, dbrValue, inputsA["A"] = &dbrValue, false, resource.NewStringProperty("bar")
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
p.Steps = []TestStep{{
Op: Update,
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
AssertSameSteps(t, []StepSummary{
{Op: deploy.OpSame, URN: provURN},
{Op: deploy.OpCreateReplacement, URN: urnA},
{Op: deploy.OpReplace, URN: urnA},
{Op: deploy.OpSame, URN: urnB},
{Op: deploy.OpDeleteReplaced, URN: urnA},
}, j.SuccessfulSteps())
Use result.Result in more places. (#2568)
2019-03-20 00:21:50 +01:00
return res
Add support for explicit delete-before-replace (#2415) These changes add a new flag to the various `ResourceOptions` types that indicates that a resource should be deleted before it is replaced, even if the provider does not require this behavior. The usual delete-before-replace cascade semantics apply. Fixes #1620.
2019-01-31 23:27:53 +01:00
},
}}
p.Run(t, snap)
}
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
func TestSingleResourceIgnoreChanges(t *testing.T) {
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
var expectedIgnoreChanges []string
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
return &deploytest.Provider{
DiffF: func(urn resource.URN, id resource.ID,
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
assert.Equal(t, expectedIgnoreChanges, ignoreChanges)
return plugin.DiffResult{}, nil
},
UpdateF: func(urn resource.URN, id resource.ID, olds, news resource.PropertyMap,
timeout float64, ignoreChanges []string) (resource.PropertyMap, resource.Status, error) {
assert.Equal(t, expectedIgnoreChanges, ignoreChanges)
return resource.PropertyMap{}, resource.StatusOK, nil
},
}, nil
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
}),
}
updateProgramWithProps := func(snap *deploy.Snapshot, props resource.PropertyMap, ignoreChanges []string,
allowedOps []deploy.StepOp) *deploy.Snapshot {
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
expectedIgnoreChanges = ignoreChanges
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Inputs: props,
IgnoreChanges: ignoreChanges,
})
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
events []Event, res result.Result) result.Result {
for _, event := range events {
if event.Type == ResourcePreEvent {
payload := event.Payload.(ResourcePreEventPayload)
assert.Subset(t, allowedOps, []deploy.StepOp{payload.Metadata.Op})
}
}
return res
},
},
},
}
return p.Run(t, snap)
}
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
snap := updateProgramWithProps(nil, resource.NewPropertyMapFromMap(map[string]interface{}{
"a": 1,
"b": map[string]interface{}{
"c": "foo",
},
}), []string{"a", "b.c"}, []deploy.StepOp{deploy.OpCreate})
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
// Ensure that a change to an ignored property results in an OpSame
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
snap = updateProgramWithProps(snap, resource.NewPropertyMapFromMap(map[string]interface{}{
"a": 2,
"b": map[string]interface{}{
"c": "bar",
},
}), []string{"a", "b.c"}, []deploy.StepOp{deploy.OpSame})
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
// Ensure that a change to an un-ignored property results in an OpUpdate
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
snap = updateProgramWithProps(snap, resource.NewPropertyMapFromMap(map[string]interface{}{
"a": 3,
"b": map[string]interface{}{
"c": "qux",
},
}), nil, []deploy.StepOp{deploy.OpUpdate})
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
// Ensure that a removing an ignored property results in an OpSame
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
snap = updateProgramWithProps(snap, resource.PropertyMap{}, []string{"a", "b"}, []deploy.StepOp{deploy.OpSame})
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
// Ensure that a removing an un-ignored property results in an OpUpdate
snap = updateProgramWithProps(snap, resource.PropertyMap{}, nil, []deploy.StepOp{deploy.OpUpdate})
// Ensure that adding an ignored property results in an OpSame
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
snap = updateProgramWithProps(snap, resource.NewPropertyMapFromMap(map[string]interface{}{
"a": 4,
"b": map[string]interface{}{
"c": "zed",
},
}), []string{"a", "b"}, []deploy.StepOp{deploy.OpSame})
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
// Ensure that adding an un-ignored property results in an OpUpdate
_ = updateProgramWithProps(snap, resource.PropertyMap{
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
"c": resource.NewNumberProperty(4),
}, []string{"a", "b"}, []deploy.StepOp{deploy.OpUpdate})
Add an ignoreChanges resource option (#2657) Fixes #2277. Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements. This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges. Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 22:54:48 +02:00
}
Support aliases for renaming, re-typing, or re-parenting resources (#2774) Adds a new resource option `aliases` which can be used to rename a resource. When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource. There are two key places this change is implemented. The first is the step generator in the engine. When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource. That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order). This can allow the resource to be matched as a (potential) update to an existing resource with a different URN. The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs. This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases. It is similar to how many other resource options are "inherited" implicitly from the parent. Four specific scenarios are explicitly tested as part of this PR: 1. Renaming a resource 2. Adopting a resource into a component (as the owner of both component and consumption codebases) 3. Renaming a component instance (as the owner of the consumption codebase without changes to the component) 4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase) 4. Combining (1) and (3) to make both changes to a resource at the same time
2019-06-01 08:01:01 +02:00
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
// TestDefaultProviderDiff tests that the engine can gracefully recover whenever a resource's default provider changes
// and there is no diff in the provider's inputs.
func TestDefaultProviderDiff(t *testing.T) {
const resName, resBName = "resA", "resB"
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("0.17.10"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
deploytest.NewProviderLoader("pkgA", semver.MustParse("0.17.11"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
deploytest.NewProviderLoader("pkgA", semver.MustParse("0.17.12"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
runProgram := func(base *deploy.Snapshot, versionA, versionB string, expectedStep deploy.StepOp) *deploy.Snapshot {
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", resName, true, deploytest.ResourceOptions{
Version: versionA,
})
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err = monitor.RegisterResource("pkgA:m:typA", resBName, true, deploytest.ResourceOptions{
Version: versionB,
})
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
events []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
if entry.Kind != JournalEntrySuccess {
continue
}
switch entry.Step.URN().Name().String() {
case resName, resBName:
assert.Equal(t, expectedStep, entry.Step.Op())
}
}
return res
},
},
},
}
return p.Run(t, base)
}
// This test simulates the upgrade scenario of old-style default providers to new-style versioned default providers.
//
// The first update creates a stack using a language host that does not report a version to the engine. As a result,
// the engine makes up a default provider for "pkgA" and calls it "default". It then creates the two resources that
// we are creating and associates them with the default provider.
snap := runProgram(nil, "", "", deploy.OpCreate)
for _, res := range snap.Resources {
switch {
case providers.IsDefaultProvider(res.URN):
assert.Equal(t, "default", res.URN.Name().String())
case res.URN.Name().String() == resName || res.URN.Name().String() == resBName:
provRef, err := providers.ParseReference(res.Provider)
assert.NoError(t, err)
assert.Equal(t, "default", provRef.URN().Name().String())
}
}
// The second update switches to a language host that does report a version to the engine. As a result, the engine
// uses this version to make a new provider, with a different URN, and uses that provider to operate on resA and
// resB.
//
// Despite switching out the provider, the engine should still generate a Same step for resA. It is vital that the
// engine gracefully react to changes in the default provider in this manner. See pulumi/pulumi#2753 for what
// happens when it doesn't.
snap = runProgram(snap, "0.17.10", "0.17.10", deploy.OpSame)
for _, res := range snap.Resources {
switch {
case providers.IsDefaultProvider(res.URN):
assert.Equal(t, "default_0_17_10", res.URN.Name().String())
case res.URN.Name().String() == resName || res.URN.Name().String() == resBName:
provRef, err := providers.ParseReference(res.Provider)
assert.NoError(t, err)
assert.Equal(t, "default_0_17_10", provRef.URN().Name().String())
}
}
// The third update changes the version that the language host reports to the engine. This simulates a scenario in
// which a user updates their SDK to a new version of a provider package. In order to simulate side-by-side
// packages with different versions, this update requests distinct package versions for resA and resB.
snap = runProgram(snap, "0.17.11", "0.17.12", deploy.OpSame)
for _, res := range snap.Resources {
switch {
case providers.IsDefaultProvider(res.URN):
assert.True(t, res.URN.Name().String() == "default_0_17_11" || res.URN.Name().String() == "default_0_17_12")
case res.URN.Name().String() == resName:
provRef, err := providers.ParseReference(res.Provider)
assert.NoError(t, err)
assert.Equal(t, "default_0_17_11", provRef.URN().Name().String())
case res.URN.Name().String() == resBName:
provRef, err := providers.ParseReference(res.Provider)
assert.NoError(t, err)
assert.Equal(t, "default_0_17_12", provRef.URN().Name().String())
}
}
}
// TestDefaultProviderDiffReplacement tests that, when replacing a default provider for a resource, the engine will
// replace the resource if DiffConfig on the new provider returns a diff for the provider's new state.
func TestDefaultProviderDiffReplacement(t *testing.T) {
const resName, resBName = "resA", "resB"
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("0.17.10"), func() (plugin.Provider, error) {
return &deploytest.Provider{
// This implementation of DiffConfig always requests replacement.
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
DiffConfigF: func(_ resource.URN, olds, news resource.PropertyMap,
ignoreChanges []string) (plugin.DiffResult, error) {
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
keys := []resource.PropertyKey{}
for k := range news {
keys = append(keys, k)
}
return plugin.DiffResult{
Changes: plugin.DiffSome,
ReplaceKeys: keys,
}, nil
},
}, nil
}),
deploytest.NewProviderLoader("pkgA", semver.MustParse("0.17.11"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
runProgram := func(base *deploy.Snapshot, versionA, versionB string, expectedSteps ...deploy.StepOp) *deploy.Snapshot {
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", resName, true, deploytest.ResourceOptions{
Version: versionA,
})
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
assert.NoError(t, err)
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err = monitor.RegisterResource("pkgA:m:typA", resBName, true, deploytest.ResourceOptions{
Version: versionB,
})
Refine resource replacement logic for providers (#2767) This commit touches an intersection of a few different provider-oriented features that combined to cause a particularly severe bug that made it impossible for users to upgrade provider versions without seeing replacements with their resources. For some context, Pulumi models all providers as resources and places them in the snapshot like any other resource. Every resource has a reference to the provider that created it. If a Pulumi program does not specify a particular provider to use when performing a resource operation, the Pulumi engine injects one automatically; these are called "default providers" and are the most common ways that users end up with providers in their snapshot. Default providers can be identified by their name, which is always prefixed with "default". Recently, in an effort to make the Pulumi engine more flexible with provider versions, it was made possible for the engine to have multiple default providers active for a provider of a particular type, which was previously not possible. Because a provider is identified as a tuple of package name and version, it was difficult to find a name for these duplicate default providers that did not cause additional problems. The provider versioning PR gave these default providers a name that was derived from the version of the package. This proved to be a problem, because when users upgraded from one version of a package to another, this changed the name of their default provider which in turn caused all of their resources created using that provider (read: everything) to be replaced. To combat this, this PR introduces a rule that the engine will apply when diffing a resource to determine whether or not it needs to be replaced: "If a resource's provider changes, and both old and new providers are default providers whose properties do not require replacement, proceed as if there were no diff." This allows the engine to gracefully recognize and recover when a resource's default provider changes names, as long as the provider's config has not changed.
2019-06-03 21:16:31 +02:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
events []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
if entry.Kind != JournalEntrySuccess {
continue
}
switch entry.Step.URN().Name().String() {
case resName:
assert.Subset(t, expectedSteps, []deploy.StepOp{entry.Step.Op()})
case resBName:
assert.Subset(t,
[]deploy.StepOp{deploy.OpCreate, deploy.OpSame}, []deploy.StepOp{entry.Step.Op()})
}
}
return res
},
},
},
}
return p.Run(t, base)
}
// This test simulates the upgrade scenario of default providers, except that the requested upgrade results in the
// provider getting replaced. Because of this, the engine should decide to replace resA. It should not decide to
// replace resB, as its change does not require replacement.
snap := runProgram(nil, "", "", deploy.OpCreate)
for _, res := range snap.Resources {
switch {
case providers.IsDefaultProvider(res.URN):
assert.Equal(t, "default", res.URN.Name().String())
case res.URN.Name().String() == resName || res.URN.Name().String() == resBName:
provRef, err := providers.ParseReference(res.Provider)
assert.NoError(t, err)
assert.Equal(t, "default", provRef.URN().Name().String())
}
}
// Upon update, now that the language host is sending a version, DiffConfig reports that there's a diff between the
// old and new provider and so we must replace resA.
snap = runProgram(snap, "0.17.10", "0.17.11", deploy.OpCreateReplacement, deploy.OpReplace, deploy.OpDeleteReplaced)
for _, res := range snap.Resources {
switch {
case providers.IsDefaultProvider(res.URN):
assert.True(t, res.URN.Name().String() == "default_0_17_10" || res.URN.Name().String() == "default_0_17_11")
case res.URN.Name().String() == resName:
provRef, err := providers.ParseReference(res.Provider)
assert.NoError(t, err)
assert.Equal(t, "default_0_17_10", provRef.URN().Name().String())
case res.URN.Name().String() == resBName:
provRef, err := providers.ParseReference(res.Provider)
assert.NoError(t, err)
assert.Equal(t, "default_0_17_11", provRef.URN().Name().String())
}
}
}
Support aliases for renaming, re-typing, or re-parenting resources (#2774) Adds a new resource option `aliases` which can be used to rename a resource. When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource. There are two key places this change is implemented. The first is the step generator in the engine. When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource. That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order). This can allow the resource to be matched as a (potential) update to an existing resource with a different URN. The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs. This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases. It is similar to how many other resource options are "inherited" implicitly from the parent. Four specific scenarios are explicitly tested as part of this PR: 1. Renaming a resource 2. Adopting a resource into a component (as the owner of both component and consumption codebases) 3. Renaming a component instance (as the owner of the consumption codebase without changes to the component) 4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase) 4. Combining (1) and (3) to make both changes to a resource at the same time
2019-06-01 08:01:01 +02:00
// Resource is an abstract representation of a resource graph
type Resource struct {
t tokens.Type
name string
children []Resource
props resource.PropertyMap
aliases []resource.URN
dependencies []resource.URN
parent resource.URN
deleteBeforeReplace bool
}
func registerResources(t *testing.T, monitor *deploytest.ResourceMonitor, resources []Resource) error {
for _, r := range resources {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource(r.t, r.name, true, deploytest.ResourceOptions{
Parent: r.parent,
Dependencies: r.dependencies,
Inputs: r.props,
Allow users to explicitly disable delete-before-replace. (#3118) With these changes, a user may explicitly set `deleteBeforeReplace` to `false` in order to disable DBR behavior for a particular resource. This is the SDK + CLI escape hatch for cases where the changes in https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable behavior.
2019-08-21 00:51:02 +02:00
DeleteBeforeReplace: &r.deleteBeforeReplace,
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
Aliases: r.aliases,
})
Support aliases for renaming, re-typing, or re-parenting resources (#2774) Adds a new resource option `aliases` which can be used to rename a resource. When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource. There are two key places this change is implemented. The first is the step generator in the engine. When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource. That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order). This can allow the resource to be matched as a (potential) update to an existing resource with a different URN. The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs. This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases. It is similar to how many other resource options are "inherited" implicitly from the parent. Four specific scenarios are explicitly tested as part of this PR: 1. Renaming a resource 2. Adopting a resource into a component (as the owner of both component and consumption codebases) 3. Renaming a component instance (as the owner of the consumption codebase without changes to the component) 4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase) 4. Combining (1) and (3) to make both changes to a resource at the same time
2019-06-01 08:01:01 +02:00
if err != nil {
return err
}
err = registerResources(t, monitor, r.children)
if err != nil {
return err
}
}
return nil
}
func TestAliases(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
// The `forcesReplacement` key forces replacement and all other keys can update in place
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
DiffF: func(res resource.URN, id resource.ID, olds, news resource.PropertyMap,
ignoreChanges []string) (plugin.DiffResult, error) {
Support aliases for renaming, re-typing, or re-parenting resources (#2774) Adds a new resource option `aliases` which can be used to rename a resource. When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource. There are two key places this change is implemented. The first is the step generator in the engine. When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource. That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order). This can allow the resource to be matched as a (potential) update to an existing resource with a different URN. The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs. This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases. It is similar to how many other resource options are "inherited" implicitly from the parent. Four specific scenarios are explicitly tested as part of this PR: 1. Renaming a resource 2. Adopting a resource into a component (as the owner of both component and consumption codebases) 3. Renaming a component instance (as the owner of the consumption codebase without changes to the component) 4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase) 4. Combining (1) and (3) to make both changes to a resource at the same time
2019-06-01 08:01:01 +02:00
replaceKeys := []resource.PropertyKey{}
old, hasOld := olds["forcesReplacement"]
new, hasNew := news["forcesReplacement"]
if hasOld && !hasNew || hasNew && !hasOld || hasOld && hasNew && old.Diff(new) != nil {
replaceKeys = append(replaceKeys, "forcesReplacement")
}
return plugin.DiffResult{ReplaceKeys: replaceKeys}, nil
},
}, nil
}),
}
updateProgramWithResource := func(
snap *deploy.Snapshot, resources []Resource, allowedOps []deploy.StepOp) *deploy.Snapshot {
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
err := registerResources(t, monitor, resources)
return err
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{
{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
events []Event, res result.Result) result.Result {
for _, event := range events {
if event.Type == ResourcePreEvent {
payload := event.Payload.(ResourcePreEventPayload)
assert.Subset(t, allowedOps, []deploy.StepOp{payload.Metadata.Op})
}
}
for _, entry := range j.Entries {
if entry.Step.Type() == "pulumi:providers:pkgA" {
continue
}
switch entry.Kind {
case JournalEntrySuccess:
assert.Subset(t, allowedOps, []deploy.StepOp{entry.Step.Op()})
case JournalEntryFailure:
assert.Fail(t, "unexpected failure in journal")
case JournalEntryBegin:
case JournalEntryOutputs:
}
}
return res
},
},
},
}
return p.Run(t, snap)
}
snap := updateProgramWithResource(nil, []Resource{{
t: "pkgA:index:t1",
name: "n1",
}}, []deploy.StepOp{deploy.OpCreate})
// Ensure that rename produces Same
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t1",
name: "n2",
aliases: []resource.URN{"urn:pulumi:test::test::pkgA:index:t1::n1"},
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that rename produces Same with multiple aliases
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t1",
name: "n3",
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n1",
"urn:pulumi:test::test::pkgA:index:t1::n2",
},
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that rename produces Same with multiple aliases (reversed)
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t1",
name: "n3",
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n2",
"urn:pulumi:test::test::pkgA:index:t1::n1",
},
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that aliasing back to original name is okay
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t1",
name: "n1",
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n3",
"urn:pulumi:test::test::pkgA:index:t1::n2",
"urn:pulumi:test::test::pkgA:index:t1::n1",
},
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that removing aliases is okay (once old names are gone from all snapshots)
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t1",
name: "n1",
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that changing the type works
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t2",
name: "n1",
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n1",
},
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that changing the type again works
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:othermod:t3",
name: "n1",
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n1",
"urn:pulumi:test::test::pkgA:index:t2::n1",
},
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that order of aliases doesn't matter
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:othermod:t3",
name: "n1",
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n1",
"urn:pulumi:test::test::pkgA:othermod:t3::n1",
"urn:pulumi:test::test::pkgA:index:t2::n1",
},
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that removing aliases is okay (once old names are gone from all snapshots)
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:othermod:t3",
name: "n1",
}}, []deploy.StepOp{deploy.OpSame})
// Ensure that changing everything (including props) leads to update not delete and re-create
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t4",
name: "n2",
props: resource.PropertyMap{
resource.PropertyKey("x"): resource.NewNumberProperty(42),
},
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:othermod:t3::n1",
},
}}, []deploy.StepOp{deploy.OpUpdate})
// Ensure that changing everything again (including props) leads to update not delete and re-create
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t5",
name: "n3",
props: resource.PropertyMap{
resource.PropertyKey("x"): resource.NewNumberProperty(1000),
},
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t4::n2",
},
}}, []deploy.StepOp{deploy.OpUpdate})
// Ensure that changing a forceNew property while also changing type and name leads to replacement not delete+create
snap = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t6",
name: "n4",
props: resource.PropertyMap{
resource.PropertyKey("forcesReplacement"): resource.NewNumberProperty(1000),
},
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t5::n3",
},
}}, []deploy.StepOp{deploy.OpReplace, deploy.OpCreateReplacement, deploy.OpDeleteReplaced})
// Ensure that changing a forceNew property and deleteBeforeReplace while also changing type and name leads to
// replacement not delete+create
_ = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t7",
name: "n5",
props: resource.PropertyMap{
resource.PropertyKey("forcesReplacement"): resource.NewNumberProperty(999),
},
deleteBeforeReplace: true,
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t6::n4",
},
}}, []deploy.StepOp{deploy.OpReplace, deploy.OpCreateReplacement, deploy.OpDeleteReplaced})
// Start again - this time with two resources with depends on relationship
snap = updateProgramWithResource(nil, []Resource{{
t: "pkgA:index:t1",
name: "n1",
props: resource.PropertyMap{
resource.PropertyKey("forcesReplacement"): resource.NewNumberProperty(1),
},
deleteBeforeReplace: true,
}, {
t: "pkgA:index:t2",
name: "n2",
dependencies: []resource.URN{"urn:pulumi:test::test::pkgA:index:t1::n1"},
}}, []deploy.StepOp{deploy.OpCreate})
_ = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t1-new",
name: "n1-new",
props: resource.PropertyMap{
resource.PropertyKey("forcesReplacement"): resource.NewNumberProperty(2),
},
deleteBeforeReplace: true,
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n1",
},
}, {
t: "pkgA:index:t2-new",
name: "n2-new",
dependencies: []resource.URN{"urn:pulumi:test::test::pkgA:index:t1-new::n1-new"},
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t2::n2",
},
}}, []deploy.StepOp{deploy.OpSame, deploy.OpReplace, deploy.OpCreateReplacement, deploy.OpDeleteReplaced})
// Start again - this time with two resources with parent relationship
snap = updateProgramWithResource(nil, []Resource{{
t: "pkgA:index:t1",
name: "n1",
props: resource.PropertyMap{
resource.PropertyKey("forcesReplacement"): resource.NewNumberProperty(1),
},
deleteBeforeReplace: true,
}, {
t: "pkgA:index:t2",
name: "n2",
parent: resource.URN("urn:pulumi:test::test::pkgA:index:t1::n1"),
}}, []deploy.StepOp{deploy.OpCreate})
_ = updateProgramWithResource(snap, []Resource{{
t: "pkgA:index:t1-new",
name: "n1-new",
props: resource.PropertyMap{
resource.PropertyKey("forcesReplacement"): resource.NewNumberProperty(2),
},
deleteBeforeReplace: true,
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1::n1",
},
}, {
t: "pkgA:index:t2-new",
name: "n2-new",
parent: resource.URN("urn:pulumi:test::test::pkgA:index:t1-new::n1-new"),
aliases: []resource.URN{
"urn:pulumi:test::test::pkgA:index:t1$pkgA:index:t2::n2",
},
}}, []deploy.StepOp{deploy.OpSame, deploy.OpReplace, deploy.OpCreateReplacement, deploy.OpDeleteReplaced})
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 21:34:19 +02:00
}
func TestPersistentDiff(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
DiffF: func(urn resource.URN, id resource.ID,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 21:34:19 +02:00
return plugin.DiffResult{Changes: plugin.DiffSome}, nil
},
}, nil
}),
}
inputs := resource.PropertyMap{}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Inputs: inputs,
})
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 21:34:19 +02:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
}
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Run the initial update.
project := p.GetProject()
snap, res := TestOp(Update).Run(project, p.GetTarget(nil), p.Options, false, p.BackendClient, nil)
assert.Nil(t, res)
// First, make no change to the inputs and run a preview. We should see an update to the resource due to
// provider diffing.
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, true, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, _ *Journal,
events []Event, res result.Result) result.Result {
found := false
for _, e := range events {
if e.Type == ResourcePreEvent {
p := e.Payload.(ResourcePreEventPayload).Metadata
if p.URN == resURN {
assert.Equal(t, deploy.OpUpdate, p.Op)
found = true
}
}
}
assert.True(t, found)
return res
})
assert.Nil(t, res)
// Next, enable legacy diff behavior. We should see no changes to the resource.
p.Options.UseLegacyDiff = true
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, true, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, _ *Journal,
events []Event, res result.Result) result.Result {
found := false
for _, e := range events {
if e.Type == ResourcePreEvent {
p := e.Payload.(ResourcePreEventPayload).Metadata
if p.URN == resURN {
assert.Equal(t, deploy.OpSame, p.Op)
found = true
}
}
}
assert.True(t, found)
return res
})
assert.Nil(t, res)
}
func TestDetailedDiffReplace(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
DiffF: func(urn resource.URN, id resource.ID,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 21:34:19 +02:00
return plugin.DiffResult{
Changes: plugin.DiffSome,
DetailedDiff: map[string]plugin.PropertyDiff{
"prop": {Kind: plugin.DiffAddReplace},
},
}, nil
},
}, nil
}),
}
Support aliases for renaming, re-typing, or re-parenting resources (#2774) Adds a new resource option `aliases` which can be used to rename a resource. When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource. There are two key places this change is implemented. The first is the step generator in the engine. When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource. That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order). This can allow the resource to be matched as a (potential) update to an existing resource with a different URN. The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs. This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases. It is similar to how many other resource options are "inherited" implicitly from the parent. Four specific scenarios are explicitly tested as part of this PR: 1. Renaming a resource 2. Adopting a resource into a component (as the owner of both component and consumption codebases) 3. Renaming a component instance (as the owner of the consumption codebase without changes to the component) 4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase) 4. Combining (1) and (3) to make both changes to a resource at the same time
2019-06-01 08:01:01 +02:00
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 21:34:19 +02:00
inputs := resource.PropertyMap{}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Inputs: inputs,
})
Defer all diffs to resource providers. (#2849) Thse changes make a subtle but critical adjustment to the process the Pulumi engine uses to determine whether or not a difference exists between a resource's actual and desired states, and adjusts the way this difference is calculated and displayed accordingly. Today, the Pulumi engine get the first chance to decide whether or not there is a difference between a resource's actual and desired states. It does this by comparing the current set of inputs for a resource (i.e. the inputs from the running Pulumi program) with the last set of inputs used to update the resource. If there is no difference between the old and new inputs, the engine decides that no change is necessary without consulting the resource's provider. Only if there are changes does the engine consult the resource's provider for more information about the difference. This can be problematic for a number of reasons: - Not all providers do input-input comparison; some do input-state comparison - Not all providers are able to update the last deployed set of inputs when performing a refresh - Some providers--either intentionally or due to bugs--may see changes in resources whose inputs have not changed All of these situations are confusing at the very least, and the first is problematic with respect to correctness. Furthermore, the display code only renders diffs it observes rather than rendering the diffs observed by the provider, which can obscure the actual changes detected at runtime. These changes address both of these issues: - Rather than comparing the current inputs against the last inputs before calling a resource provider's Diff function, the engine calls the Diff function in all cases. - Providers may now return a list of properties that differ between the requested and actual state and the way in which they differ. This information will then be used by the CLI to render the diff appropriately. A provider may also indicate that a particular diff is between old and new inputs rather than old state and new inputs. Fixes #2453.
2019-07-01 21:34:19 +02:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
}
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Run the initial update.
project := p.GetProject()
snap, res := TestOp(Update).Run(project, p.GetTarget(nil), p.Options, false, p.BackendClient, nil)
assert.Nil(t, res)
// First, make no change to the inputs and run a preview. We should see an update to the resource due to
// provider diffing.
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, true, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, _ *Journal,
events []Event, res result.Result) result.Result {
found := false
for _, e := range events {
if e.Type == ResourcePreEvent {
p := e.Payload.(ResourcePreEventPayload).Metadata
if p.URN == resURN && p.Op == deploy.OpReplace {
found = true
}
}
}
assert.True(t, found)
return res
})
assert.Nil(t, res)
Support aliases for renaming, re-typing, or re-parenting resources (#2774) Adds a new resource option `aliases` which can be used to rename a resource. When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource. There are two key places this change is implemented. The first is the step generator in the engine. When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource. That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order). This can allow the resource to be matched as a (potential) update to an existing resource with a different URN. The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs. This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases. It is similar to how many other resource options are "inherited" implicitly from the parent. Four specific scenarios are explicitly tested as part of this PR: 1. Renaming a resource 2. Adopting a resource into a component (as the owner of both component and consumption codebases) 3. Renaming a component instance (as the owner of the consumption codebase without changes to the component) 4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase) 4. Combining (1) and (3) to make both changes to a resource at the same time
2019-06-01 08:01:01 +02:00
}
Add support for importing existing resources. (#2893) A resource can be imported by setting the `import` property in the resource options bag when instantiating a resource. In order to successfully import a resource, its desired configuration (i.e. its inputs) must not differ from its actual configuration (i.e. its state) as calculated by the resource's provider. There are a few interesting state transitions hiding here when importing a resource: 1. No prior resource exists in the checkpoint file. In this case, the resource is simply imported. 2. An external resource exists in the checkpoint file. In this case, the resource is imported and the old external state is discarded. 3. A non-external resource exists in the checkpoint file and its ID is different from the ID to import. In this case, the new resource is imported and the old resource is deleted. 4. A non-external resource exists in the checkpoint file, but the ID is the same as the ID to import. In this case, the import ID is ignored and the resource is treated as it would be in all cases except for changes that would replace the resource. In that case, the step generator issues an error that indicates that the import ID should be removed: were we to move forward with the replace, the new state of the stack would fall under case (3), which is almost certainly not what the user intends. Fixes #1662.
2019-07-12 20:12:01 +02:00
func TestImport(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
DiffF: func(urn resource.URN, id resource.ID,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
Add support for importing existing resources. (#2893) A resource can be imported by setting the `import` property in the resource options bag when instantiating a resource. In order to successfully import a resource, its desired configuration (i.e. its inputs) must not differ from its actual configuration (i.e. its state) as calculated by the resource's provider. There are a few interesting state transitions hiding here when importing a resource: 1. No prior resource exists in the checkpoint file. In this case, the resource is simply imported. 2. An external resource exists in the checkpoint file. In this case, the resource is imported and the old external state is discarded. 3. A non-external resource exists in the checkpoint file and its ID is different from the ID to import. In this case, the new resource is imported and the old resource is deleted. 4. A non-external resource exists in the checkpoint file, but the ID is the same as the ID to import. In this case, the import ID is ignored and the resource is treated as it would be in all cases except for changes that would replace the resource. In that case, the step generator issues an error that indicates that the import ID should be removed: were we to move forward with the replace, the new state of the stack would fall under case (3), which is almost certainly not what the user intends. Fixes #1662.
2019-07-12 20:12:01 +02:00
if olds["foo"].DeepEquals(news["foo"]) {
return plugin.DiffResult{Changes: plugin.DiffNone}, nil
}
diffKind := plugin.DiffUpdate
if news["foo"].IsString() && news["foo"].StringValue() == "replace" {
diffKind = plugin.DiffUpdateReplace
}
return plugin.DiffResult{
Changes: plugin.DiffSome,
DetailedDiff: map[string]plugin.PropertyDiff{
"foo": {Kind: diffKind},
},
}, nil
},
CreateF: func(urn resource.URN,
Addition of Custom Timeouts (#2885) * Plumbing the custom timeouts from the engine to the providers * Plumbing the CustomTimeouts through to the engine and adding test to show this * Change the provider proto to include individual timeouts * Plumbing the CustomTimeouts from the engine through to the Provider RPC interface * Change how the CustomTimeouts are sent across RPC These errors were spotted in testing. We can now see that the timeout information is arriving in the RegisterResourceRequest ``` req=&pulumirpc.RegisterResourceRequest{ Type: "aws:s3/bucket:Bucket", Name: "my-bucket", Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev", Custom: true, Object: &structpb.Struct{}, Protect: false, Dependencies: nil, Provider: "", PropertyDependencies: {}, DeleteBeforeReplace: false, Version: "", IgnoreChanges: nil, AcceptSecrets: true, AdditionalSecretOutputs: nil, Aliases: nil, CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{ Create: 300, Update: 400, Delete: 500, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, }, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, } ``` * Changing the design to use strings * CHANGELOG entry to include the CustomTimeouts work * Changing custom timeouts to be passed around the engine as converted value We don't want to pass around strings - the user can provide it but we want to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
news resource.PropertyMap, timeout float64) (resource.ID, resource.PropertyMap, resource.Status, error) {
Add support for importing existing resources. (#2893) A resource can be imported by setting the `import` property in the resource options bag when instantiating a resource. In order to successfully import a resource, its desired configuration (i.e. its inputs) must not differ from its actual configuration (i.e. its state) as calculated by the resource's provider. There are a few interesting state transitions hiding here when importing a resource: 1. No prior resource exists in the checkpoint file. In this case, the resource is simply imported. 2. An external resource exists in the checkpoint file. In this case, the resource is imported and the old external state is discarded. 3. A non-external resource exists in the checkpoint file and its ID is different from the ID to import. In this case, the new resource is imported and the old resource is deleted. 4. A non-external resource exists in the checkpoint file, but the ID is the same as the ID to import. In this case, the import ID is ignored and the resource is treated as it would be in all cases except for changes that would replace the resource. In that case, the step generator issues an error that indicates that the import ID should be removed: were we to move forward with the replace, the new state of the stack would fall under case (3), which is almost certainly not what the user intends. Fixes #1662.
2019-07-12 20:12:01 +02:00
return "created-id", news, resource.StatusOK, nil
},
ReadF: func(urn resource.URN, id resource.ID,
inputs, state resource.PropertyMap) (plugin.ReadResult, resource.Status, error) {
return plugin.ReadResult{
Inputs: resource.PropertyMap{
"foo": resource.NewStringProperty("bar"),
},
Outputs: resource.PropertyMap{
"foo": resource.NewStringProperty("bar"),
},
}, resource.StatusOK, nil
},
}, nil
}),
}
readID, importID, inputs := resource.ID(""), resource.ID("id"), resource.PropertyMap{}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
var err error
if readID != "" {
_, _, err = monitor.ReadResource("pkgA:m:typA", "resA", readID, "", resource.PropertyMap{}, "", "")
} else {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err = monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Inputs: inputs,
ImportID: importID,
})
Add support for importing existing resources. (#2893) A resource can be imported by setting the `import` property in the resource options bag when instantiating a resource. In order to successfully import a resource, its desired configuration (i.e. its inputs) must not differ from its actual configuration (i.e. its state) as calculated by the resource's provider. There are a few interesting state transitions hiding here when importing a resource: 1. No prior resource exists in the checkpoint file. In this case, the resource is simply imported. 2. An external resource exists in the checkpoint file. In this case, the resource is imported and the old external state is discarded. 3. A non-external resource exists in the checkpoint file and its ID is different from the ID to import. In this case, the new resource is imported and the old resource is deleted. 4. A non-external resource exists in the checkpoint file, but the ID is the same as the ID to import. In this case, the import ID is ignored and the resource is treated as it would be in all cases except for changes that would replace the resource. In that case, the step generator issues an error that indicates that the import ID should be removed: were we to move forward with the replace, the new state of the stack would fall under case (3), which is almost certainly not what the user intends. Fixes #1662.
2019-07-12 20:12:01 +02:00
}
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
}
provURN := p.NewProviderURN("pkgA", "default", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Run the initial update. The import should fail due to a mismatch in inputs between the program and the
// actual resource state.
project := p.GetProject()
_, res := TestOp(Update).Run(project, p.GetTarget(nil), p.Options, false, p.BackendClient, nil)
assert.NotNil(t, res)
// Run a second update after fixing the inputs. The import should succeed.
inputs["foo"] = resource.NewStringProperty("bar")
snap, res := TestOp(Update).Run(project, p.GetTarget(nil), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN:
assert.Equal(t, deploy.OpCreate, entry.Step.Op())
case resURN:
assert.Equal(t, deploy.OpImport, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
assert.Len(t, snap.Resources, 2)
// Now, run another update. The update should succeed and there should be no diffs.
snap, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN, resURN:
assert.Equal(t, deploy.OpSame, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
// Change a property value and run a third update. The update should succeed.
inputs["foo"] = resource.NewStringProperty("rab")
snap, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN:
assert.Equal(t, deploy.OpSame, entry.Step.Op())
case resURN:
assert.Equal(t, deploy.OpUpdate, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
// Change the property value s.t. the resource requires replacement. The update should fail.
inputs["foo"] = resource.NewStringProperty("replace")
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, false, p.BackendClient, nil)
assert.NotNil(t, res)
// Finally, destroy the stack. The `Delete` function should be called.
_, res = TestOp(Destroy).Run(project, p.GetTarget(snap), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN, resURN:
assert.Equal(t, deploy.OpDelete, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
// Now clear the ID to import and run an initial update to create a resource that we will import-replace.
importID, inputs["foo"] = "", resource.NewStringProperty("bar")
snap, res = TestOp(Update).Run(project, p.GetTarget(nil), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN, resURN:
assert.Equal(t, deploy.OpCreate, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
assert.Len(t, snap.Resources, 2)
// Set the import ID to the same ID as the existing resource and run an update. This should produce no changes.
for _, r := range snap.Resources {
if r.URN == resURN {
importID = r.ID
}
}
snap, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN, resURN:
assert.Equal(t, deploy.OpSame, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
// Then set the import ID and run another update. The update should succeed and should show an import-replace and
// a delete-replaced.
importID = "id"
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN:
assert.Equal(t, deploy.OpSame, entry.Step.Op())
case resURN:
switch entry.Step.Op() {
case deploy.OpReplace, deploy.OpImportReplacement:
assert.Equal(t, importID, entry.Step.New().ID)
case deploy.OpDeleteReplaced:
assert.NotEqual(t, importID, entry.Step.Old().ID)
}
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
// Change the program to read a resource rather than creating one.
readID = "id"
snap, res = TestOp(Update).Run(project, p.GetTarget(nil), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN:
assert.Equal(t, deploy.OpCreate, entry.Step.Op())
case resURN:
assert.Equal(t, deploy.OpRead, entry.Step.Op())
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
assert.Len(t, snap.Resources, 2)
// Now have the program import the resource. We should see an import-replace and a read-discard.
readID, importID = "", readID
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, false, p.BackendClient,
func(_ workspace.Project, _ deploy.Target, j *Journal, _ []Event, res result.Result) result.Result {
for _, entry := range j.Entries {
switch urn := entry.Step.URN(); urn {
case provURN:
assert.Equal(t, deploy.OpSame, entry.Step.Op())
case resURN:
switch entry.Step.Op() {
case deploy.OpReplace, deploy.OpImportReplacement:
assert.Equal(t, importID, entry.Step.New().ID)
case deploy.OpDiscardReplaced:
assert.Equal(t, importID, entry.Step.Old().ID)
}
default:
t.Fatalf("unexpected resource %v", urn)
}
}
return res
})
assert.Nil(t, res)
}
Addition of Custom Timeouts (#2885) * Plumbing the custom timeouts from the engine to the providers * Plumbing the CustomTimeouts through to the engine and adding test to show this * Change the provider proto to include individual timeouts * Plumbing the CustomTimeouts from the engine through to the Provider RPC interface * Change how the CustomTimeouts are sent across RPC These errors were spotted in testing. We can now see that the timeout information is arriving in the RegisterResourceRequest ``` req=&pulumirpc.RegisterResourceRequest{ Type: "aws:s3/bucket:Bucket", Name: "my-bucket", Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev", Custom: true, Object: &structpb.Struct{}, Protect: false, Dependencies: nil, Provider: "", PropertyDependencies: {}, DeleteBeforeReplace: false, Version: "", IgnoreChanges: nil, AcceptSecrets: true, AdditionalSecretOutputs: nil, Aliases: nil, CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{ Create: 300, Update: 400, Delete: 500, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, }, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, } ``` * Changing the design to use strings * CHANGELOG entry to include the CustomTimeouts work * Changing custom timeouts to be passed around the engine as converted value We don't want to pass around strings - the user can provide it but we want to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
func TestCustomTimeouts(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
CustomTimeouts: &resource.CustomTimeouts{
Addition of Custom Timeouts (#2885) * Plumbing the custom timeouts from the engine to the providers * Plumbing the CustomTimeouts through to the engine and adding test to show this * Change the provider proto to include individual timeouts * Plumbing the CustomTimeouts from the engine through to the Provider RPC interface * Change how the CustomTimeouts are sent across RPC These errors were spotted in testing. We can now see that the timeout information is arriving in the RegisterResourceRequest ``` req=&pulumirpc.RegisterResourceRequest{ Type: "aws:s3/bucket:Bucket", Name: "my-bucket", Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev", Custom: true, Object: &structpb.Struct{}, Protect: false, Dependencies: nil, Provider: "", PropertyDependencies: {}, DeleteBeforeReplace: false, Version: "", IgnoreChanges: nil, AcceptSecrets: true, AdditionalSecretOutputs: nil, Aliases: nil, CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{ Create: 300, Update: 400, Delete: 500, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, }, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, } ``` * Changing the design to use strings * CHANGELOG entry to include the CustomTimeouts work * Changing custom timeouts to be passed around the engine as converted value We don't want to pass around strings - the user can provide it but we want to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
Create: 60, Delete: 60, Update: 240,
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
},
})
Addition of Custom Timeouts (#2885) * Plumbing the custom timeouts from the engine to the providers * Plumbing the CustomTimeouts through to the engine and adding test to show this * Change the provider proto to include individual timeouts * Plumbing the CustomTimeouts from the engine through to the Provider RPC interface * Change how the CustomTimeouts are sent across RPC These errors were spotted in testing. We can now see that the timeout information is arriving in the RegisterResourceRequest ``` req=&pulumirpc.RegisterResourceRequest{ Type: "aws:s3/bucket:Bucket", Name: "my-bucket", Parent: "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev", Custom: true, Object: &structpb.Struct{}, Protect: false, Dependencies: nil, Provider: "", PropertyDependencies: {}, DeleteBeforeReplace: false, Version: "", IgnoreChanges: nil, AcceptSecrets: true, AdditionalSecretOutputs: nil, Aliases: nil, CustomTimeouts: &pulumirpc.RegisterResourceRequest_CustomTimeouts{ Create: 300, Update: 400, Delete: 500, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, }, XXX_NoUnkeyedLiteral: struct {}{}, XXX_unrecognized: nil, XXX_sizecache: 0, } ``` * Changing the design to use strings * CHANGELOG entry to include the CustomTimeouts work * Changing custom timeouts to be passed around the engine as converted value We don't want to pass around strings - the user can provide it but we want to make the engine aware of the timeout in seconds as a float64
2019-07-15 23:26:28 +02:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
}
p.Steps = []TestStep{{Op: Update}}
snap := p.Run(t, nil)
assert.Len(t, snap.Resources, 2)
assert.Equal(t, string(snap.Resources[0].URN.Name()), "default")
assert.Equal(t, string(snap.Resources[1].URN.Name()), "resA")
assert.NotNil(t, snap.Resources[1].CustomTimeouts)
assert.Equal(t, snap.Resources[1].CustomTimeouts.Create, float64(60))
assert.Equal(t, snap.Resources[1].CustomTimeouts.Update, float64(240))
assert.Equal(t, snap.Resources[1].CustomTimeouts.Delete, float64(60))
}
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
func TestProviderDiffMissingOldOutputs(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
DiffConfigF: func(urn resource.URN, olds, news resource.PropertyMap,
Pass ignoreChanges to providers. (#3005) These changes add support for passing `ignoreChanges` paths to resource providers. This is intended to accommodate providers that perform diffs between resource inputs and resource state (e.g. all Terraform-based providers, the k8s provider when using API server dry-runs). These paths are specified using the same syntax as the paths used in detailed diffs. In addition to passing these paths to providers, the existing support for `ignoreChanges` in inputs has been extended to accept paths rather than top-level keys. It is an error to specify a path that is missing one or more component in the old or new inputs. Fixes #2936, #2663.
2019-07-31 18:39:07 +02:00
ignoreChanges []string) (plugin.DiffResult, error) {
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
// Always require replacement if any diff exists.
if !olds.DeepEquals(news) {
keys := []resource.PropertyKey{}
for k := range news {
keys = append(keys, k)
}
return plugin.DiffResult{Changes: plugin.DiffSome, ReplaceKeys: keys}, nil
}
return plugin.DiffResult{Changes: plugin.DiffNone}, nil
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
Use a bag for misc. resource options in deploytest (#2977) Most of these options are typically left unset. In order to make it easier to update the lifecycle test when adding new options, collect them in a bag s.t. most callsites can go without being updated.
2019-07-25 20:18:40 +02:00
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", true)
Migrate old providers without outputs. (#2973) If we encounter a provider with old inputs but no old outputs when reading a checkpoint file, use the old inputs as the old outputs. This handles the scenario where the CLI is being upgraded from a version that did not reflect provider inputs to provider outputs, and a provider is being upgraded from a version that did not implement `DiffConfig` to a version that does. Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 22:39:21 +02:00
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Config: config.Map{
config.MustMakeKey("pkgA", "foo"): config.NewValue("bar"),
},
}
// Build a basic lifecycle.
steps := MakeBasicLifecycleSteps(t, 2)
// Run the lifecycle through its initial update and refresh.
p.Steps = steps[:2]
snap := p.Run(t, nil)
// Delete the old provider outputs (if any) from the checkpoint, then run the no-op update.
providerURN := p.NewProviderURN("pkgA", "default", "")
for _, r := range snap.Resources {
if r.URN == providerURN {
r.Outputs = nil
}
}
p.Steps = steps[2:3]
snap = p.Run(t, snap)
// Change the config, delete the old provider outputs, and run an update. We expect everything to require
// replacement.
p.Config[config.MustMakeKey("pkgA", "foo")] = config.NewValue("baz")
for _, r := range snap.Resources {
if r.URN == providerURN {
r.Outputs = nil
}
}
p.Steps = []TestStep{{
Op: Update,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
_ []Event, res result.Result) result.Result {
resURN := p.NewURN("pkgA:m:typA", "resA", "")
// Look for replace steps on the provider and the resource.
replacedProvider, replacedResource := false, false
for _, entry := range j.Entries {
if entry.Kind != JournalEntrySuccess || entry.Step.Op() != deploy.OpDeleteReplaced {
continue
}
switch urn := entry.Step.URN(); urn {
case providerURN:
replacedProvider = true
case resURN:
replacedResource = true
default:
t.Fatalf("unexpected resource %v", urn)
}
}
assert.True(t, replacedProvider)
assert.True(t, replacedResource)
return res
},
}}
p.Run(t, snap)
}
Allow resource IDs to change on reresh steps (#3087) * Allow resource IDs to change on reresh steps This is a requirement for us to be able to move forward with versions of the Terraform Azurerm provider. In v1.32.1, there was a state migration that changed the ID format of the azure table storage resource We used to have a check in place for old ID being equal to new ID. This has been changed now and we allow the change of ID to happen in the RefreshStep * Update pkg/resource/deploy/step.go Co-Authored-By: Pat Gavlin <pat@pulumi.com>
2019-08-16 20:04:03 +02:00
func TestRefreshStepWillPersistUpdatedIDs(t *testing.T) {
p := &TestPlan{}
provURN := p.NewProviderURN("pkgA", "default", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
idBefore := resource.ID("myid")
idAfter := resource.ID("mynewid")
outputs := resource.PropertyMap{"foo": resource.NewStringProperty("bar")}
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(
urn resource.URN, id resource.ID, inputs, state resource.PropertyMap,
) (plugin.ReadResult, resource.Status, error) {
return plugin.ReadResult{ID: idAfter, Outputs: outputs, Inputs: resource.PropertyMap{}}, resource.StatusOK, nil
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
_, _, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", false)
assert.NoError(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p.Options.host = host
old := &deploy.Snapshot{
Resources: []*resource.State{
{
Type: resURN.Type(),
URN: resURN,
Custom: true,
ID: idBefore,
Inputs: resource.PropertyMap{},
Outputs: outputs,
InitErrors: []string{"Resource failed to initialize"},
},
},
}
p.Steps = []TestStep{{Op: Refresh, SkipPreview: true}}
snap := p.Run(t, old)
for _, resource := range snap.Resources {
switch urn := resource.URN; urn {
case provURN:
// break
case resURN:
assert.Empty(t, resource.InitErrors)
assert.Equal(t, idAfter, resource.ID)
default:
t.Fatalf("unexpected resource %v", urn)
}
}
}
Fail read steps with missing resources. (#3123) Just what it says on the tin. Fixes #262.
2019-08-21 19:09:02 +02:00
func TestMissingRead(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(_ resource.URN, _ resource.ID, _, _ resource.PropertyMap) (plugin.ReadResult, resource.Status, error) {
return plugin.ReadResult{}, resource.StatusOK, nil
},
}, nil
}),
}
// Our program reads a resource and exits.
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
_, _, err := monitor.ReadResource("pkgA:m:typA", "resA", "resA-some-id", "", resource.PropertyMap{}, "", "")
assert.Error(t, err)
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
Steps: []TestStep{{Op: Update, ExpectFailure: true}},
}
p.Run(t, nil)
}
Allow IDs to change during import. (#3133) This is necessary for resources like `aws.ec2.RouteTableAssociation`. Part of https://github.com/pulumi/pulumi-aws/issues/708.
2019-08-24 00:00:24 +02:00
func TestImportUpdatedID(t *testing.T) {
p := &TestPlan{}
provURN := p.NewProviderURN("pkgA", "default", "")
resURN := p.NewURN("pkgA:m:typA", "resA", "")
importID := resource.ID("myID")
actualID := resource.ID("myNewID")
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
ReadF: func(
urn resource.URN, id resource.ID, inputs, state resource.PropertyMap,
) (plugin.ReadResult, resource.Status, error) {
return plugin.ReadResult{
ID: actualID,
Outputs: resource.PropertyMap{},
Inputs: resource.PropertyMap{},
}, resource.StatusOK, nil
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
_, id, _, err := monitor.RegisterResource("pkgA:m:typA", "resA", false, deploytest.ResourceOptions{
ImportID: importID,
})
assert.NoError(t, err)
assert.Equal(t, actualID, id)
return nil
})
p.Options.host = deploytest.NewPluginHost(nil, nil, program, loaders...)
p.Steps = []TestStep{{Op: Refresh, SkipPreview: true}}
snap := p.Run(t, nil)
for _, resource := range snap.Resources {
switch urn := resource.URN; urn {
case provURN:
// break
case resURN:
assert.Equal(t, actualID, resource.ID)
default:
t.Fatalf("unexpected resource %v", urn)
}
}
}
Delete specific target (#3244)
2019-09-20 04:28:14 +02:00
func TestDeleteTarget(t *testing.T) {
names := []string{"A", "B", "C", "D", "E", "F", "G", "H", "I", "J", "K", "L"}
// Try refreshing a stack with combinations of the above resources as target to destroy.
subsets := combinations.All(names)
for _, subset := range subsets {
// limit to up to 3 resources to destroy. This keeps the test running time under
// control as it only generates a few hundred combinations instead of several thousand.
if len(subset) <= 3 {
deleteSpecificTargets(t, names, subset, func(urns []resource.URN, deleted map[resource.URN]bool) {})
}
}
deleteSpecificTargets(t, names, []string{"A"}, func(urns []resource.URN, deleted map[resource.URN]bool) {
// when deleting 'A' we expect A, B, C, E, F, and K to be deleted
assert.Equal(t, map[resource.URN]bool{
pickURN(t, urns, names, "A"): true,
pickURN(t, urns, names, "B"): true,
pickURN(t, urns, names, "C"): true,
pickURN(t, urns, names, "E"): true,
pickURN(t, urns, names, "F"): true,
pickURN(t, urns, names, "K"): true,
}, deleted)
})
}
func deleteSpecificTargets(
t *testing.T, names []string, targets []string,
validate func(urns []resource.URN, deleted map[resource.URN]bool)) {
// A
// _________|_________
// B C D
// ___|___ ___|___
// E F G H I J
// |__|
// K L
p := &TestPlan{}
const resType = "pkgA:m:typA"
type propertyDependencies map[resource.PropertyKey][]resource.URN
urnA := p.NewProviderURN("pkgA", names[0], "")
urnB := p.NewURN(resType, names[1], "")
urnC := p.NewProviderURN("pkgA", names[2], "")
urnD := p.NewProviderURN("pkgA", names[3], "")
urnE := p.NewURN(resType, names[4], "")
urnF := p.NewURN(resType, names[5], "")
urnG := p.NewURN(resType, names[6], "")
urnH := p.NewURN(resType, names[7], "")
urnI := p.NewURN(resType, names[8], "")
urnJ := p.NewURN(resType, names[9], "")
urnK := p.NewURN(resType, names[10], "")
urnL := p.NewURN(resType, names[11], "")
urns := []resource.URN{
urnA, urnB, urnC, urnD, urnE, urnF,
urnG, urnH, urnI, urnJ, urnK, urnL,
}
newResource := func(urn resource.URN, id resource.ID, provider string, dependencies []resource.URN,
propertyDeps propertyDependencies, outputs resource.PropertyMap) *resource.State {
inputs := resource.PropertyMap{}
for k := range propertyDeps {
inputs[k] = resource.NewStringProperty("foo")
}
return &resource.State{
Type: urn.Type(),
URN: urn,
Custom: true,
Delete: false,
ID: id,
Inputs: inputs,
Outputs: outputs,
Dependencies: dependencies,
Provider: provider,
PropertyDependencies: propertyDeps,
}
}
old := &deploy.Snapshot{
Resources: []*resource.State{
newResource(urnA, "0", "", nil, nil, resource.PropertyMap{"A": resource.NewStringProperty("foo")}),
newResource(urnB, "1", string(urnA)+"::0", nil, nil, nil),
newResource(urnC, "2", "",
[]resource.URN{urnA},
propertyDependencies{"A": []resource.URN{urnA}},
resource.PropertyMap{"A": resource.NewStringProperty("bar")}),
newResource(urnD, "3", "",
[]resource.URN{urnA},
propertyDependencies{"B": []resource.URN{urnA}}, nil),
newResource(urnE, "4", string(urnC)+"::2", nil, nil, nil),
newResource(urnF, "5", "",
[]resource.URN{urnC},
propertyDependencies{"A": []resource.URN{urnC}}, nil),
newResource(urnG, "6", "",
[]resource.URN{urnC},
propertyDependencies{"B": []resource.URN{urnC}}, nil),
newResource(urnH, "4", string(urnD)+"::3", nil, nil, nil),
newResource(urnI, "5", "",
[]resource.URN{urnD},
propertyDependencies{"A": []resource.URN{urnD}}, nil),
newResource(urnJ, "6", "",
[]resource.URN{urnD},
propertyDependencies{"B": []resource.URN{urnD}}, nil),
newResource(urnK, "7", "",
[]resource.URN{urnF, urnG},
propertyDependencies{"A": []resource.URN{urnF, urnG}}, nil),
newResource(urnL, "8", "",
[]resource.URN{urnF, urnG},
propertyDependencies{"B": []resource.URN{urnF, urnG}}, nil),
},
}
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
DiffConfigF: func(urn resource.URN, olds, news resource.PropertyMap,
ignoreChanges []string) (plugin.DiffResult, error) {
if !olds["A"].DeepEquals(news["A"]) {
return plugin.DiffResult{
ReplaceKeys: []resource.PropertyKey{"A"},
DeleteBeforeReplace: true,
}, nil
}
return plugin.DiffResult{}, nil
},
DiffF: func(urn resource.URN, id resource.ID,
olds, news resource.PropertyMap, ignoreChanges []string) (plugin.DiffResult, error) {
if !olds["A"].DeepEquals(news["A"]) {
return plugin.DiffResult{ReplaceKeys: []resource.PropertyKey{"A"}}, nil
}
return plugin.DiffResult{}, nil
},
}, nil
}),
}
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
register := func(urn resource.URN, provider string, inputs resource.PropertyMap) resource.ID {
_, id, _, err := monitor.RegisterResource(urn.Type(), string(urn.Name()), true, deploytest.ResourceOptions{
Provider: provider,
Inputs: inputs,
})
assert.NoError(t, err)
return id
}
idA := register(urnA, "", resource.PropertyMap{"A": resource.NewStringProperty("bar")})
register(urnB, string(urnA)+"::"+string(idA), nil)
idC := register(urnC, "", nil)
idD := register(urnD, "", nil)
register(urnE, string(urnC)+"::"+string(idC), nil)
register(urnF, "", nil)
register(urnG, "", nil)
register(urnH, string(urnD)+"::"+string(idD), nil)
register(urnI, "", nil)
register(urnJ, "", nil)
register(urnK, "", nil)
register(urnL, "", nil)
return nil
})
p.Options.host = deploytest.NewPluginHost(nil, nil, program, loaders...)
destroyTargets := []resource.URN{}
for _, target := range targets {
destroyTargets = append(destroyTargets, pickURN(t, urns, names, target))
}
p.Options.DestroyTargets = destroyTargets
t.Logf("Destroying targets: %v", destroyTargets)
p.Steps = []TestStep{{
Op: Destroy,
ExpectFailure: false,
Validate: func(project workspace.Project, target deploy.Target, j *Journal,
evts []Event, res result.Result) result.Result {
assert.Nil(t, res)
assert.True(t, len(j.Entries) > 0)
deleted := make(map[resource.URN]bool)
for _, entry := range j.Entries {
assert.Equal(t, deploy.OpDelete, entry.Step.Op())
deleted[entry.Step.URN()] = true
}
for _, target := range p.Options.DestroyTargets {
assert.Contains(t, deleted, target)
}
validate(urns, deleted)
return res
},
}}
p.Run(t, old)
}
Propagate inputs to outputs during preview. (#3245) These changes restore a more-correct version of the behavior that was disabled with #3014. The original implementation of this behavior was done in the SDKs, which do not have access to the complete inputs for a resource (in particular, default values filled in by the provider during `Check` are not exposed to the SDK). This lack of information meant that the resolved output values could disagree with the typings present in a provider SDK. Exacerbating this problem was the fact that unknown values were dropped entirely, causing `undefined` values to appear in unexpected places. By doing this in the engine and allowing unknown values to be represented in a first-class manner in the SDK, we can attack both of these issues. Although this behavior is not _strictly_ consistent with respect to the resource model--in an update, a resource's output properties will come from its provider and may differ from its input properties--this behavior was present in the product for a fairly long time without significant issues. In the future, we may be able to improve the accuracy of resource outputs during a preview by allowing the provider to dry-run CRUD operations and return partially-known values where possible. These changes also introduce new APIs in the Node and Python SDKs that work with unknown values in a first-class fashion: - A new parameter to the `apply` function that indicates that the callback should be run even if the result of the apply contains unknown values - `containsUnknowns` and `isUnknown`, which return true if a value either contains nested unknown values or is exactly an unknown value - The `Unknown` type, which represents unknown values The primary use case for these APIs is to allow nested, properties with known values to be accessed via the lifted property accessor even when the containing property is not fully know. A common example of this pattern is the `metadata.name` property of a Kubernetes `Namespace` object: while other properties of the `metadata` bag may be unknown, `name` is often known. These APIs allow `ns.metadata.name` to return a known value in this case. In order to avoid exposing downlevel SDKs to unknown values--a change which could break user code by exposing it to unexpected values--a language SDK must indicate whether or not it supports first-class unknown values as part of each `RegisterResourceRequest`. These changes also allow us to avoid breaking user code with the new behavior introduced by the prior commit. Fixes #3190.
2019-09-30 20:03:58 +02:00
func TestPreviewInputPropagation(t *testing.T) {
loaders := []*deploytest.ProviderLoader{
deploytest.NewProviderLoader("pkgA", semver.MustParse("1.0.0"), func() (plugin.Provider, error) {
return &deploytest.Provider{
CreateF: func(urn resource.URN,
news resource.PropertyMap, timeout float64) (resource.ID, resource.PropertyMap, resource.Status, error) {
return "created-id", news, resource.StatusOK, nil
},
}, nil
}),
}
preview := true
program := deploytest.NewLanguageRuntime(func(_ plugin.RunInfo, monitor *deploytest.ResourceMonitor) error {
computed := interface{}(resource.Computed{Element: resource.NewStringProperty("")})
if !preview {
computed = "alpha"
}
ins := resource.NewPropertyMapFromMap(map[string]interface{}{
"foo": "bar",
"baz": map[string]interface{}{
"a": 42,
"b": computed,
},
"qux": []interface{}{
computed,
24,
},
"zed": computed,
})
_, _, state, err := monitor.RegisterResource("pkgA:m:typA", "resA", true, deploytest.ResourceOptions{
Inputs: ins,
})
assert.NoError(t, err)
assert.True(t, state.DeepEquals(ins))
return nil
})
host := deploytest.NewPluginHost(nil, nil, program, loaders...)
p := &TestPlan{
Options: UpdateOptions{host: host},
}
project := p.GetProject()
// Run a preview. The inputs should be propagated to the outputs during the create.
preview = true
_, res := TestOp(Update).Run(project, p.GetTarget(nil), p.Options, preview, p.BackendClient, nil)
assert.Nil(t, res)
// Run an update.
preview = false
snap, res := TestOp(Update).Run(project, p.GetTarget(nil), p.Options, preview, p.BackendClient, nil)
assert.Nil(t, res)
// Run another preview. The inputs should be propagated to the outputs during the update.
preview = true
_, res = TestOp(Update).Run(project, p.GetTarget(snap), p.Options, preview, p.BackendClient, nil)
assert.Nil(t, res)
}
Reference in a new issue Copy permalink