2018-03-21 20:43:21 +01:00
|
|
|
// Copyright 2016-2018, Pulumi Corporation. All rights reserved.
|
2017-11-16 16:49:07 +01:00
|
|
|
|
|
|
|
package ints
|
|
|
|
|
|
|
|
import (
|
2018-06-19 01:03:26 +02:00
|
|
|
"bytes"
|
2018-02-22 21:52:50 +01:00
|
|
|
"fmt"
|
2018-11-14 22:33:35 +01:00
|
|
|
"os"
|
2018-04-11 19:08:32 +02:00
|
|
|
"path"
|
2017-12-11 23:41:57 +01:00
|
|
|
"path/filepath"
|
2018-02-22 21:52:50 +01:00
|
|
|
"strings"
|
2017-11-16 16:49:07 +01:00
|
|
|
"testing"
|
2019-06-28 18:40:21 +02:00
|
|
|
"time"
|
2017-11-16 16:49:07 +01:00
|
|
|
|
2019-02-07 04:07:31 +01:00
|
|
|
"github.com/pulumi/pulumi/pkg/util/contract"
|
|
|
|
|
2017-11-16 16:49:07 +01:00
|
|
|
"github.com/stretchr/testify/assert"
|
|
|
|
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/apitype"
|
2017-12-08 22:14:58 +01:00
|
|
|
"github.com/pulumi/pulumi/pkg/resource"
|
2018-02-28 00:51:19 +01:00
|
|
|
"github.com/pulumi/pulumi/pkg/resource/config"
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/resource/deploy/providers"
|
2019-08-03 01:12:16 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/secrets/cloud"
|
2017-11-30 17:14:47 +01:00
|
|
|
ptesting "github.com/pulumi/pulumi/pkg/testing"
|
2017-11-16 16:49:07 +01:00
|
|
|
"github.com/pulumi/pulumi/pkg/testing/integration"
|
2018-02-14 22:56:16 +01:00
|
|
|
"github.com/pulumi/pulumi/pkg/workspace"
|
2017-11-16 16:49:07 +01:00
|
|
|
)
|
|
|
|
|
2019-01-03 23:18:19 +01:00
|
|
|
// assertPerfBenchmark implements the integration.TestStatsReporter interface, and reports test
|
|
|
|
// failures when a scenario exceeds the provided threshold.
|
2019-06-28 18:40:21 +02:00
|
|
|
type assertPerfBenchmark struct {
|
|
|
|
T *testing.T
|
|
|
|
MaxPreviewDuration time.Duration
|
|
|
|
MaxUpdateDuration time.Duration
|
|
|
|
}
|
2019-01-03 23:18:19 +01:00
|
|
|
|
2019-06-28 18:40:21 +02:00
|
|
|
func (t assertPerfBenchmark) ReportCommand(stats integration.TestCommandStats) {
|
|
|
|
var maxDuration *time.Duration
|
|
|
|
if strings.HasPrefix(stats.StepName, "pulumi-preview") {
|
|
|
|
maxDuration = &t.MaxPreviewDuration
|
|
|
|
}
|
|
|
|
if strings.HasPrefix(stats.StepName, "pulumi-update") {
|
|
|
|
maxDuration = &t.MaxUpdateDuration
|
|
|
|
}
|
2019-01-03 23:18:19 +01:00
|
|
|
|
2019-06-28 18:40:21 +02:00
|
|
|
if maxDuration != nil && *maxDuration != 0 {
|
|
|
|
if stats.ElapsedSeconds < maxDuration.Seconds() {
|
|
|
|
t.T.Logf(
|
|
|
|
"Test step %q was under threshold. %.2fs (max %.2fs)",
|
|
|
|
stats.StepName, stats.ElapsedSeconds, maxDuration.Seconds())
|
|
|
|
} else {
|
|
|
|
t.T.Errorf(
|
|
|
|
"Test step %q took longer than expected. %.2fs vs. max %.2fs",
|
|
|
|
stats.StepName, stats.ElapsedSeconds, maxDuration.Seconds())
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
2019-01-03 23:18:19 +01:00
|
|
|
|
Get the empty Python program working
This change gets enough of the Python SDK up and running that the
empty Python program will work. Mostly just scaffolding, but the
basic structure is now in place. The primary remaining work is to
wire up resource creation to the gRPC interfaces.
In summary:
* The basic structure is as follows:
- Everything goes into sdk/python/.
- sdk/python/cmd/pulumi-langhost-python is a Go language host
that simply knows how to spawn Python processes to run out
entrypoint in response to requests by the engine.
- sdk/python/cmd/pulumi-langhost-python-exec is a little Python
shim that is invoked by the language host to run Python programs,
and is responsible for setting up the minimal goo before we can
do so (RPC connections and the like).
- sdk/python/lib/ contains a Python Pip package suitable for PyPi.
- In there, we have two packages: the root pulumi package that
contains all of the basic Pulumi programming model abstractions,
and pulumi.runtime, which contains the implementation of
resource registration, RPC interfacing with the engine, and so on.
* Add logic in our test framework to conditionalize on the language
type and react accordingly. This will allow us to skip Yarn for
Python projects and eventually run Pip if there's a requirements.txt.
* Created the basic project structure, including all of the usual
Make targets for installing into the proper places.
* Building also runs Pylint and we are clean.
There are a few other minor things in here:
* Add an "empty" test for both Node.js and Python. These pass.
* Fix an existing bug in plugin shutdown logic. At some point, we
started waiting for stderr/stdout to flush before shutting down
the plugin; but if certain failures happen "early" during the
plugin launch process, these channels will never get initialized
and so waiting for them deadlocks.
* Recently we seem to have added logic to delete test temp
directories if a failure happened during initialization of said
temp directories. This is unfortunate, because you often need to
look at the temp directory to see what failed. We already clean
them up elsewhere after the full test completes successfully, so
I don't think we need to be doing this, and I've removed it.
Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
|
|
|
// TestEmptyNodeJS simply tests that we can run an empty NodeJS project.
|
|
|
|
func TestEmptyNodeJS(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("empty", "nodejs"),
|
2018-02-24 19:39:18 +01:00
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
Get the empty Python program working
This change gets enough of the Python SDK up and running that the
empty Python program will work. Mostly just scaffolding, but the
basic structure is now in place. The primary remaining work is to
wire up resource creation to the gRPC interfaces.
In summary:
* The basic structure is as follows:
- Everything goes into sdk/python/.
- sdk/python/cmd/pulumi-langhost-python is a Go language host
that simply knows how to spawn Python processes to run out
entrypoint in response to requests by the engine.
- sdk/python/cmd/pulumi-langhost-python-exec is a little Python
shim that is invoked by the language host to run Python programs,
and is responsible for setting up the minimal goo before we can
do so (RPC connections and the like).
- sdk/python/lib/ contains a Python Pip package suitable for PyPi.
- In there, we have two packages: the root pulumi package that
contains all of the basic Pulumi programming model abstractions,
and pulumi.runtime, which contains the implementation of
resource registration, RPC interfacing with the engine, and so on.
* Add logic in our test framework to conditionalize on the language
type and react accordingly. This will allow us to skip Yarn for
Python projects and eventually run Pip if there's a requirements.txt.
* Created the basic project structure, including all of the usual
Make targets for installing into the proper places.
* Building also runs Pylint and we are clean.
There are a few other minor things in here:
* Add an "empty" test for both Node.js and Python. These pass.
* Fix an existing bug in plugin shutdown logic. At some point, we
started waiting for stderr/stdout to flush before shutting down
the plugin; but if certain failures happen "early" during the
plugin launch process, these channels will never get initialized
and so waiting for them deadlocks.
* Recently we seem to have added logic to delete test temp
directories if a failure happened during initialization of said
temp directories. This is unfortunate, because you often need to
look at the temp directory to see what failed. We already clean
them up elsewhere after the full test completes successfully, so
I don't think we need to be doing this, and I've removed it.
Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
|
|
|
Quick: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
|
|
|
// TestEmptyPython simply tests that we can run an empty Python project.
|
|
|
|
func TestEmptyPython(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
2018-11-05 22:52:37 +01:00
|
|
|
Dir: filepath.Join("empty", "python"),
|
|
|
|
Dependencies: []string{
|
|
|
|
path.Join("..", "..", "sdk", "python", "env", "src"),
|
|
|
|
},
|
Get the empty Python program working
This change gets enough of the Python SDK up and running that the
empty Python program will work. Mostly just scaffolding, but the
basic structure is now in place. The primary remaining work is to
wire up resource creation to the gRPC interfaces.
In summary:
* The basic structure is as follows:
- Everything goes into sdk/python/.
- sdk/python/cmd/pulumi-langhost-python is a Go language host
that simply knows how to spawn Python processes to run out
entrypoint in response to requests by the engine.
- sdk/python/cmd/pulumi-langhost-python-exec is a little Python
shim that is invoked by the language host to run Python programs,
and is responsible for setting up the minimal goo before we can
do so (RPC connections and the like).
- sdk/python/lib/ contains a Python Pip package suitable for PyPi.
- In there, we have two packages: the root pulumi package that
contains all of the basic Pulumi programming model abstractions,
and pulumi.runtime, which contains the implementation of
resource registration, RPC interfacing with the engine, and so on.
* Add logic in our test framework to conditionalize on the language
type and react accordingly. This will allow us to skip Yarn for
Python projects and eventually run Pip if there's a requirements.txt.
* Created the basic project structure, including all of the usual
Make targets for installing into the proper places.
* Building also runs Pylint and we are clean.
There are a few other minor things in here:
* Add an "empty" test for both Node.js and Python. These pass.
* Fix an existing bug in plugin shutdown logic. At some point, we
started waiting for stderr/stdout to flush before shutting down
the plugin; but if certain failures happen "early" during the
plugin launch process, these channels will never get initialized
and so waiting for them deadlocks.
* Recently we seem to have added logic to delete test temp
directories if a failure happened during initialization of said
temp directories. This is unfortunate, because you often need to
look at the temp directory to see what failed. We already clean
them up elsewhere after the full test completes successfully, so
I don't think we need to be doing this, and I've removed it.
Still many loose ends (config, resources, etc), but it's a start!
2018-01-13 19:29:34 +01:00
|
|
|
Quick: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2019-11-14 18:25:55 +01:00
|
|
|
// TestEmptyGo simply tests that we can build and run an empty Go project.
|
2018-06-10 18:17:19 +02:00
|
|
|
func TestEmptyGo(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
2019-11-14 18:25:55 +01:00
|
|
|
Dir: filepath.Join("empty", "go"),
|
|
|
|
Quick: true,
|
|
|
|
RunBuild: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
|
|
|
// TestEmptyGoRun exercises the 'go run' invocation path that doesn't require an explicit build step.
|
|
|
|
func TestEmptyGoRun(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("empty", "gorun"),
|
|
|
|
Quick: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
|
|
|
// TestEmptyGoRunMain exercises the 'go run' invocation path with a 'main' entrypoint specified in Pulumi.yml
|
|
|
|
func TestEmptyGoRunMain(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("empty", "gorun_main"),
|
2018-06-10 18:17:19 +02:00
|
|
|
Quick: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2019-10-26 01:59:50 +02:00
|
|
|
// TestEmptyDotNet simply tests that we can run an empty .NET project.
|
|
|
|
func TestEmptyDotNet(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("empty", "dotnet"),
|
|
|
|
Quick: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2019-01-03 23:18:19 +01:00
|
|
|
// Tests emitting many engine events doesn't result in a performance problem.
|
2019-06-28 18:40:21 +02:00
|
|
|
func TestEngineEventPerf(t *testing.T) {
|
|
|
|
// Prior to pulumi/pulumi#2303, a preview or update would take ~40s.
|
|
|
|
// Since then, it should now be down to ~4s, with additional padding,
|
|
|
|
// since some Travis machines (especially the macOS ones) seem quite slow
|
|
|
|
// to begin with.
|
|
|
|
benchmarkEnforcer := &assertPerfBenchmark{
|
|
|
|
T: t,
|
|
|
|
MaxPreviewDuration: 8 * time.Second,
|
|
|
|
MaxUpdateDuration: 8 * time.Second,
|
|
|
|
}
|
2019-01-03 23:18:19 +01:00
|
|
|
|
2019-06-28 18:40:21 +02:00
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "ee_perf",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
ReportStats: benchmarkEnforcer,
|
|
|
|
// Don't run in parallel since it is sensitive to system resources.
|
|
|
|
NoParallel: true,
|
|
|
|
})
|
|
|
|
}
|
2019-01-03 23:18:19 +01:00
|
|
|
|
2019-09-07 02:07:54 +02:00
|
|
|
// TestEngineEvents ensures that the test framework properly records and reads engine events.
|
|
|
|
func TestEngineEvents(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "single_resource",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
// Ensure that we have a non-empty list of events.
|
|
|
|
assert.NotEmpty(t, stackInfo.Events)
|
|
|
|
|
|
|
|
// Ensure that we have two "ResourcePre" events: one for the stack and one for our resource.
|
|
|
|
preEventResourceTypes := []string{}
|
|
|
|
for _, e := range stackInfo.Events {
|
|
|
|
if e.ResourcePreEvent != nil {
|
|
|
|
preEventResourceTypes = append(preEventResourceTypes, e.ResourcePreEvent.Metadata.Type)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
assert.Equal(t, 2, len(preEventResourceTypes))
|
|
|
|
assert.Contains(t, preEventResourceTypes, "pulumi:pulumi:Stack")
|
|
|
|
assert.Contains(t, preEventResourceTypes, "pulumi-nodejs:dynamic:Resource")
|
|
|
|
},
|
|
|
|
})
|
|
|
|
|
|
|
|
}
|
|
|
|
|
2017-11-16 17:15:56 +01:00
|
|
|
// TestProjectMain tests out the ability to override the main entrypoint.
|
2017-11-16 16:49:07 +01:00
|
|
|
func TestProjectMain(t *testing.T) {
|
2018-11-05 22:36:35 +01:00
|
|
|
test := integration.ProgramTestOptions{
|
2017-11-16 16:49:07 +01:00
|
|
|
Dir: "project_main",
|
2018-02-12 22:13:13 +01:00
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
2017-12-14 01:09:14 +01:00
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
2017-11-16 16:49:07 +01:00
|
|
|
// Simple runtime validation that just ensures the checkpoint was written and read.
|
2018-04-03 06:34:54 +02:00
|
|
|
assert.NotNil(t, stackInfo.Deployment)
|
2017-11-16 16:49:07 +01:00
|
|
|
},
|
|
|
|
}
|
2017-12-09 01:59:28 +01:00
|
|
|
integration.ProgramTest(t, &test)
|
2017-11-30 17:14:47 +01:00
|
|
|
|
|
|
|
t.Run("Error_AbsolutePath", func(t *testing.T) {
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
e.ImportDirectory("project_main_abs")
|
Remove the need to `pulumi init` for the local backend
This change removes the need to `pulumi init` when targeting the local
backend. A fair amount of the change lays the foundation that the next
set of changes to stop having `pulumi init` be used for cloud stacks
as well.
Previously, `pulumi init` logically did two things:
1. It created the bookkeeping directory for local stacks, this was
stored in `<repository-root>/.pulumi`, where `<repository-root>` was
the path to what we belived the "root" of your project was. In the
case of git repositories, this was the directory that contained your
`.git` folder.
2. It recorded repository information in
`<repository-root>/.pulumi/repository.json`. This was used by the
cloud backend when computing what project to interact with on
Pulumi.com
The new identity model will remove the need for (2), since we only
need an owner and stack name to fully qualify a stack on
pulumi.com, so it's easy enough to stop creating a folder just for
that.
However, for the local backend, we need to continue to retain some
information about stacks (e.g. checkpoints, history, etc). In
addition, we need to store our workspace settings (which today just
contains the selected stack) somehere.
For state stored by the local backend, we change the URL scheme from
`local://` to `local://<optional-root-path>`. When
`<optional-root-path>` is unset, it defaults to `$HOME`. We create our
`.pulumi` folder in that directory. This is important because stack
names now must be unique within the backend, but we have some tests
using local stacks which use fixed stack names, so each integration
test really wants its own "view" of the world.
For the workspace settings, we introduce a new `workspaces` directory
in `~/.pulumi`. In this folder we write the workspace settings file
for each project. The file name is the name of the project, combined
with the SHA1 of the path of the project file on disk, to ensure that
multiple pulumi programs with the same project name have different
workspace settings.
This does mean that moving a project's location on disk will cause the
CLI to "forget" what the selected stack was, which is unfortunate, but
not the end of the world. If this ends up being a big pain point, we
can certianly try to play games in the future (for example, if we saw
a .git folder in a parent folder, we could store data in there).
With respect to compatibility, we don't attempt to migrate older files
to their newer locations. For long lived stacks managed using the
local backend, we can provide information on where to move things
to. For all stacks (regardless of backend) we'll require the user to
`pulumi stack select` their stack again, but that seems like the
correct trade-off vs writing complicated upgrade code.
2018-04-17 01:15:10 +02:00
|
|
|
e.RunCommand("pulumi", "login", "--cloud-url", e.LocalURL())
|
2018-04-05 00:31:01 +02:00
|
|
|
e.RunCommand("pulumi", "stack", "init", "main-abs")
|
2019-04-26 02:32:31 +02:00
|
|
|
stdout, stderr := e.RunCommandExpectError("pulumi", "up", "--non-interactive", "--skip-preview")
|
|
|
|
assert.Equal(t, "Updating (main-abs):\n", stdout)
|
2017-11-30 17:14:47 +01:00
|
|
|
assert.Contains(t, stderr, "project 'main' must be a relative path")
|
2018-04-17 01:03:40 +02:00
|
|
|
e.RunCommand("pulumi", "stack", "rm", "--yes")
|
2017-11-30 17:14:47 +01:00
|
|
|
})
|
|
|
|
|
|
|
|
t.Run("Error_ParentFolder", func(t *testing.T) {
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
e.ImportDirectory("project_main_parent")
|
Remove the need to `pulumi init` for the local backend
This change removes the need to `pulumi init` when targeting the local
backend. A fair amount of the change lays the foundation that the next
set of changes to stop having `pulumi init` be used for cloud stacks
as well.
Previously, `pulumi init` logically did two things:
1. It created the bookkeeping directory for local stacks, this was
stored in `<repository-root>/.pulumi`, where `<repository-root>` was
the path to what we belived the "root" of your project was. In the
case of git repositories, this was the directory that contained your
`.git` folder.
2. It recorded repository information in
`<repository-root>/.pulumi/repository.json`. This was used by the
cloud backend when computing what project to interact with on
Pulumi.com
The new identity model will remove the need for (2), since we only
need an owner and stack name to fully qualify a stack on
pulumi.com, so it's easy enough to stop creating a folder just for
that.
However, for the local backend, we need to continue to retain some
information about stacks (e.g. checkpoints, history, etc). In
addition, we need to store our workspace settings (which today just
contains the selected stack) somehere.
For state stored by the local backend, we change the URL scheme from
`local://` to `local://<optional-root-path>`. When
`<optional-root-path>` is unset, it defaults to `$HOME`. We create our
`.pulumi` folder in that directory. This is important because stack
names now must be unique within the backend, but we have some tests
using local stacks which use fixed stack names, so each integration
test really wants its own "view" of the world.
For the workspace settings, we introduce a new `workspaces` directory
in `~/.pulumi`. In this folder we write the workspace settings file
for each project. The file name is the name of the project, combined
with the SHA1 of the path of the project file on disk, to ensure that
multiple pulumi programs with the same project name have different
workspace settings.
This does mean that moving a project's location on disk will cause the
CLI to "forget" what the selected stack was, which is unfortunate, but
not the end of the world. If this ends up being a big pain point, we
can certianly try to play games in the future (for example, if we saw
a .git folder in a parent folder, we could store data in there).
With respect to compatibility, we don't attempt to migrate older files
to their newer locations. For long lived stacks managed using the
local backend, we can provide information on where to move things
to. For all stacks (regardless of backend) we'll require the user to
`pulumi stack select` their stack again, but that seems like the
correct trade-off vs writing complicated upgrade code.
2018-04-17 01:15:10 +02:00
|
|
|
e.RunCommand("pulumi", "login", "--cloud-url", e.LocalURL())
|
2018-04-05 00:31:01 +02:00
|
|
|
e.RunCommand("pulumi", "stack", "init", "main-parent")
|
2019-04-26 02:32:31 +02:00
|
|
|
stdout, stderr := e.RunCommandExpectError("pulumi", "up", "--non-interactive", "--skip-preview")
|
|
|
|
assert.Equal(t, "Updating (main-parent):\n", stdout)
|
2017-11-30 17:14:47 +01:00
|
|
|
assert.Contains(t, stderr, "project 'main' must be a subfolder")
|
2018-04-17 01:03:40 +02:00
|
|
|
e.RunCommand("pulumi", "stack", "rm", "--yes")
|
2017-11-30 17:14:47 +01:00
|
|
|
})
|
2017-11-16 16:49:07 +01:00
|
|
|
}
|
2017-11-16 17:15:56 +01:00
|
|
|
|
|
|
|
// TestStackProjectName ensures we can read the Pulumi stack and project name from within the program.
|
|
|
|
func TestStackProjectName(t *testing.T) {
|
2017-12-09 01:59:28 +01:00
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
2017-11-16 17:15:56 +01:00
|
|
|
Dir: "stack_project_name",
|
2018-02-12 22:13:13 +01:00
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
2017-11-16 17:15:56 +01:00
|
|
|
Quick: true,
|
2017-12-05 22:01:54 +01:00
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2018-04-11 19:08:32 +02:00
|
|
|
// TestStackTagValidation verifies various error scenarios related to stack names and tags.
|
|
|
|
func TestStackTagValidation(t *testing.T) {
|
|
|
|
t.Run("Error_StackName", func(t *testing.T) {
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
e.RunCommand("git", "init")
|
|
|
|
|
|
|
|
e.ImportDirectory("stack_project_name")
|
Remove the need to `pulumi init` for the local backend
This change removes the need to `pulumi init` when targeting the local
backend. A fair amount of the change lays the foundation that the next
set of changes to stop having `pulumi init` be used for cloud stacks
as well.
Previously, `pulumi init` logically did two things:
1. It created the bookkeeping directory for local stacks, this was
stored in `<repository-root>/.pulumi`, where `<repository-root>` was
the path to what we belived the "root" of your project was. In the
case of git repositories, this was the directory that contained your
`.git` folder.
2. It recorded repository information in
`<repository-root>/.pulumi/repository.json`. This was used by the
cloud backend when computing what project to interact with on
Pulumi.com
The new identity model will remove the need for (2), since we only
need an owner and stack name to fully qualify a stack on
pulumi.com, so it's easy enough to stop creating a folder just for
that.
However, for the local backend, we need to continue to retain some
information about stacks (e.g. checkpoints, history, etc). In
addition, we need to store our workspace settings (which today just
contains the selected stack) somehere.
For state stored by the local backend, we change the URL scheme from
`local://` to `local://<optional-root-path>`. When
`<optional-root-path>` is unset, it defaults to `$HOME`. We create our
`.pulumi` folder in that directory. This is important because stack
names now must be unique within the backend, but we have some tests
using local stacks which use fixed stack names, so each integration
test really wants its own "view" of the world.
For the workspace settings, we introduce a new `workspaces` directory
in `~/.pulumi`. In this folder we write the workspace settings file
for each project. The file name is the name of the project, combined
with the SHA1 of the path of the project file on disk, to ensure that
multiple pulumi programs with the same project name have different
workspace settings.
This does mean that moving a project's location on disk will cause the
CLI to "forget" what the selected stack was, which is unfortunate, but
not the end of the world. If this ends up being a big pain point, we
can certianly try to play games in the future (for example, if we saw
a .git folder in a parent folder, we could store data in there).
With respect to compatibility, we don't attempt to migrate older files
to their newer locations. For long lived stacks managed using the
local backend, we can provide information on where to move things
to. For all stacks (regardless of backend) we'll require the user to
`pulumi stack select` their stack again, but that seems like the
correct trade-off vs writing complicated upgrade code.
2018-04-17 01:15:10 +02:00
|
|
|
e.RunCommand("pulumi", "login", "--cloud-url", e.LocalURL())
|
2018-04-11 19:08:32 +02:00
|
|
|
|
|
|
|
stdout, stderr := e.RunCommandExpectError("pulumi", "stack", "init", "invalid name (spaces, parens, etc.)")
|
|
|
|
assert.Equal(t, "", stdout)
|
|
|
|
assert.Contains(t, stderr, "error: could not create stack:")
|
|
|
|
assert.Contains(t, stderr, "validating stack properties:")
|
2018-05-29 22:52:11 +02:00
|
|
|
assert.Contains(t, stderr, "stack name may only contain alphanumeric, hyphens, underscores, or periods")
|
2018-04-11 19:08:32 +02:00
|
|
|
})
|
|
|
|
|
|
|
|
t.Run("Error_DescriptionLength", func(t *testing.T) {
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
e.RunCommand("git", "init")
|
|
|
|
|
|
|
|
e.ImportDirectory("stack_project_name")
|
Remove the need to `pulumi init` for the local backend
This change removes the need to `pulumi init` when targeting the local
backend. A fair amount of the change lays the foundation that the next
set of changes to stop having `pulumi init` be used for cloud stacks
as well.
Previously, `pulumi init` logically did two things:
1. It created the bookkeeping directory for local stacks, this was
stored in `<repository-root>/.pulumi`, where `<repository-root>` was
the path to what we belived the "root" of your project was. In the
case of git repositories, this was the directory that contained your
`.git` folder.
2. It recorded repository information in
`<repository-root>/.pulumi/repository.json`. This was used by the
cloud backend when computing what project to interact with on
Pulumi.com
The new identity model will remove the need for (2), since we only
need an owner and stack name to fully qualify a stack on
pulumi.com, so it's easy enough to stop creating a folder just for
that.
However, for the local backend, we need to continue to retain some
information about stacks (e.g. checkpoints, history, etc). In
addition, we need to store our workspace settings (which today just
contains the selected stack) somehere.
For state stored by the local backend, we change the URL scheme from
`local://` to `local://<optional-root-path>`. When
`<optional-root-path>` is unset, it defaults to `$HOME`. We create our
`.pulumi` folder in that directory. This is important because stack
names now must be unique within the backend, but we have some tests
using local stacks which use fixed stack names, so each integration
test really wants its own "view" of the world.
For the workspace settings, we introduce a new `workspaces` directory
in `~/.pulumi`. In this folder we write the workspace settings file
for each project. The file name is the name of the project, combined
with the SHA1 of the path of the project file on disk, to ensure that
multiple pulumi programs with the same project name have different
workspace settings.
This does mean that moving a project's location on disk will cause the
CLI to "forget" what the selected stack was, which is unfortunate, but
not the end of the world. If this ends up being a big pain point, we
can certianly try to play games in the future (for example, if we saw
a .git folder in a parent folder, we could store data in there).
With respect to compatibility, we don't attempt to migrate older files
to their newer locations. For long lived stacks managed using the
local backend, we can provide information on where to move things
to. For all stacks (regardless of backend) we'll require the user to
`pulumi stack select` their stack again, but that seems like the
correct trade-off vs writing complicated upgrade code.
2018-04-17 01:15:10 +02:00
|
|
|
e.RunCommand("pulumi", "login", "--cloud-url", e.LocalURL())
|
2018-04-11 19:08:32 +02:00
|
|
|
|
|
|
|
prefix := "lorem ipsum dolor sit amet" // 26
|
|
|
|
prefix = prefix + prefix + prefix + prefix // 104
|
|
|
|
prefix = prefix + prefix + prefix + prefix // 416 + the current Pulumi.yaml's description
|
|
|
|
|
|
|
|
// Change the contents of the Description property of Pulumi.yaml.
|
|
|
|
yamlPath := path.Join(e.CWD, "Pulumi.yaml")
|
|
|
|
err := integration.ReplaceInFile("description: ", "description: "+prefix, yamlPath)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
|
|
|
|
stdout, stderr := e.RunCommandExpectError("pulumi", "stack", "init", "valid-name")
|
|
|
|
assert.Equal(t, "", stdout)
|
|
|
|
assert.Contains(t, stderr, "error: could not create stack:")
|
|
|
|
assert.Contains(t, stderr, "validating stack properties:")
|
2018-05-29 22:52:11 +02:00
|
|
|
assert.Contains(t, stderr, "stack tag \"pulumi:description\" value is too long (max length 256 characters)")
|
2018-04-11 19:08:32 +02:00
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2019-02-07 04:07:31 +01:00
|
|
|
func TestRemoveWithResourcesBlocked(t *testing.T) {
|
|
|
|
if os.Getenv("PULUMI_ACCESS_TOKEN") == "" {
|
|
|
|
t.Skipf("Skipping: PULUMI_ACCESS_TOKEN is not set")
|
|
|
|
}
|
|
|
|
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
|
|
|
|
stackName, err := resource.NewUniqueHex("rm-test-", 8, -1)
|
2019-06-18 23:39:42 +02:00
|
|
|
contract.AssertNoErrorf(err, "resource.NewUniqueHex should not fail with no maximum length is set")
|
2019-02-07 04:07:31 +01:00
|
|
|
|
2019-02-12 23:49:43 +01:00
|
|
|
e.ImportDirectory("single_resource")
|
2019-02-07 04:07:31 +01:00
|
|
|
e.RunCommand("pulumi", "stack", "init", stackName)
|
|
|
|
e.RunCommand("yarn", "link", "@pulumi/pulumi")
|
|
|
|
e.RunCommand("pulumi", "up", "--non-interactive", "--skip-preview")
|
|
|
|
_, stderr := e.RunCommandExpectError("pulumi", "stack", "rm", "--yes")
|
|
|
|
assert.Contains(t, stderr, "--force")
|
|
|
|
e.RunCommand("pulumi", "destroy", "--skip-preview", "--non-interactive", "--yes")
|
|
|
|
e.RunCommand("pulumi", "stack", "rm", "--yes")
|
|
|
|
}
|
|
|
|
|
2017-12-05 22:01:54 +01:00
|
|
|
// TestStackOutputs ensures we can export variables from a stack and have them get recorded as outputs.
|
2018-11-08 18:44:34 +01:00
|
|
|
func TestStackOutputsNodeJS(t *testing.T) {
|
2017-12-09 01:59:28 +01:00
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
2018-11-08 18:44:34 +01:00
|
|
|
Dir: filepath.Join("stack_outputs", "nodejs"),
|
2018-02-12 22:13:13 +01:00
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
2017-12-05 22:01:54 +01:00
|
|
|
Quick: true,
|
2017-12-14 01:09:14 +01:00
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
2017-12-05 22:01:54 +01:00
|
|
|
// Ensure the checkpoint contains a single resource, the Stack, with two outputs.
|
2018-04-03 06:34:54 +02:00
|
|
|
fmt.Printf("Deployment: %v", stackInfo.Deployment)
|
|
|
|
assert.NotNil(t, stackInfo.Deployment)
|
|
|
|
if assert.Equal(t, 1, len(stackInfo.Deployment.Resources)) {
|
|
|
|
stackRes := stackInfo.Deployment.Resources[0]
|
2017-12-05 22:01:54 +01:00
|
|
|
assert.NotNil(t, stackRes)
|
2017-12-08 22:14:58 +01:00
|
|
|
assert.Equal(t, resource.RootStackType, stackRes.URN.Type())
|
2017-12-05 22:01:54 +01:00
|
|
|
assert.Equal(t, 0, len(stackRes.Inputs))
|
|
|
|
assert.Equal(t, 2, len(stackRes.Outputs))
|
|
|
|
assert.Equal(t, "ABC", stackRes.Outputs["xyz"])
|
|
|
|
assert.Equal(t, float64(42), stackRes.Outputs["foo"])
|
|
|
|
}
|
|
|
|
},
|
|
|
|
})
|
2017-11-16 17:15:56 +01:00
|
|
|
}
|
2017-12-06 04:14:28 +01:00
|
|
|
|
2018-11-08 18:44:34 +01:00
|
|
|
func TestStackOutputsPython(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("stack_outputs", "python"),
|
|
|
|
Dependencies: []string{
|
|
|
|
filepath.Join("..", "..", "sdk", "python", "env", "src"),
|
|
|
|
},
|
|
|
|
Quick: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
// Ensure the checkpoint contains a single resource, the Stack, with two outputs.
|
|
|
|
fmt.Printf("Deployment: %v", stackInfo.Deployment)
|
|
|
|
assert.NotNil(t, stackInfo.Deployment)
|
|
|
|
if assert.Equal(t, 1, len(stackInfo.Deployment.Resources)) {
|
|
|
|
stackRes := stackInfo.Deployment.Resources[0]
|
|
|
|
assert.NotNil(t, stackRes)
|
|
|
|
assert.Equal(t, resource.RootStackType, stackRes.URN.Type())
|
|
|
|
assert.Equal(t, 0, len(stackRes.Inputs))
|
|
|
|
assert.Equal(t, 2, len(stackRes.Outputs))
|
|
|
|
assert.Equal(t, "ABC", stackRes.Outputs["xyz"])
|
|
|
|
assert.Equal(t, float64(42), stackRes.Outputs["foo"])
|
|
|
|
}
|
|
|
|
},
|
|
|
|
})
|
2019-10-26 01:59:50 +02:00
|
|
|
}
|
2018-11-08 18:44:34 +01:00
|
|
|
|
2019-10-26 01:59:50 +02:00
|
|
|
func TestStackOutputsDotNet(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("stack_outputs", "dotnet"),
|
|
|
|
Quick: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
// Ensure the checkpoint contains a single resource, the Stack, with two outputs.
|
|
|
|
fmt.Printf("Deployment: %v", stackInfo.Deployment)
|
|
|
|
assert.NotNil(t, stackInfo.Deployment)
|
|
|
|
if assert.Equal(t, 1, len(stackInfo.Deployment.Resources)) {
|
|
|
|
stackRes := stackInfo.Deployment.Resources[0]
|
|
|
|
assert.NotNil(t, stackRes)
|
|
|
|
assert.Equal(t, resource.RootStackType, stackRes.URN.Type())
|
|
|
|
assert.Equal(t, 0, len(stackRes.Inputs))
|
|
|
|
assert.Equal(t, 2, len(stackRes.Outputs))
|
|
|
|
assert.Equal(t, "ABC", stackRes.Outputs["xyz"])
|
|
|
|
assert.Equal(t, float64(42), stackRes.Outputs["foo"])
|
|
|
|
}
|
|
|
|
},
|
|
|
|
})
|
2018-11-08 18:44:34 +01:00
|
|
|
}
|
|
|
|
|
2018-09-29 18:57:58 +02:00
|
|
|
// TestStackOutputsJSON ensures the CLI properly formats stack outputs as JSON when requested.
|
|
|
|
func TestStackOutputsJSON(t *testing.T) {
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
2018-11-08 18:44:34 +01:00
|
|
|
e.ImportDirectory(filepath.Join("stack_outputs", "nodejs"))
|
2018-09-29 18:57:58 +02:00
|
|
|
e.RunCommand("yarn", "link", "@pulumi/pulumi")
|
|
|
|
e.RunCommand("pulumi", "login", "--cloud-url", e.LocalURL())
|
|
|
|
e.RunCommand("pulumi", "stack", "init", "stack-outs")
|
2018-10-26 23:50:47 +02:00
|
|
|
e.RunCommand("pulumi", "up", "--non-interactive", "--skip-preview")
|
2018-11-17 03:46:09 +01:00
|
|
|
stdout, _ := e.RunCommand("pulumi", "stack", "output", "--json")
|
2018-09-29 18:57:58 +02:00
|
|
|
assert.Equal(t, `{
|
2018-10-25 22:56:30 +02:00
|
|
|
"foo": 42,
|
|
|
|
"xyz": "ABC"
|
2018-09-29 18:57:58 +02:00
|
|
|
}
|
|
|
|
`, stdout)
|
|
|
|
}
|
|
|
|
|
2018-06-19 01:03:26 +02:00
|
|
|
// TestStackOutputsDisplayed ensures that outputs are printed at the end of an update
|
|
|
|
func TestStackOutputsDisplayed(t *testing.T) {
|
|
|
|
stdout := &bytes.Buffer{}
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
2018-11-08 18:44:34 +01:00
|
|
|
Dir: filepath.Join("stack_outputs", "nodejs"),
|
2018-06-19 01:03:26 +02:00
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: false,
|
|
|
|
Verbose: true,
|
|
|
|
Stdout: stdout,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
output := stdout.String()
|
|
|
|
|
2018-09-25 00:22:50 +02:00
|
|
|
// ensure we get the outputs info both for the normal update, and for the no-change update.
|
2018-10-31 05:57:38 +01:00
|
|
|
assert.Contains(t, output, "Outputs:\n foo: 42\n xyz: \"ABC\"\n\nResources:\n + 1 created")
|
|
|
|
assert.Contains(t, output, "Outputs:\n foo: 42\n xyz: \"ABC\"\n\nResources:\n 1 unchanged")
|
2018-06-19 01:03:26 +02:00
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2018-10-06 23:13:02 +02:00
|
|
|
// TestStackOutputsSuppressed ensures that outputs whose values are intentionally suppresses don't show.
|
|
|
|
func TestStackOutputsSuppressed(t *testing.T) {
|
|
|
|
stdout := &bytes.Buffer{}
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
2018-11-08 18:44:34 +01:00
|
|
|
Dir: filepath.Join("stack_outputs", "nodejs"),
|
2018-10-06 23:13:02 +02:00
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: false,
|
|
|
|
Verbose: true,
|
|
|
|
Stdout: stdout,
|
|
|
|
UpdateCommandlineFlags: []string{"--suppress-outputs"},
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
output := stdout.String()
|
|
|
|
assert.NotContains(t, output, "Outputs:\n foo: 42\n xyz: \"ABC\"\n")
|
|
|
|
assert.NotContains(t, output, "Outputs:\n foo: 42\n xyz: \"ABC\"\n")
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2017-12-06 04:14:28 +01:00
|
|
|
// TestStackParenting tests out that stacks and components are parented correctly.
|
|
|
|
func TestStackParenting(t *testing.T) {
|
2017-12-09 01:59:28 +01:00
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
2017-12-06 04:14:28 +01:00
|
|
|
Dir: "stack_parenting",
|
2018-02-12 22:13:13 +01:00
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
2017-12-06 04:14:28 +01:00
|
|
|
Quick: true,
|
2017-12-14 01:09:14 +01:00
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
2017-12-06 04:14:28 +01:00
|
|
|
// Ensure the checkpoint contains resources parented correctly. This should look like this:
|
|
|
|
//
|
|
|
|
// A F
|
|
|
|
// / \ \
|
|
|
|
// B C G
|
|
|
|
// / \
|
|
|
|
// D E
|
|
|
|
//
|
|
|
|
// with the caveat, of course, that A and F will share a common parent, the implicit stack.
|
|
|
|
|
2018-04-03 06:34:54 +02:00
|
|
|
assert.NotNil(t, stackInfo.Deployment)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
if assert.Equal(t, 9, len(stackInfo.Deployment.Resources)) {
|
2018-04-03 06:34:54 +02:00
|
|
|
stackRes := stackInfo.Deployment.Resources[0]
|
2017-12-06 04:14:28 +01:00
|
|
|
assert.NotNil(t, stackRes)
|
2017-12-08 22:14:58 +01:00
|
|
|
assert.Equal(t, resource.RootStackType, stackRes.Type)
|
2017-12-06 04:14:28 +01:00
|
|
|
assert.Equal(t, "", string(stackRes.Parent))
|
2018-02-06 01:29:20 +01:00
|
|
|
|
|
|
|
urns := make(map[string]resource.URN)
|
2018-04-03 06:34:54 +02:00
|
|
|
for _, res := range stackInfo.Deployment.Resources[1:] {
|
2018-02-06 01:29:20 +01:00
|
|
|
assert.NotNil(t, res)
|
|
|
|
|
|
|
|
urns[string(res.URN.Name())] = res.URN
|
|
|
|
switch res.URN.Name() {
|
|
|
|
case "a", "f":
|
|
|
|
assert.NotEqual(t, "", res.Parent)
|
|
|
|
assert.Equal(t, stackRes.URN, res.Parent)
|
|
|
|
case "b", "c":
|
|
|
|
assert.Equal(t, urns["a"], res.Parent)
|
|
|
|
case "d", "e":
|
|
|
|
assert.Equal(t, urns["c"], res.Parent)
|
|
|
|
case "g":
|
|
|
|
assert.Equal(t, urns["f"], res.Parent)
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
case "default":
|
|
|
|
// Default providers are not parented.
|
|
|
|
assert.Equal(t, "", string(res.Parent))
|
2018-02-06 01:29:20 +01:00
|
|
|
default:
|
|
|
|
t.Fatalf("unexpected name %s", res.URN.Name())
|
|
|
|
}
|
|
|
|
}
|
2017-12-06 04:14:28 +01:00
|
|
|
}
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
2017-12-11 23:41:57 +01:00
|
|
|
|
2018-04-25 02:23:18 +02:00
|
|
|
func TestStackBadParenting(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "stack_bad_parenting",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
ExpectFailure: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2018-02-22 21:52:50 +01:00
|
|
|
// TestStackDependencyGraph tests that the dependency graph of a stack is saved
|
|
|
|
// in the checkpoint file.
|
|
|
|
func TestStackDependencyGraph(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "stack_dependencies",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
2018-04-03 06:34:54 +02:00
|
|
|
assert.NotNil(t, stackInfo.Deployment)
|
|
|
|
latest := stackInfo.Deployment
|
2018-02-22 21:52:50 +01:00
|
|
|
assert.True(t, len(latest.Resources) >= 2)
|
|
|
|
sawFirst := false
|
|
|
|
sawSecond := false
|
|
|
|
for _, res := range latest.Resources {
|
|
|
|
urn := string(res.URN)
|
|
|
|
if strings.Contains(urn, "dynamic:Resource::first") {
|
|
|
|
// The first resource doesn't depend on anything.
|
|
|
|
assert.Equal(t, 0, len(res.Dependencies))
|
|
|
|
sawFirst = true
|
|
|
|
} else if strings.Contains(urn, "dynamic:Resource::second") {
|
|
|
|
// The second resource uses an Output property of the first resource, so it
|
|
|
|
// depends directly on first.
|
|
|
|
assert.Equal(t, 1, len(res.Dependencies))
|
|
|
|
assert.True(t, strings.Contains(string(res.Dependencies[0]), "dynamic:Resource::first"))
|
|
|
|
sawSecond = true
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
assert.True(t, sawFirst && sawSecond)
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2017-12-11 23:41:57 +01:00
|
|
|
// TestConfigSave ensures that config commands in the Pulumi CLI work as expected.
|
|
|
|
func TestConfigSave(t *testing.T) {
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
|
|
|
|
// Initialize an empty stack.
|
2018-02-14 22:56:16 +01:00
|
|
|
path := filepath.Join(e.RootPath, "Pulumi.yaml")
|
|
|
|
err := (&workspace.Project{
|
2018-11-01 16:28:11 +01:00
|
|
|
Name: "testing-config",
|
|
|
|
Runtime: workspace.NewProjectRuntimeInfo("nodejs", nil),
|
2018-02-14 22:56:16 +01:00
|
|
|
}).Save(path)
|
2017-12-11 23:41:57 +01:00
|
|
|
assert.NoError(t, err)
|
Remove the need to `pulumi init` for the local backend
This change removes the need to `pulumi init` when targeting the local
backend. A fair amount of the change lays the foundation that the next
set of changes to stop having `pulumi init` be used for cloud stacks
as well.
Previously, `pulumi init` logically did two things:
1. It created the bookkeeping directory for local stacks, this was
stored in `<repository-root>/.pulumi`, where `<repository-root>` was
the path to what we belived the "root" of your project was. In the
case of git repositories, this was the directory that contained your
`.git` folder.
2. It recorded repository information in
`<repository-root>/.pulumi/repository.json`. This was used by the
cloud backend when computing what project to interact with on
Pulumi.com
The new identity model will remove the need for (2), since we only
need an owner and stack name to fully qualify a stack on
pulumi.com, so it's easy enough to stop creating a folder just for
that.
However, for the local backend, we need to continue to retain some
information about stacks (e.g. checkpoints, history, etc). In
addition, we need to store our workspace settings (which today just
contains the selected stack) somehere.
For state stored by the local backend, we change the URL scheme from
`local://` to `local://<optional-root-path>`. When
`<optional-root-path>` is unset, it defaults to `$HOME`. We create our
`.pulumi` folder in that directory. This is important because stack
names now must be unique within the backend, but we have some tests
using local stacks which use fixed stack names, so each integration
test really wants its own "view" of the world.
For the workspace settings, we introduce a new `workspaces` directory
in `~/.pulumi`. In this folder we write the workspace settings file
for each project. The file name is the name of the project, combined
with the SHA1 of the path of the project file on disk, to ensure that
multiple pulumi programs with the same project name have different
workspace settings.
This does mean that moving a project's location on disk will cause the
CLI to "forget" what the selected stack was, which is unfortunate, but
not the end of the world. If this ends up being a big pain point, we
can certianly try to play games in the future (for example, if we saw
a .git folder in a parent folder, we could store data in there).
With respect to compatibility, we don't attempt to migrate older files
to their newer locations. For long lived stacks managed using the
local backend, we can provide information on where to move things
to. For all stacks (regardless of backend) we'll require the user to
`pulumi stack select` their stack again, but that seems like the
correct trade-off vs writing complicated upgrade code.
2018-04-17 01:15:10 +02:00
|
|
|
e.RunCommand("pulumi", "login", "--cloud-url", e.LocalURL())
|
2018-04-05 00:31:01 +02:00
|
|
|
e.RunCommand("pulumi", "stack", "init", "testing-2")
|
|
|
|
e.RunCommand("pulumi", "stack", "init", "testing-1")
|
2017-12-11 23:41:57 +01:00
|
|
|
|
|
|
|
// Now configure and save a few different things:
|
2018-02-28 00:51:19 +01:00
|
|
|
e.RunCommand("pulumi", "config", "set", "configA", "value1")
|
|
|
|
e.RunCommand("pulumi", "config", "set", "configB", "value2", "--stack", "testing-2")
|
|
|
|
|
|
|
|
e.RunCommand("pulumi", "stack", "select", "testing-2")
|
|
|
|
|
|
|
|
e.RunCommand("pulumi", "config", "set", "configD", "value4")
|
|
|
|
e.RunCommand("pulumi", "config", "set", "configC", "value3", "--stack", "testing-1")
|
2017-12-11 23:41:57 +01:00
|
|
|
|
|
|
|
// Now read back the config using the CLI:
|
|
|
|
{
|
2017-12-13 19:46:54 +01:00
|
|
|
stdout, _ := e.RunCommand("pulumi", "config", "get", "configB")
|
2017-12-11 23:41:57 +01:00
|
|
|
assert.Equal(t, "value2\n", stdout)
|
|
|
|
}
|
|
|
|
{
|
2018-02-28 00:51:19 +01:00
|
|
|
// the config in a different stack, so this should error.
|
|
|
|
stdout, stderr := e.RunCommandExpectError("pulumi", "config", "get", "configA")
|
2017-12-11 23:41:57 +01:00
|
|
|
assert.Equal(t, "", stdout)
|
|
|
|
assert.NotEqual(t, "", stderr)
|
|
|
|
}
|
|
|
|
{
|
2018-02-28 00:51:19 +01:00
|
|
|
// but selecting the stack should let you see it
|
|
|
|
stdout, _ := e.RunCommand("pulumi", "config", "get", "configA", "--stack", "testing-1")
|
|
|
|
assert.Equal(t, "value1\n", stdout)
|
2017-12-11 23:41:57 +01:00
|
|
|
}
|
2018-02-28 00:51:19 +01:00
|
|
|
|
|
|
|
// Finally, check that the stack file contains what we expected.
|
|
|
|
validate := func(k string, v string, cfg config.Map) {
|
2018-03-02 01:51:09 +01:00
|
|
|
key, err := config.ParseKey("testing-config:config:" + k)
|
|
|
|
assert.NoError(t, err)
|
2018-02-28 00:51:19 +01:00
|
|
|
d, ok := cfg[key]
|
|
|
|
assert.True(t, ok, "config key %v should be set", k)
|
|
|
|
dv, err := d.Value(nil)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
assert.Equal(t, v, dv)
|
2017-12-11 23:41:57 +01:00
|
|
|
}
|
|
|
|
|
2018-02-28 00:51:19 +01:00
|
|
|
testStack1, err := workspace.LoadProjectStack(filepath.Join(e.CWD, "Pulumi.testing-1.yaml"))
|
2017-12-11 23:41:57 +01:00
|
|
|
assert.NoError(t, err)
|
2018-02-28 00:51:19 +01:00
|
|
|
testStack2, err := workspace.LoadProjectStack(filepath.Join(e.CWD, "Pulumi.testing-2.yaml"))
|
2017-12-11 23:41:57 +01:00
|
|
|
assert.NoError(t, err)
|
2017-12-28 03:39:08 +01:00
|
|
|
|
2018-02-28 00:51:19 +01:00
|
|
|
assert.Equal(t, 2, len(testStack1.Config))
|
|
|
|
assert.Equal(t, 2, len(testStack2.Config))
|
|
|
|
|
|
|
|
validate("configA", "value1", testStack1.Config)
|
|
|
|
validate("configC", "value3", testStack1.Config)
|
|
|
|
|
|
|
|
validate("configB", "value2", testStack2.Config)
|
|
|
|
validate("configD", "value4", testStack2.Config)
|
2018-04-17 01:03:40 +02:00
|
|
|
|
|
|
|
e.RunCommand("pulumi", "stack", "rm", "--yes")
|
2017-12-11 23:41:57 +01:00
|
|
|
}
|
2018-02-28 04:21:31 +01:00
|
|
|
|
Support lists and maps in config (#3342)
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).
For example:
```yaml
config:
proj:blah:
- a
- b
- c
proj:hello: world
proj:outer:
inner: value
proj:servers:
- port: 80
```
While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.
As always, you can specify single values with:
```shell
$ pulumi config set hello world
```
Which results in the following YAML:
```yaml
proj:hello world
```
And single value secrets via:
```shell
$ pulumi config set --secret token shhh
```
Which results in the following YAML:
```yaml
proj:token:
secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```
Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:
```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```
Which results in:
```yaml
proj:names
- a
- b
- c
```
Values can be obtained similarly:
```shell
$ pulumi config get --path names[1]
b
```
Or setting values in a map:
```shell
$ pulumi config set --path outer.inner value
```
Which results in:
```yaml
proj:outer:
inner: value
```
Of course, setting values in nested structures is supported:
```shell
$ pulumi config set --path servers[0].port 80
```
Which results in:
```yaml
proj:servers:
- port: 80
```
If you want to include a period in the name of a property, it can be
specified as:
```
$ pulumi config set --path 'nested["foo.bar"]' baz
```
Which results in:
```yaml
proj:nested:
foo.bar: baz
```
Examples of valid paths:
- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'
Note: paths that contain quotes can be surrounded by single quotes.
When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.
Secure values are supported in lists/maps as well:
```shell
$ pulumi config set --path --secret tokens[0] shh
```
Will result in:
```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```
Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:
```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```
**Accessing config values from the command line with JSON**
```shell
$ pulumi config --json
```
Will output:
```json
{
"proj:hello": {
"value": "world",
"secret": false,
"object": false
},
"proj:names": {
"value": "[\"a\",\"b\",\"c\"]",
"secret": false,
"object": true,
"objectValue": [
"a",
"b",
"c"
]
},
"proj:nested": {
"value": "{\"foo.bar\":\"baz\"}",
"secret": false,
"object": true,
"objectValue": {
"foo.bar": "baz"
}
},
"proj:outer": {
"value": "{\"inner\":\"value\"}",
"secret": false,
"object": true,
"objectValue": {
"inner": "value"
}
},
"proj:servers": {
"value": "[{\"port\":80}]",
"secret": false,
"object": true,
"objectValue": [
{
"port": 80
}
]
},
"proj:token": {
"secret": true,
"object": false
},
"proj:tokens": {
"secret": true,
"object": true
}
}
```
If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.
If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.
**Accessing config values from Pulumi programs**
Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:
```typescript
import * as pulumi from "@pulumi/pulumi";
interface Server {
port: number;
}
const config = new pulumi.Config();
const names = config.requireObject<string[]>("names");
for (const n of names) {
console.log(n);
}
const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
console.log(s.port);
}
```
2019-11-01 21:41:27 +01:00
|
|
|
// TestConfigPaths ensures that config commands with paths work as expected.
|
|
|
|
func TestConfigPaths(t *testing.T) {
|
|
|
|
e := ptesting.NewEnvironment(t)
|
|
|
|
defer func() {
|
|
|
|
if !t.Failed() {
|
|
|
|
e.DeleteEnvironment()
|
|
|
|
}
|
|
|
|
}()
|
|
|
|
|
|
|
|
// Initialize an empty stack.
|
|
|
|
path := filepath.Join(e.RootPath, "Pulumi.yaml")
|
|
|
|
err := (&workspace.Project{
|
|
|
|
Name: "testing-config",
|
|
|
|
Runtime: workspace.NewProjectRuntimeInfo("nodejs", nil),
|
|
|
|
}).Save(path)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
e.RunCommand("pulumi", "login", "--cloud-url", e.LocalURL())
|
|
|
|
e.RunCommand("pulumi", "stack", "init", "testing")
|
|
|
|
|
|
|
|
namespaces := []string{"", "my:"}
|
|
|
|
|
|
|
|
tests := []struct {
|
|
|
|
Key string
|
|
|
|
Value string
|
|
|
|
Secret bool
|
|
|
|
Path bool
|
|
|
|
TopLevelKey string
|
|
|
|
TopLevelExpectedValue string
|
|
|
|
}{
|
|
|
|
{
|
|
|
|
Key: "aConfigValue",
|
|
|
|
Value: "this value is a value",
|
|
|
|
TopLevelKey: "aConfigValue",
|
|
|
|
TopLevelExpectedValue: "this value is a value",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "anotherConfigValue",
|
|
|
|
Value: "this value is another value",
|
|
|
|
TopLevelKey: "anotherConfigValue",
|
|
|
|
TopLevelExpectedValue: "this value is another value",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "bEncryptedSecret",
|
|
|
|
Value: "this super secret is encrypted",
|
|
|
|
Secret: true,
|
|
|
|
TopLevelKey: "bEncryptedSecret",
|
|
|
|
TopLevelExpectedValue: "this super secret is encrypted",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "anotherEncryptedSecret",
|
|
|
|
Value: "another encrypted secret",
|
|
|
|
Secret: true,
|
|
|
|
TopLevelKey: "anotherEncryptedSecret",
|
|
|
|
TopLevelExpectedValue: "another encrypted secret",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "[]",
|
|
|
|
Value: "square brackets value",
|
|
|
|
TopLevelKey: "[]",
|
|
|
|
TopLevelExpectedValue: "square brackets value",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "x.y",
|
|
|
|
Value: "x.y value",
|
|
|
|
TopLevelKey: "x.y",
|
|
|
|
TopLevelExpectedValue: "x.y value",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "0",
|
|
|
|
Value: "0 value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "0",
|
|
|
|
TopLevelExpectedValue: "0 value",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "true",
|
|
|
|
Value: "value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "true",
|
|
|
|
TopLevelExpectedValue: "value",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: `["test.Key"]`,
|
|
|
|
Value: "test key value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "test.Key",
|
|
|
|
TopLevelExpectedValue: "test key value",
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: `nested["test.Key"]`,
|
|
|
|
Value: "nested test key value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "nested",
|
|
|
|
TopLevelExpectedValue: `{"test.Key":"nested test key value"}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "outer.inner",
|
|
|
|
Value: "value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "outer",
|
|
|
|
TopLevelExpectedValue: `{"inner":"value"}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "names[0]",
|
|
|
|
Value: "a",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "names",
|
|
|
|
TopLevelExpectedValue: `["a"]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "names[1]",
|
|
|
|
Value: "b",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "names",
|
|
|
|
TopLevelExpectedValue: `["a","b"]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "names[2]",
|
|
|
|
Value: "c",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "names",
|
|
|
|
TopLevelExpectedValue: `["a","b","c"]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "names[3]",
|
|
|
|
Value: "super secret name",
|
|
|
|
Path: true,
|
|
|
|
Secret: true,
|
|
|
|
TopLevelKey: "names",
|
|
|
|
TopLevelExpectedValue: `["a","b","c","super secret name"]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "servers[0].port",
|
|
|
|
Value: "80",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "servers",
|
|
|
|
TopLevelExpectedValue: `[{"port":80}]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "servers[0].host",
|
|
|
|
Value: "example",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "servers",
|
|
|
|
TopLevelExpectedValue: `[{"host":"example","port":80}]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "a.b[0].c",
|
|
|
|
Value: "true",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "a",
|
|
|
|
TopLevelExpectedValue: `{"b":[{"c":true}]}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "a.b[1].c",
|
|
|
|
Value: "false",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "a",
|
|
|
|
TopLevelExpectedValue: `{"b":[{"c":true},{"c":false}]}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "tokens[0]",
|
|
|
|
Value: "shh",
|
|
|
|
Path: true,
|
|
|
|
Secret: true,
|
|
|
|
TopLevelKey: "tokens",
|
|
|
|
TopLevelExpectedValue: `["shh"]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "foo.bar",
|
|
|
|
Value: "don't tell",
|
|
|
|
Path: true,
|
|
|
|
Secret: true,
|
|
|
|
TopLevelKey: "foo",
|
|
|
|
TopLevelExpectedValue: `{"bar":"don't tell"}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "semiInner.a.b.c.d",
|
|
|
|
Value: "1",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "semiInner",
|
|
|
|
TopLevelExpectedValue: `{"a":{"b":{"c":{"d":1}}}}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "wayInner.a.b.c.d.e.f.g.h.i.j.k",
|
|
|
|
Value: "false",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "wayInner",
|
|
|
|
TopLevelExpectedValue: `{"a":{"b":{"c":{"d":{"e":{"f":{"g":{"h":{"i":{"j":{"k":false}}}}}}}}}}}`,
|
|
|
|
},
|
|
|
|
|
|
|
|
// Overwriting a top-level string value is allowed.
|
|
|
|
{
|
|
|
|
Key: "aConfigValue.inner",
|
|
|
|
Value: "new value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "aConfigValue",
|
|
|
|
TopLevelExpectedValue: `{"inner":"new value"}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "anotherConfigValue[0]",
|
|
|
|
Value: "new value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "anotherConfigValue",
|
|
|
|
TopLevelExpectedValue: `["new value"]`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "bEncryptedSecret.inner",
|
|
|
|
Value: "new value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "bEncryptedSecret",
|
|
|
|
TopLevelExpectedValue: `{"inner":"new value"}`,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Key: "anotherEncryptedSecret[0]",
|
|
|
|
Value: "new value",
|
|
|
|
Path: true,
|
|
|
|
TopLevelKey: "anotherEncryptedSecret",
|
|
|
|
TopLevelExpectedValue: `["new value"]`,
|
|
|
|
},
|
|
|
|
}
|
|
|
|
|
|
|
|
validateConfigGet := func(key string, value string, path bool) {
|
|
|
|
args := []string{"config", "get", key}
|
|
|
|
if path {
|
|
|
|
args = append(args, "--path")
|
|
|
|
}
|
|
|
|
stdout, stderr := e.RunCommand("pulumi", args...)
|
|
|
|
assert.Equal(t, fmt.Sprintf("%s\n", value), stdout)
|
|
|
|
assert.Equal(t, "", stderr)
|
|
|
|
}
|
|
|
|
|
|
|
|
for _, ns := range namespaces {
|
|
|
|
for _, test := range tests {
|
|
|
|
key := fmt.Sprintf("%s%s", ns, test.Key)
|
|
|
|
topLevelKey := fmt.Sprintf("%s%s", ns, test.TopLevelKey)
|
|
|
|
|
|
|
|
// Set the value.
|
|
|
|
args := []string{"config", "set"}
|
|
|
|
if test.Secret {
|
|
|
|
args = append(args, "--secret")
|
|
|
|
}
|
|
|
|
if test.Path {
|
|
|
|
args = append(args, "--path")
|
|
|
|
}
|
|
|
|
args = append(args, key, test.Value)
|
|
|
|
stdout, stderr := e.RunCommand("pulumi", args...)
|
|
|
|
assert.Equal(t, "", stdout)
|
|
|
|
assert.Equal(t, "", stderr)
|
|
|
|
|
|
|
|
// Get the value and validate it.
|
|
|
|
validateConfigGet(key, test.Value, test.Path)
|
|
|
|
|
|
|
|
// Get the top-level value and validate it.
|
|
|
|
validateConfigGet(topLevelKey, test.TopLevelExpectedValue, false /*path*/)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
badKeys := []string{
|
|
|
|
// Syntax errors.
|
|
|
|
"root[",
|
|
|
|
`root["nested]`,
|
|
|
|
"root.array[abc]",
|
|
|
|
"root.[1]",
|
|
|
|
|
|
|
|
// First path segment must be a non-empty string.
|
|
|
|
`[""]`,
|
|
|
|
"[0]",
|
|
|
|
|
|
|
|
// Index out of range.
|
|
|
|
"names[-1]",
|
|
|
|
"names[5]",
|
|
|
|
|
|
|
|
// A "secure" key that is a map with a single string value is reserved by the system.
|
|
|
|
"key.secure",
|
|
|
|
"super.nested.map.secure",
|
|
|
|
|
|
|
|
// Type mismatch.
|
|
|
|
"outer[0]",
|
|
|
|
"names.nested",
|
|
|
|
"outer.inner.nested",
|
|
|
|
"outer.inner[0]",
|
|
|
|
}
|
|
|
|
|
|
|
|
for _, ns := range namespaces {
|
|
|
|
for _, badKey := range badKeys {
|
|
|
|
key := fmt.Sprintf("%s%s", ns, badKey)
|
|
|
|
stdout, stderr := e.RunCommandExpectError("pulumi", "config", "set", "--path", key, "value")
|
|
|
|
assert.Equal(t, "", stdout)
|
|
|
|
assert.NotEqual(t, "", stderr)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
e.RunCommand("pulumi", "stack", "rm", "--yes")
|
|
|
|
}
|
|
|
|
|
2018-03-14 20:24:49 +01:00
|
|
|
// Tests basic configuration from the perspective of a Pulumi program.
|
|
|
|
func TestConfigBasicNodeJS(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("config_basic", "nodejs"),
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{
|
|
|
|
"aConfigValue": "this value is a value",
|
|
|
|
},
|
|
|
|
Secrets: map[string]string{
|
|
|
|
"bEncryptedSecret": "this super secret is encrypted",
|
|
|
|
},
|
Support lists and maps in config (#3342)
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).
For example:
```yaml
config:
proj:blah:
- a
- b
- c
proj:hello: world
proj:outer:
inner: value
proj:servers:
- port: 80
```
While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.
As always, you can specify single values with:
```shell
$ pulumi config set hello world
```
Which results in the following YAML:
```yaml
proj:hello world
```
And single value secrets via:
```shell
$ pulumi config set --secret token shhh
```
Which results in the following YAML:
```yaml
proj:token:
secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```
Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:
```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```
Which results in:
```yaml
proj:names
- a
- b
- c
```
Values can be obtained similarly:
```shell
$ pulumi config get --path names[1]
b
```
Or setting values in a map:
```shell
$ pulumi config set --path outer.inner value
```
Which results in:
```yaml
proj:outer:
inner: value
```
Of course, setting values in nested structures is supported:
```shell
$ pulumi config set --path servers[0].port 80
```
Which results in:
```yaml
proj:servers:
- port: 80
```
If you want to include a period in the name of a property, it can be
specified as:
```
$ pulumi config set --path 'nested["foo.bar"]' baz
```
Which results in:
```yaml
proj:nested:
foo.bar: baz
```
Examples of valid paths:
- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'
Note: paths that contain quotes can be surrounded by single quotes.
When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.
Secure values are supported in lists/maps as well:
```shell
$ pulumi config set --path --secret tokens[0] shh
```
Will result in:
```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```
Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:
```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```
**Accessing config values from the command line with JSON**
```shell
$ pulumi config --json
```
Will output:
```json
{
"proj:hello": {
"value": "world",
"secret": false,
"object": false
},
"proj:names": {
"value": "[\"a\",\"b\",\"c\"]",
"secret": false,
"object": true,
"objectValue": [
"a",
"b",
"c"
]
},
"proj:nested": {
"value": "{\"foo.bar\":\"baz\"}",
"secret": false,
"object": true,
"objectValue": {
"foo.bar": "baz"
}
},
"proj:outer": {
"value": "{\"inner\":\"value\"}",
"secret": false,
"object": true,
"objectValue": {
"inner": "value"
}
},
"proj:servers": {
"value": "[{\"port\":80}]",
"secret": false,
"object": true,
"objectValue": [
{
"port": 80
}
]
},
"proj:token": {
"secret": true,
"object": false
},
"proj:tokens": {
"secret": true,
"object": true
}
}
```
If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.
If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.
**Accessing config values from Pulumi programs**
Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:
```typescript
import * as pulumi from "@pulumi/pulumi";
interface Server {
port: number;
}
const config = new pulumi.Config();
const names = config.requireObject<string[]>("names");
for (const n of names) {
console.log(n);
}
const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
console.log(s.port);
}
```
2019-11-01 21:41:27 +01:00
|
|
|
OrderedConfig: []integration.ConfigValue{
|
|
|
|
{Key: "outer.inner", Value: "value", Path: true},
|
|
|
|
{Key: "names[0]", Value: "a", Path: true},
|
|
|
|
{Key: "names[1]", Value: "b", Path: true},
|
|
|
|
{Key: "names[2]", Value: "c", Path: true},
|
|
|
|
{Key: "names[3]", Value: "super secret name", Path: true, Secret: true},
|
|
|
|
{Key: "servers[0].port", Value: "80", Path: true},
|
|
|
|
{Key: "servers[0].host", Value: "example", Path: true},
|
|
|
|
{Key: "a.b[0].c", Value: "true", Path: true},
|
|
|
|
{Key: "a.b[1].c", Value: "false", Path: true},
|
|
|
|
{Key: "tokens[0]", Value: "shh", Path: true, Secret: true},
|
|
|
|
{Key: "foo.bar", Value: "don't tell", Path: true, Secret: true},
|
|
|
|
},
|
2018-03-14 20:24:49 +01:00
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2018-08-08 22:13:38 +02:00
|
|
|
func TestConfigCaptureNodeJS(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("config_capture_e2e", "nodejs"),
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{
|
|
|
|
"value": "it works",
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2018-09-07 01:35:14 +02:00
|
|
|
func TestInvalidVersionInPackageJson(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("invalid_package_json"),
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{},
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2018-03-14 20:24:49 +01:00
|
|
|
// Tests basic configuration from the perspective of a Pulumi program.
|
|
|
|
func TestConfigBasicPython(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
Support lists and maps in config (#3342)
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).
For example:
```yaml
config:
proj:blah:
- a
- b
- c
proj:hello: world
proj:outer:
inner: value
proj:servers:
- port: 80
```
While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.
As always, you can specify single values with:
```shell
$ pulumi config set hello world
```
Which results in the following YAML:
```yaml
proj:hello world
```
And single value secrets via:
```shell
$ pulumi config set --secret token shhh
```
Which results in the following YAML:
```yaml
proj:token:
secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```
Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:
```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```
Which results in:
```yaml
proj:names
- a
- b
- c
```
Values can be obtained similarly:
```shell
$ pulumi config get --path names[1]
b
```
Or setting values in a map:
```shell
$ pulumi config set --path outer.inner value
```
Which results in:
```yaml
proj:outer:
inner: value
```
Of course, setting values in nested structures is supported:
```shell
$ pulumi config set --path servers[0].port 80
```
Which results in:
```yaml
proj:servers:
- port: 80
```
If you want to include a period in the name of a property, it can be
specified as:
```
$ pulumi config set --path 'nested["foo.bar"]' baz
```
Which results in:
```yaml
proj:nested:
foo.bar: baz
```
Examples of valid paths:
- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'
Note: paths that contain quotes can be surrounded by single quotes.
When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.
Secure values are supported in lists/maps as well:
```shell
$ pulumi config set --path --secret tokens[0] shh
```
Will result in:
```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```
Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:
```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```
**Accessing config values from the command line with JSON**
```shell
$ pulumi config --json
```
Will output:
```json
{
"proj:hello": {
"value": "world",
"secret": false,
"object": false
},
"proj:names": {
"value": "[\"a\",\"b\",\"c\"]",
"secret": false,
"object": true,
"objectValue": [
"a",
"b",
"c"
]
},
"proj:nested": {
"value": "{\"foo.bar\":\"baz\"}",
"secret": false,
"object": true,
"objectValue": {
"foo.bar": "baz"
}
},
"proj:outer": {
"value": "{\"inner\":\"value\"}",
"secret": false,
"object": true,
"objectValue": {
"inner": "value"
}
},
"proj:servers": {
"value": "[{\"port\":80}]",
"secret": false,
"object": true,
"objectValue": [
{
"port": 80
}
]
},
"proj:token": {
"secret": true,
"object": false
},
"proj:tokens": {
"secret": true,
"object": true
}
}
```
If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.
If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.
**Accessing config values from Pulumi programs**
Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:
```typescript
import * as pulumi from "@pulumi/pulumi";
interface Server {
port: number;
}
const config = new pulumi.Config();
const names = config.requireObject<string[]>("names");
for (const n of names) {
console.log(n);
}
const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
console.log(s.port);
}
```
2019-11-01 21:41:27 +01:00
|
|
|
Dir: filepath.Join("config_basic", "python"),
|
|
|
|
Dependencies: []string{
|
|
|
|
path.Join("..", "..", "sdk", "python", "env", "src"),
|
|
|
|
},
|
2018-03-14 20:24:49 +01:00
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{
|
|
|
|
"aConfigValue": "this value is a Pythonic value",
|
|
|
|
},
|
|
|
|
Secrets: map[string]string{
|
|
|
|
"bEncryptedSecret": "this super Pythonic secret is encrypted",
|
|
|
|
},
|
Support lists and maps in config (#3342)
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).
For example:
```yaml
config:
proj:blah:
- a
- b
- c
proj:hello: world
proj:outer:
inner: value
proj:servers:
- port: 80
```
While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.
As always, you can specify single values with:
```shell
$ pulumi config set hello world
```
Which results in the following YAML:
```yaml
proj:hello world
```
And single value secrets via:
```shell
$ pulumi config set --secret token shhh
```
Which results in the following YAML:
```yaml
proj:token:
secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```
Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:
```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```
Which results in:
```yaml
proj:names
- a
- b
- c
```
Values can be obtained similarly:
```shell
$ pulumi config get --path names[1]
b
```
Or setting values in a map:
```shell
$ pulumi config set --path outer.inner value
```
Which results in:
```yaml
proj:outer:
inner: value
```
Of course, setting values in nested structures is supported:
```shell
$ pulumi config set --path servers[0].port 80
```
Which results in:
```yaml
proj:servers:
- port: 80
```
If you want to include a period in the name of a property, it can be
specified as:
```
$ pulumi config set --path 'nested["foo.bar"]' baz
```
Which results in:
```yaml
proj:nested:
foo.bar: baz
```
Examples of valid paths:
- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'
Note: paths that contain quotes can be surrounded by single quotes.
When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.
Secure values are supported in lists/maps as well:
```shell
$ pulumi config set --path --secret tokens[0] shh
```
Will result in:
```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```
Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:
```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```
**Accessing config values from the command line with JSON**
```shell
$ pulumi config --json
```
Will output:
```json
{
"proj:hello": {
"value": "world",
"secret": false,
"object": false
},
"proj:names": {
"value": "[\"a\",\"b\",\"c\"]",
"secret": false,
"object": true,
"objectValue": [
"a",
"b",
"c"
]
},
"proj:nested": {
"value": "{\"foo.bar\":\"baz\"}",
"secret": false,
"object": true,
"objectValue": {
"foo.bar": "baz"
}
},
"proj:outer": {
"value": "{\"inner\":\"value\"}",
"secret": false,
"object": true,
"objectValue": {
"inner": "value"
}
},
"proj:servers": {
"value": "[{\"port\":80}]",
"secret": false,
"object": true,
"objectValue": [
{
"port": 80
}
]
},
"proj:token": {
"secret": true,
"object": false
},
"proj:tokens": {
"secret": true,
"object": true
}
}
```
If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.
If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.
**Accessing config values from Pulumi programs**
Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:
```typescript
import * as pulumi from "@pulumi/pulumi";
interface Server {
port: number;
}
const config = new pulumi.Config();
const names = config.requireObject<string[]>("names");
for (const n of names) {
console.log(n);
}
const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
console.log(s.port);
}
```
2019-11-01 21:41:27 +01:00
|
|
|
OrderedConfig: []integration.ConfigValue{
|
|
|
|
{Key: "outer.inner", Value: "value", Path: true},
|
|
|
|
{Key: "names[0]", Value: "a", Path: true},
|
|
|
|
{Key: "names[1]", Value: "b", Path: true},
|
|
|
|
{Key: "names[2]", Value: "c", Path: true},
|
|
|
|
{Key: "names[3]", Value: "super secret name", Path: true, Secret: true},
|
|
|
|
{Key: "servers[0].port", Value: "80", Path: true},
|
|
|
|
{Key: "servers[0].host", Value: "example", Path: true},
|
|
|
|
{Key: "a.b[0].c", Value: "true", Path: true},
|
|
|
|
{Key: "a.b[1].c", Value: "false", Path: true},
|
|
|
|
{Key: "tokens[0]", Value: "shh", Path: true, Secret: true},
|
|
|
|
{Key: "foo.bar", Value: "don't tell", Path: true, Secret: true},
|
|
|
|
},
|
2018-03-14 20:24:49 +01:00
|
|
|
})
|
|
|
|
}
|
2018-06-10 18:24:46 +02:00
|
|
|
|
|
|
|
// Tests basic configuration from the perspective of a Pulumi Go program.
|
|
|
|
func TestConfigBasicGo(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("config_basic", "go"),
|
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{
|
|
|
|
"aConfigValue": "this value is a value",
|
|
|
|
},
|
|
|
|
Secrets: map[string]string{
|
|
|
|
"bEncryptedSecret": "this super secret is encrypted",
|
|
|
|
},
|
Support lists and maps in config (#3342)
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).
For example:
```yaml
config:
proj:blah:
- a
- b
- c
proj:hello: world
proj:outer:
inner: value
proj:servers:
- port: 80
```
While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.
As always, you can specify single values with:
```shell
$ pulumi config set hello world
```
Which results in the following YAML:
```yaml
proj:hello world
```
And single value secrets via:
```shell
$ pulumi config set --secret token shhh
```
Which results in the following YAML:
```yaml
proj:token:
secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```
Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:
```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```
Which results in:
```yaml
proj:names
- a
- b
- c
```
Values can be obtained similarly:
```shell
$ pulumi config get --path names[1]
b
```
Or setting values in a map:
```shell
$ pulumi config set --path outer.inner value
```
Which results in:
```yaml
proj:outer:
inner: value
```
Of course, setting values in nested structures is supported:
```shell
$ pulumi config set --path servers[0].port 80
```
Which results in:
```yaml
proj:servers:
- port: 80
```
If you want to include a period in the name of a property, it can be
specified as:
```
$ pulumi config set --path 'nested["foo.bar"]' baz
```
Which results in:
```yaml
proj:nested:
foo.bar: baz
```
Examples of valid paths:
- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'
Note: paths that contain quotes can be surrounded by single quotes.
When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.
Secure values are supported in lists/maps as well:
```shell
$ pulumi config set --path --secret tokens[0] shh
```
Will result in:
```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```
Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:
```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```
**Accessing config values from the command line with JSON**
```shell
$ pulumi config --json
```
Will output:
```json
{
"proj:hello": {
"value": "world",
"secret": false,
"object": false
},
"proj:names": {
"value": "[\"a\",\"b\",\"c\"]",
"secret": false,
"object": true,
"objectValue": [
"a",
"b",
"c"
]
},
"proj:nested": {
"value": "{\"foo.bar\":\"baz\"}",
"secret": false,
"object": true,
"objectValue": {
"foo.bar": "baz"
}
},
"proj:outer": {
"value": "{\"inner\":\"value\"}",
"secret": false,
"object": true,
"objectValue": {
"inner": "value"
}
},
"proj:servers": {
"value": "[{\"port\":80}]",
"secret": false,
"object": true,
"objectValue": [
{
"port": 80
}
]
},
"proj:token": {
"secret": true,
"object": false
},
"proj:tokens": {
"secret": true,
"object": true
}
}
```
If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.
If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.
**Accessing config values from Pulumi programs**
Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:
```typescript
import * as pulumi from "@pulumi/pulumi";
interface Server {
port: number;
}
const config = new pulumi.Config();
const names = config.requireObject<string[]>("names");
for (const n of names) {
console.log(n);
}
const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
console.log(s.port);
}
```
2019-11-01 21:41:27 +01:00
|
|
|
OrderedConfig: []integration.ConfigValue{
|
|
|
|
{Key: "outer.inner", Value: "value", Path: true},
|
|
|
|
{Key: "names[0]", Value: "a", Path: true},
|
|
|
|
{Key: "names[1]", Value: "b", Path: true},
|
|
|
|
{Key: "names[2]", Value: "c", Path: true},
|
|
|
|
{Key: "names[3]", Value: "super secret name", Path: true, Secret: true},
|
|
|
|
{Key: "servers[0].port", Value: "80", Path: true},
|
|
|
|
{Key: "servers[0].host", Value: "example", Path: true},
|
|
|
|
{Key: "a.b[0].c", Value: "true", Path: true},
|
|
|
|
{Key: "a.b[1].c", Value: "false", Path: true},
|
|
|
|
{Key: "tokens[0]", Value: "shh", Path: true, Secret: true},
|
|
|
|
{Key: "foo.bar", Value: "don't tell", Path: true, Secret: true},
|
|
|
|
},
|
2019-11-14 18:25:55 +01:00
|
|
|
RunBuild: true,
|
2018-06-10 18:24:46 +02:00
|
|
|
})
|
|
|
|
}
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
|
2019-10-26 01:59:50 +02:00
|
|
|
// Tests basic configuration from the perspective of a Pulumi .NET program.
|
|
|
|
func TestConfigBasicDotNet(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("config_basic", "dotnet"),
|
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{
|
|
|
|
"aConfigValue": "this value is a value",
|
|
|
|
},
|
|
|
|
Secrets: map[string]string{
|
|
|
|
"bEncryptedSecret": "this super secret is encrypted",
|
|
|
|
},
|
Support lists and maps in config (#3342)
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).
For example:
```yaml
config:
proj:blah:
- a
- b
- c
proj:hello: world
proj:outer:
inner: value
proj:servers:
- port: 80
```
While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.
As always, you can specify single values with:
```shell
$ pulumi config set hello world
```
Which results in the following YAML:
```yaml
proj:hello world
```
And single value secrets via:
```shell
$ pulumi config set --secret token shhh
```
Which results in the following YAML:
```yaml
proj:token:
secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```
Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:
```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```
Which results in:
```yaml
proj:names
- a
- b
- c
```
Values can be obtained similarly:
```shell
$ pulumi config get --path names[1]
b
```
Or setting values in a map:
```shell
$ pulumi config set --path outer.inner value
```
Which results in:
```yaml
proj:outer:
inner: value
```
Of course, setting values in nested structures is supported:
```shell
$ pulumi config set --path servers[0].port 80
```
Which results in:
```yaml
proj:servers:
- port: 80
```
If you want to include a period in the name of a property, it can be
specified as:
```
$ pulumi config set --path 'nested["foo.bar"]' baz
```
Which results in:
```yaml
proj:nested:
foo.bar: baz
```
Examples of valid paths:
- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'
Note: paths that contain quotes can be surrounded by single quotes.
When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.
Secure values are supported in lists/maps as well:
```shell
$ pulumi config set --path --secret tokens[0] shh
```
Will result in:
```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```
Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:
```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```
**Accessing config values from the command line with JSON**
```shell
$ pulumi config --json
```
Will output:
```json
{
"proj:hello": {
"value": "world",
"secret": false,
"object": false
},
"proj:names": {
"value": "[\"a\",\"b\",\"c\"]",
"secret": false,
"object": true,
"objectValue": [
"a",
"b",
"c"
]
},
"proj:nested": {
"value": "{\"foo.bar\":\"baz\"}",
"secret": false,
"object": true,
"objectValue": {
"foo.bar": "baz"
}
},
"proj:outer": {
"value": "{\"inner\":\"value\"}",
"secret": false,
"object": true,
"objectValue": {
"inner": "value"
}
},
"proj:servers": {
"value": "[{\"port\":80}]",
"secret": false,
"object": true,
"objectValue": [
{
"port": 80
}
]
},
"proj:token": {
"secret": true,
"object": false
},
"proj:tokens": {
"secret": true,
"object": true
}
}
```
If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.
If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.
**Accessing config values from Pulumi programs**
Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:
```typescript
import * as pulumi from "@pulumi/pulumi";
interface Server {
port: number;
}
const config = new pulumi.Config();
const names = config.requireObject<string[]>("names");
for (const n of names) {
console.log(n);
}
const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
console.log(s.port);
}
```
2019-11-01 21:41:27 +01:00
|
|
|
OrderedConfig: []integration.ConfigValue{
|
|
|
|
{Key: "outer.inner", Value: "value", Path: true},
|
|
|
|
{Key: "names[0]", Value: "a", Path: true},
|
|
|
|
{Key: "names[1]", Value: "b", Path: true},
|
|
|
|
{Key: "names[2]", Value: "c", Path: true},
|
|
|
|
{Key: "names[3]", Value: "super secret name", Path: true, Secret: true},
|
|
|
|
{Key: "servers[0].port", Value: "80", Path: true},
|
|
|
|
{Key: "servers[0].host", Value: "example", Path: true},
|
|
|
|
{Key: "a.b[0].c", Value: "true", Path: true},
|
|
|
|
{Key: "a.b[1].c", Value: "false", Path: true},
|
|
|
|
{Key: "tokens[0]", Value: "shh", Path: true, Secret: true},
|
|
|
|
{Key: "foo.bar", Value: "don't tell", Path: true, Secret: true},
|
|
|
|
},
|
2019-10-26 01:59:50 +02:00
|
|
|
})
|
|
|
|
}
|
|
|
|
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
// Tests an explicit provider instance.
|
|
|
|
func TestExplicitProvider(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "explicit_provider",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
assert.NotNil(t, stackInfo.Deployment)
|
|
|
|
latest := stackInfo.Deployment
|
|
|
|
|
|
|
|
// Expect one stack resource, two provider resources, and two custom resources.
|
|
|
|
assert.True(t, len(latest.Resources) == 5)
|
|
|
|
|
Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.
We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.
This is perhaps clearer when described by example. Consider the
following dependency graph:
A
__|__
B C
| _|_
D E F
In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.
In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 18:46:30 +01:00
|
|
|
var defaultProvider *apitype.ResourceV3
|
|
|
|
var explicitProvider *apitype.ResourceV3
|
Implement first-class providers. (#1695)
### First-Class Providers
These changes implement support for first-class providers. First-class
providers are provider plugins that are exposed as resources via the
Pulumi programming model so that they may be explicitly and multiply
instantiated. Each instance of a provider resource may be configured
differently, and configuration parameters may be source from the
outputs of other resources.
### Provider Plugin Changes
In order to accommodate the need to verify and diff provider
configuration and configure providers without complete configuration
information, these changes adjust the high-level provider plugin
interface. Two new methods for validating a provider's configuration
and diffing changes to the same have been added (`CheckConfig` and
`DiffConfig`, respectively), and the type of the configuration bag
accepted by `Configure` has been changed to a `PropertyMap`.
These changes have not yet been reflected in the provider plugin gRPC
interface. We will do this in a set of follow-up changes. Until then,
these methods are implemented by adapters:
- `CheckConfig` validates that all configuration parameters are string
or unknown properties. This is necessary because existing plugins
only accept string-typed configuration values.
- `DiffConfig` either returns "never replace" if all configuration
values are known or "must replace" if any configuration value is
unknown. The justification for this behavior is given
[here](https://github.com/pulumi/pulumi/pull/1695/files#diff-a6cd5c7f337665f5bb22e92ca5f07537R106)
- `Configure` converts the config bag to a legacy config map and
configures the provider plugin if all config values are known. If any
config value is unknown, the underlying plugin is not configured and
the provider may only perform `Check`, `Read`, and `Invoke`, all of
which return empty results. We justify this behavior becuase it is
only possible during a preview and provides the best experience we
can manage with the existing gRPC interface.
### Resource Model Changes
Providers are now exposed as resources that participate in a stack's
dependency graph. Like other resources, they are explicitly created,
may have multiple instances, and may have dependencies on other
resources. Providers are referred to using provider references, which
are a combination of the provider's URN and its ID. This design
addresses the need during a preview to refer to providers that have not
yet been physically created and therefore have no ID.
All custom resources that are not themselves providers must specify a
single provider via a provider reference. The named provider will be
used to manage that resource's CRUD operations. If a resource's
provider reference changes, the resource must be replaced. Though its
URN is not present in the resource's dependency list, the provider
should be treated as a dependency of the resource when topologically
sorting the dependency graph.
Finally, `Invoke` operations must now specify a provider to use for the
invocation via a provider reference.
### Engine Changes
First-class providers support requires a few changes to the engine:
- The engine must have some way to map from provider references to
provider plugins. It must be possible to add providers from a stack's
checkpoint to this map and to register new/updated providers during
the execution of a plan in response to CRUD operations on provider
resources.
- In order to support updating existing stacks using existing Pulumi
programs that may not explicitly instantiate providers, the engine
must be able to manage the "default" providers for each package
referenced by a checkpoint or Pulumi program. The configuration for
a "default" provider is taken from the stack's configuration data.
The former need is addressed by adding a provider registry type that is
responsible for managing all of the plugins required by a plan. In
addition to loading plugins froma checkpoint and providing the ability
to map from a provider reference to a provider plugin, this type serves
as the provider plugin for providers themselves (i.e. it is the
"provider provider").
The latter need is solved via two relatively self-contained changes to
plan setup and the eval source.
During plan setup, the old checkpoint is scanned for custom resources
that do not have a provider reference in order to compute the set of
packages that require a default provider. Once this set has been
computed, the required default provider definitions are conjured and
prepended to the checkpoint's resource list. Each resource that
requires a default provider is then updated to refer to the default
provider for its package.
While an eval source is running, each custom resource registration,
resource read, and invoke that does not name a provider is trapped
before being returned by the source iterator. If no default provider
for the appropriate package has been registered, the eval source
synthesizes an appropriate registration, waits for it to complete, and
records the registered provider's reference. This reference is injected
into the original request, which is then processed as usual. If a
default provider was already registered, the recorded reference is
used and no new registration occurs.
### SDK Changes
These changes only expose first-class providers from the Node.JS SDK.
- A new abstract class, `ProviderResource`, can be subclassed and used
to instantiate first-class providers.
- A new field in `ResourceOptions`, `provider`, can be used to supply
a particular provider instance to manage a `CustomResource`'s CRUD
operations.
- A new type, `InvokeOptions`, can be used to specify options that
control the behavior of a call to `pulumi.runtime.invoke`. This type
includes a `provider` field that is analogous to
`ResourceOptions.provider`.
2018-08-07 02:50:29 +02:00
|
|
|
for _, res := range latest.Resources {
|
|
|
|
urn := res.URN
|
|
|
|
switch urn.Name() {
|
|
|
|
case "default":
|
|
|
|
assert.True(t, providers.IsProviderType(res.Type))
|
|
|
|
assert.Nil(t, defaultProvider)
|
|
|
|
prov := res
|
|
|
|
defaultProvider = &prov
|
|
|
|
|
|
|
|
case "p":
|
|
|
|
assert.True(t, providers.IsProviderType(res.Type))
|
|
|
|
assert.Nil(t, explicitProvider)
|
|
|
|
prov := res
|
|
|
|
explicitProvider = &prov
|
|
|
|
|
|
|
|
case "a":
|
|
|
|
prov, err := providers.ParseReference(res.Provider)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
assert.NotNil(t, defaultProvider)
|
|
|
|
defaultRef, err := providers.NewReference(defaultProvider.URN, defaultProvider.ID)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
assert.Equal(t, defaultRef.String(), prov.String())
|
|
|
|
|
|
|
|
case "b":
|
|
|
|
prov, err := providers.ParseReference(res.Provider)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
assert.NotNil(t, explicitProvider)
|
|
|
|
explicitRef, err := providers.NewReference(explicitProvider.URN, explicitProvider.ID)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
assert.Equal(t, explicitRef.String(), prov.String())
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
assert.NotNil(t, defaultProvider)
|
|
|
|
assert.NotNil(t, explicitProvider)
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
2018-08-08 21:06:20 +02:00
|
|
|
|
|
|
|
// Tests that reads of unknown IDs do not fail.
|
|
|
|
func TestGetCreated(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "get_created",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
})
|
|
|
|
}
|
2018-11-14 22:33:35 +01:00
|
|
|
|
2019-05-30 23:12:37 +02:00
|
|
|
// Tests that stack references work in Node.
|
|
|
|
func TestStackReferenceNodeJS(t *testing.T) {
|
|
|
|
if owner := os.Getenv("PULUMI_TEST_OWNER"); owner == "" {
|
2019-01-31 01:54:12 +01:00
|
|
|
t.Skipf("Skipping: PULUMI_TEST_OWNER is not set")
|
|
|
|
}
|
|
|
|
|
2018-11-14 22:33:35 +01:00
|
|
|
opts := &integration.ProgramTestOptions{
|
|
|
|
Dir: "stack_reference",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
2019-01-31 01:54:12 +01:00
|
|
|
Config: map[string]string{
|
|
|
|
"org": os.Getenv("PULUMI_TEST_OWNER"),
|
|
|
|
},
|
2019-06-17 21:25:56 +02:00
|
|
|
EditDirs: []integration.EditDir{
|
|
|
|
{
|
|
|
|
Dir: "step1",
|
|
|
|
Additive: true,
|
|
|
|
},
|
|
|
|
{
|
|
|
|
Dir: "step2",
|
|
|
|
Additive: true,
|
|
|
|
},
|
|
|
|
},
|
2018-11-14 22:33:35 +01:00
|
|
|
}
|
|
|
|
integration.ProgramTest(t, opts)
|
|
|
|
}
|
2018-11-19 23:54:24 +01:00
|
|
|
|
2019-05-30 23:12:37 +02:00
|
|
|
func TestStackReferencePython(t *testing.T) {
|
|
|
|
if owner := os.Getenv("PULUMI_TEST_OWNER"); owner == "" {
|
|
|
|
t.Skipf("Skipping: PULUMI_TEST_OWNER is not set")
|
|
|
|
}
|
|
|
|
|
|
|
|
opts := &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("stack_reference", "python"),
|
|
|
|
Dependencies: []string{
|
|
|
|
filepath.Join("..", "..", "sdk", "python", "env", "src"),
|
|
|
|
},
|
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{
|
|
|
|
"org": os.Getenv("PULUMI_TEST_OWNER"),
|
|
|
|
},
|
|
|
|
}
|
|
|
|
integration.ProgramTest(t, opts)
|
|
|
|
}
|
|
|
|
|
2019-10-26 01:59:50 +02:00
|
|
|
func TestStackReferenceDotNet(t *testing.T) {
|
|
|
|
if owner := os.Getenv("PULUMI_TEST_OWNER"); owner == "" {
|
|
|
|
t.Skipf("Skipping: PULUMI_TEST_OWNER is not set")
|
|
|
|
}
|
|
|
|
|
|
|
|
opts := &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("stack_reference", "dotnet"),
|
|
|
|
Quick: true,
|
|
|
|
Config: map[string]string{
|
|
|
|
"org": os.Getenv("PULUMI_TEST_OWNER"),
|
|
|
|
},
|
|
|
|
}
|
|
|
|
integration.ProgramTest(t, opts)
|
|
|
|
}
|
|
|
|
|
2018-11-19 23:54:24 +01:00
|
|
|
// Tests that we issue an error if we fail to locate the Python command when running
|
|
|
|
// a Python example.
|
|
|
|
func TestPython3NotInstalled(t *testing.T) {
|
|
|
|
stderr := &bytes.Buffer{}
|
|
|
|
badPython := "python3000"
|
|
|
|
expectedError := fmt.Sprintf(
|
|
|
|
"error: Failed to locate '%s' on your PATH. Have you installed Python 3.6 or greater?",
|
|
|
|
badPython)
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: path.Join("empty", "python"),
|
|
|
|
Dependencies: []string{
|
|
|
|
path.Join("..", "..", "sdk", "python", "env", "src"),
|
|
|
|
},
|
|
|
|
Quick: true,
|
|
|
|
Env: []string{
|
|
|
|
// Note: we use PULUMI_PYTHON_CMD to override the default behavior of searching
|
|
|
|
// for Python 3, since anyone running tests surely already has Python 3 installed on their
|
|
|
|
// machine. The code paths are functionally the same.
|
|
|
|
fmt.Sprintf("PULUMI_PYTHON_CMD=%s", badPython),
|
|
|
|
},
|
|
|
|
ExpectFailure: true,
|
|
|
|
Stderr: stderr,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
output := stderr.String()
|
|
|
|
assert.Contains(t, output, expectedError)
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
2019-05-17 22:51:28 +02:00
|
|
|
|
|
|
|
// TestProviderSecretConfig that a first class provider can be created when it has secrets as part of its config.
|
|
|
|
func TestProviderSecretConfig(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "provider_secret_config",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
})
|
|
|
|
}
|
2019-06-25 01:45:54 +02:00
|
|
|
|
Python Dynamic Providers (#2900)
Dynamic providers in Python.
This PR uses [dill](https://pypi.org/project/dill/) for code serialization, along with a customization to help ensure deterministic serialization results.
One notable limitation - which I believe is a general requirement of Python - is that any serialization of Python functions must serialize byte code, and byte code is not safely versioned across Python versions. So any resource created with Python `3.x.y` can only be updated by exactly the same version of Python. This is very constraining, but it's not clear there is any other option within the realm of what "dynamic providers" are as a feature. It is plausible that we could ensure that updates which only update the serialized provider can avoid calling the dynamic provider operations, so that version updates could still be accomplished. We can explore this separately.
```py
from pulumi import ComponentResource, export, Input, Output
from pulumi.dynamic import Resource, ResourceProvider, CreateResult, UpdateResult
from typing import Optional
from github import Github, GithubObject
auth = "<auth token>"
g = Github(auth)
class GithubLabelArgs(object):
owner: Input[str]
repo: Input[str]
name: Input[str]
color: Input[str]
description: Optional[Input[str]]
def __init__(self, owner, repo, name, color, description=None):
self.owner = owner
self.repo = repo
self.name = name
self.color = color
self.description = description
class GithubLabelProvider(ResourceProvider):
def create(self, props):
l = g.get_user(props["owner"]).get_repo(props["repo"]).create_label(
name=props["name"],
color=props["color"],
description=props.get("description", GithubObject.NotSet))
return CreateResult(l.name, {**props, **l.raw_data})
def update(self, id, _olds, props):
l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
l.edit(name=props["name"],
color=props["color"],
description=props.get("description", GithubObject.NotSet))
return UpdateResult({**props, **l.raw_data})
def delete(self, id, props):
l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
l.delete()
class GithubLabel(Resource):
name: Output[str]
color: Output[str]
url: Output[str]
description: Output[str]
def __init__(self, name, args: GithubLabelArgs, opts = None):
full_args = {'url':None, 'description':None, 'name':None, 'color':None, **vars(args)}
super().__init__(GithubLabelProvider(), name, full_args, opts)
label = GithubLabel("foo", GithubLabelArgs("lukehoban", "todo", "mylabel", "d94f0b"))
export("label_color", label.color)
export("label_url", label.url)
```
Fixes https://github.com/pulumi/pulumi/issues/2902.
2019-07-19 19:18:25 +02:00
|
|
|
// Tests dynamic provider in Python.
|
|
|
|
func TestDynamicPython(t *testing.T) {
|
|
|
|
var randomVal string
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("dynamic", "python"),
|
|
|
|
Dependencies: []string{
|
|
|
|
path.Join("..", "..", "sdk", "python", "env", "src"),
|
|
|
|
},
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stack integration.RuntimeValidationStackInfo) {
|
|
|
|
randomVal = stack.Outputs["random_val"].(string)
|
|
|
|
},
|
|
|
|
EditDirs: []integration.EditDir{{
|
|
|
|
Dir: "step1",
|
|
|
|
Additive: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stack integration.RuntimeValidationStackInfo) {
|
|
|
|
assert.Equal(t, randomVal, stack.Outputs["random_val"].(string))
|
|
|
|
},
|
|
|
|
}},
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2019-06-25 01:45:54 +02:00
|
|
|
func TestResourceWithSecretSerialization(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: "secret_outputs",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Quick: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
// The program exports two resources, one named `withSecret` who's prefix property should be secret
|
|
|
|
// and one named `withoutSecret` which should not. We serialize both of the these as POJO objects, so
|
|
|
|
// they appear as maps in the output.
|
|
|
|
withSecretProps, ok := stackInfo.Outputs["withSecret"].(map[string]interface{})
|
|
|
|
assert.Truef(t, ok, "POJO output was not serialized as a map")
|
|
|
|
|
|
|
|
withoutSecretProps, ok := stackInfo.Outputs["withoutSecret"].(map[string]interface{})
|
|
|
|
assert.Truef(t, ok, "POJO output was not serialized as a map")
|
|
|
|
|
|
|
|
// The secret prop should have been serialized as a secret
|
|
|
|
secretPropValue, ok := withSecretProps["prefix"].(map[string]interface{})
|
|
|
|
assert.Truef(t, ok, "secret output was not serialized as a secret")
|
|
|
|
assert.Equal(t, resource.SecretSig, secretPropValue[resource.SigKey].(string))
|
|
|
|
|
|
|
|
// And here, the prop was not set, it should just be a string value
|
|
|
|
_, isString := withoutSecretProps["prefix"].(string)
|
|
|
|
assert.Truef(t, isString, "non-secret output was not a string")
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
2019-08-03 01:12:16 +02:00
|
|
|
|
2019-08-13 23:57:26 +02:00
|
|
|
func TestStackReferenceSecrets(t *testing.T) {
|
|
|
|
owner := os.Getenv("PULUMI_TEST_OWNER")
|
|
|
|
if owner == "" {
|
|
|
|
t.Skipf("Skipping: PULUMI_TEST_OWNER is not set")
|
|
|
|
}
|
|
|
|
|
|
|
|
d := "stack_reference_secrets"
|
|
|
|
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: path.Join(d, "step1"),
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
Config: map[string]string{
|
|
|
|
"org": owner,
|
|
|
|
},
|
|
|
|
Quick: true,
|
|
|
|
EditDirs: []integration.EditDir{
|
|
|
|
{
|
|
|
|
Dir: path.Join(d, "step2"),
|
|
|
|
Additive: true,
|
|
|
|
ExpectNoChanges: true,
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
_, isString := stackInfo.Outputs["refNormal"].(string)
|
|
|
|
assert.Truef(t, isString, "referenced non-secret output was not a string")
|
|
|
|
|
|
|
|
secretPropValue, ok := stackInfo.Outputs["refSecret"].(map[string]interface{})
|
|
|
|
assert.Truef(t, ok, "secret output was not serialized as a secret")
|
|
|
|
assert.Equal(t, resource.SecretSig, secretPropValue[resource.SigKey].(string))
|
|
|
|
},
|
|
|
|
},
|
|
|
|
},
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
2019-08-03 01:12:16 +02:00
|
|
|
func TestCloudSecretProvider(t *testing.T) {
|
|
|
|
kmsKeyAlias := os.Getenv("PULUMI_TEST_KMS_KEY_ALIAS")
|
|
|
|
if kmsKeyAlias == "" {
|
|
|
|
t.Skipf("Skipping: PULUMI_TEST_KMS_KEY_ALIAS is not set")
|
|
|
|
}
|
|
|
|
|
2019-09-07 02:13:40 +02:00
|
|
|
testOptions := integration.ProgramTestOptions{
|
2019-08-03 01:12:16 +02:00
|
|
|
Dir: "cloud_secrets_provider",
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
SecretsProvider: fmt.Sprintf("awskms://alias/%s", kmsKeyAlias),
|
|
|
|
Secrets: map[string]string{
|
|
|
|
"mysecret": "THISISASECRET",
|
|
|
|
},
|
|
|
|
ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
|
|
|
|
secretsProvider := stackInfo.Deployment.SecretsProviders
|
|
|
|
assert.NotNil(t, secretsProvider)
|
|
|
|
assert.Equal(t, secretsProvider.Type, "cloud")
|
|
|
|
|
|
|
|
_, err := cloud.NewCloudSecretsManagerFromState(secretsProvider.State)
|
|
|
|
assert.NoError(t, err)
|
|
|
|
|
|
|
|
out, ok := stackInfo.Outputs["out"].(map[string]interface{})
|
|
|
|
assert.True(t, ok)
|
|
|
|
|
|
|
|
_, ok = out["ciphertext"]
|
|
|
|
assert.True(t, ok)
|
|
|
|
},
|
2019-09-07 02:13:40 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
localTestOptions := testOptions.With(integration.ProgramTestOptions{
|
|
|
|
CloudURL: "file://~",
|
2019-08-03 01:12:16 +02:00
|
|
|
})
|
2019-09-07 02:13:40 +02:00
|
|
|
|
|
|
|
// Run with default Pulumi service backend
|
|
|
|
t.Run("service", func(t *testing.T) { integration.ProgramTest(t, &testOptions) })
|
|
|
|
|
|
|
|
// Also run with local backend
|
|
|
|
t.Run("local", func(t *testing.T) { integration.ProgramTest(t, &localTestOptions) })
|
2019-08-03 01:12:16 +02:00
|
|
|
}
|
Propagate inputs to outputs during preview. (#3327)
These changes restore a more-correct version of the behavior that was
disabled with #3014. The original implementation of this behavior was
done in the SDKs, which do not have access to the complete inputs for a
resource (in particular, default values filled in by the provider during
`Check` are not exposed to the SDK). This lack of information meant that
the resolved output values could disagree with the typings present in
a provider SDK. Exacerbating this problem was the fact that unknown
values were dropped entirely, causing `undefined` values to appear in
unexpected places.
By doing this in the engine and allowing unknown values to be
represented in a first-class manner in the SDK, we can attack both of
these issues.
Although this behavior is not _strictly_ consistent with respect to the
resource model--in an update, a resource's output properties will come
from its provider and may differ from its input properties--this
behavior was present in the product for a fairly long time without
significant issues. In the future, we may be able to improve the
accuracy of resource outputs during a preview by allowing the provider
to dry-run CRUD operations and return partially-known values where
possible.
These changes also introduce new APIs in the Node and Python SDKs
that work with unknown values in a first-class fashion:
- A new parameter to the `apply` function that indicates that the
callback should be run even if the result of the apply contains
unknown values
- `containsUnknowns` and `isUnknown`, which return true if a value
either contains nested unknown values or is exactly an unknown value
- The `Unknown` type, which represents unknown values
The primary use case for these APIs is to allow nested, properties with
known values to be accessed via the lifted property accessor even when
the containing property is not fully know. A common example of this
pattern is the `metadata.name` property of a Kubernetes `Namespace`
object: while other properties of the `metadata` bag may be unknown,
`name` is often known. These APIs allow `ns.metadata.name` to return a
known value in this case.
In order to avoid exposing downlevel SDKs to unknown values--a change
which could break user code by exposing it to unexpected values--a
language SDK must indicate whether or not it supports first-class
unknown values as part of each `RegisterResourceRequest`.
These changes also allow us to avoid breaking user code with the new
behavior introduced by the prior commit.
Fixes #3190.
2019-11-11 21:09:34 +01:00
|
|
|
|
|
|
|
func TestPartialValuesNode(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("partial_values", "nodejs"),
|
|
|
|
Dependencies: []string{"@pulumi/pulumi"},
|
|
|
|
AllowEmptyPreviewChanges: true,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
|
|
|
|
func TestPartialValuesPython(t *testing.T) {
|
|
|
|
integration.ProgramTest(t, &integration.ProgramTestOptions{
|
|
|
|
Dir: filepath.Join("partial_values", "python"),
|
|
|
|
Dependencies: []string{
|
|
|
|
path.Join("..", "..", "sdk", "python", "env", "src"),
|
|
|
|
},
|
|
|
|
AllowEmptyPreviewChanges: true,
|
|
|
|
})
|
|
|
|
}
|