Commit graph

224 commits

Author SHA1 Message Date
Justin Van Patten 066507cd00
Do not specify a specific version of Python on Windows (#6316)
We install the version of Python we need in the CI environment (e.g. 3.9.x).
2021-02-11 08:22:04 -08:00
Paul Stack 3b8c3f7e61
Add an option to be able to stack export showing secrets for use in ExtraRuntimeValidation (#6094)
Fixes: #2993

```
func TestAccMyExample(t *testing.T) {
	test := getBaseOptions().
		With(integration.ProgramTestOptions{
			Dir: filepath.Join(getCwd(t), "test-dir"),
			DecryptSecretsInOutput: true,
			ExtraRuntimeValidation: func(t *testing.T, stackInfo integration.RuntimeValidationStackInfo) {
			  // manipulate the output of the stack here
			}),
		})

	integration.ProgramTest(t, &test)
}
```
2021-01-19 19:18:45 +00:00
Levi Blackstone 709fcbad51
Document Go packages (#6009)
Co-authored-by: Pat Gavlin <pat@pulumi.com>
2021-01-11 11:07:59 -07:00
Vivek Lakshmanan 221105a072 Ensure stderr is updated on error even if verbose is not set 2020-11-18 00:56:26 -08:00
Vivek Lakshmanan 302f869dc5 Drop unnecessary prefix in test output since we rely on go's testing to do the right thing 2020-11-17 23:24:08 -08:00
Vivek Lakshmanan 84d1505807 Switch to using t.Logf instead of printf 2020-11-17 23:09:18 -08:00
Vivek Lakshmanan d12f59be46 Reduce verbosity on tests 2020-11-17 23:09:18 -08:00
Justin Van Patten 9bcf02e7ed
Support remote components in Python (#5375) 2020-09-30 14:09:20 -07:00
Lee Briggs f5a54cf6bb
add more tests 2020-08-31 19:01:23 -07:00
Lee Briggs ed94132080
make function name clearer 2020-08-31 18:57:05 -07:00
Lee Briggs 0459767ceb
handle cases with no subpackages 2020-08-31 18:54:14 -07:00
Lee Briggs 6c2f7994fa
refactor and add tests 2020-08-31 16:08:15 -07:00
Lee Briggs 759019a7ad
use string index to calc package name 2020-08-31 15:36:02 -07:00
Lee Briggs 40745c126c
improve go mod replace logic
this commit updates the way we do the go mod replace during tests
2020-08-31 13:45:10 -07:00
Lee Briggs bad67d3242
switch os/user with luser
We make several calls to `os/user`, which uses CGO and means
cross-compilation is not possible. This replaces `os/user` with the
`luser` package, which is a drop-in replacement which does not use `CGO`
2020-07-27 14:44:08 -07:00
Pat Gavlin 3e6e28af36
Fix lint errors. (#4856)
These errors appear when using golangci-lint 1.27.0.
2020-06-18 14:17:11 -07:00
Justin Van Patten b77ec919d4
Install and use dependencies automatically for new Python projects (#4775)
Automatically create a virtual environment and install dependencies in it with `pulumi new` and `pulumi policy new` for Python templates.

This will save a new `virtualenv` runtime option in `Pulumi.yaml` (`PulumiPolicy.yaml` for policy packs):

```yaml
runtime:
  name: python
  options:
    virtualenv: venv
```

`virtualenv` is the path to a virtual environment that Pulumi will use when running `python` commands.

Existing projects are unaffected and can opt-in to using this by setting `virtualenv`, otherwise, they'll continue to work as-is.
2020-06-09 16:42:53 -07:00
stack72 e701e24151 Ensure Go build is an optional step in integration testing framework
Fixes: #4585

This was happening by default in the test framework and then again
in the go language runtime

Also we were replacing a module and then running go mod tidy - this
is unnecessary - go mod download is sufficient here to ensure we
have the correct dependency we need. The tidy may have been removing
the replacements added
2020-05-07 09:23:05 +01:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls (#3750)

* Switch away from native grpc impl. (#3728)

* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)

* Only retry as long as we get unavailable back.  Anything else continues. (#3769)

* Handle all errors for now. (#3781)


* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes (#4288)

Adjust C# generation

* Replace IDeployment with a sealed class (#4318)

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty (#4320)

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen (#4379)
2020-04-14 09:30:25 +01:00
Evan Boyle 80f1989600
Removing the need for vendoring (#4167) 2020-03-25 15:57:46 -07:00
evanboyle ec686bbaf6 Merge branch 'master' of https://github.com/pulumi/pulumi into evan/gomod 2020-03-18 17:57:58 -07:00
evanboyle d3f5bbce48 go fmt 2020-03-18 17:27:02 -07:00
Levi Blackstone 5a730d6ebc
Use Go's file handling libs for integration test framework (#4107)
Rather than shelling out to the `rm` and `cp` commands,
use Go's os utils to perform these operations. This will
work on any platform rather than just Linux.
2020-03-18 17:57:50 -06:00
evanboyle 6933745342 move pkg/tools -> sdk/go/common/tools 2020-03-18 15:56:58 -07:00
evanboyle a4ec3ec81b move pkg/testing -> sdk/go/common/testing, leave behind pkg/testing/integration 2020-03-18 15:55:41 -07:00
evanboyle 30e1a5917e move pkg/util/fsutil -> sdk/go/common/util/fsutil 2020-03-18 15:48:38 -07:00
evanboyle 33e23bdf0c move pkg/util/retry -> sdk/go/common/util/retry 2020-03-18 15:47:01 -07:00
evanboyle fa348ceb1b move pkg/util/ciutil -> sdk/go/common/util/ciutil 2020-03-18 15:43:31 -07:00
evanboyle c1d3a8524b move pkg/util/cmdutil -> sdk/go/common/util/cmdutil 2020-03-18 15:39:00 -07:00
evanboyle f754b486b8 move pkg/resource/config -> sdk/go/common/resource/config 2020-03-18 15:03:37 -07:00
evanboyle 67cb405c93 move pkg/apitype -> sdk/common/apitype 2020-03-18 15:00:30 -07:00
evanboyle 70f386a967 move pkg/tokens -> sdk/go/common/tokens 2020-03-18 14:49:56 -07:00
evanboyle fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
evanboyle 8fb3f428b0 move pkg/workspace -> sdk/go/common/workspace 2020-03-18 14:35:53 -07:00
evanboyle fba783caf9 move pkg/resource -> sdk/go/common/resource, but leave nested resource packages 2020-03-18 13:36:19 -07:00
Luke Hoban 2050f64322
Support "link" style dependencies in Go tests (#4063)
The integration test framework currently supports using `dep` for dependency management.

However, `dep` has no native ability to manage "yarn link"-style dependencies on locally available packages.

This is a necessary scenario for testing in most repos though, as (e.g.) examples in the Kubernetes repo need to test against the locally available version of `pulumi-kubernetes`.

The best we can do is a trick of (a) deleting the vendored copy of the locally available dependency (b) copying the locally available dependency into the vendor folder (c) deleting the nested vendor folder in the new copy of the locally available dependency.
2020-03-12 08:49:52 -07:00
stack72 f11b7129a0 Run Dep Ensure when testing Go integration projects
Fixes: #3939
2020-03-10 03:11:44 +02:00
Lee Zen f6402882c2
Regression tests for StackReference in the Python SDK (#3913)
* Make Python StackReference test similar to others (with two steps)
* Include new Python StackReference integration test that uses multiple stacks
* Expose various life cycle methods for ProgramTester
2020-02-17 10:40:46 -08:00
CyrusNajmabadi 060450395c
Breaking out individual testing knobs. (#3894) 2020-02-07 12:36:23 -08:00
Luke Hoban 5bf490228a
Use \n for provider protocol even on Windows (#3855)
The provider plugin protocol is to write a port number followed by `\n`.  We must guarantee we do that even on Windows, so must avoid Python `print` statements which implicitly rewrite newlines to platform specific character sequences.

Fixes #3807.
2020-02-06 11:34:05 -08:00
CyrusNajmabadi 2d117e6acf
Fix integration test harness. (#3831) 2020-01-29 11:57:45 -08:00
stack72 81d271d9ed Changing build.proj to run all languages and tests on windows 2020-01-27 21:16:37 +02:00
Mikhail Shilkov ed6060e0c9
Relax the pattern to search for local NuGet packages (#3595) 2019-12-03 08:22:32 +01:00
Pat Gavlin a600d16526
Retry failed update steps in integration tests. (#3542)
Add support for a test option that indicates that failed update steps
should be retried. Currently the maximum number of retries (3) is not
configurable.
2019-11-20 12:52:57 -08:00
Luke Hoban f9085bf799
Properly support Dependencies in .NET integration tests (#3527)
Allow any .NET pacakge dependency to be provided instead of hardcoding `Pulumi`.
2019-11-19 12:01:29 -08:00
Evan Boyle 5ae4149af5
Add support for "go run" style execution (#3503) 2019-11-14 09:25:55 -08:00
Justin Van Patten c08714ffb4
Support lists and maps in config (#3342)
This change adds support for lists and maps in config. We now allow
lists/maps (and nested structures) in `Pulumi.<stack>.yaml` (or
`Pulumi.<stack>.json`; yes, we currently support that).

For example:

```yaml
config:
  proj:blah:
  - a
  - b
  - c
  proj:hello: world
  proj:outer:
    inner: value
  proj:servers:
  - port: 80
```

While such structures could be specified in the `.yaml` file manually,
we support setting values in maps/lists from the command line.

As always, you can specify single values with:

```shell
$ pulumi config set hello world
```

Which results in the following YAML:

```yaml
proj:hello world
```

And single value secrets via:

```shell
$ pulumi config set --secret token shhh
```

Which results in the following YAML:

```yaml
proj:token:
  secure: v1:VZAhuroR69FkEPTk:isKafsoZVMWA9pQayGzbWNynww==
```

Values in a list can be set from the command line using the new
`--path` flag, which indicates the config key contains a path to a
property in a map or list:

```shell
$ pulumi config set --path names[0] a
$ pulumi config set --path names[1] b
$ pulumi config set --path names[2] c
```

Which results in:

```yaml
proj:names
- a
- b
- c
```

Values can be obtained similarly:

```shell
$ pulumi config get --path names[1]
b
```

Or setting values in a map:

```shell
$ pulumi config set --path outer.inner value
```

Which results in:

```yaml
proj:outer:
  inner: value
```

Of course, setting values in nested structures is supported:

```shell
$ pulumi config set --path servers[0].port 80
```

Which results in:

```yaml
proj:servers:
- port: 80
```

If you want to include a period in the name of a property, it can be
specified as:

```
$ pulumi config set --path 'nested["foo.bar"]' baz
```

Which results in:

```yaml
proj:nested:
  foo.bar: baz
```

Examples of valid paths:

- root
- root.nested
- 'root["nested"]'
- root.double.nest
- 'root["double"].nest'
- 'root["double"]["nest"]'
- root.array[0]
- root.array[100]
- root.array[0].nested
- root.array[0][1].nested
- root.nested.array[0].double[1]
- 'root["key with \"escaped\" quotes"]'
- 'root["key with a ."]'
- '["root key with \"escaped\" quotes"].nested'
- '["root key with a ."][100]'

Note: paths that contain quotes can be surrounded by single quotes.

When setting values with `--path`, if the value is `"false"` or
`"true"`, it will be saved as the boolean value, and if it is
convertible to an integer, it will be saved as an integer.

Secure values are supported in lists/maps as well:

```shell
$ pulumi config set --path --secret tokens[0] shh
```

Will result in:

```yaml
proj:tokens:
- secure: v1:wpZRCe36sFg1RxwG:WzPeQrCn4n+m4Ks8ps15MxvFXg==
```

Note: maps of length 1 with a key of “secure” and string value are
reserved for storing secret values. Attempting to create such a value
manually will result in an error:

```shell
$ pulumi config set --path parent.secure foo
error: "secure" key in maps of length 1 are reserved
```

**Accessing config values from the command line with JSON**

```shell
$ pulumi config --json
```

Will output:

```json
{
  "proj:hello": {
    "value": "world",
    "secret": false,
    "object": false
  },
  "proj:names": {
    "value": "[\"a\",\"b\",\"c\"]",
    "secret": false,
    "object": true,
    "objectValue": [
      "a",
      "b",
      "c"
    ]
  },
  "proj:nested": {
    "value": "{\"foo.bar\":\"baz\"}",
    "secret": false,
    "object": true,
    "objectValue": {
      "foo.bar": "baz"
    }
  },
  "proj:outer": {
    "value": "{\"inner\":\"value\"}",
    "secret": false,
    "object": true,
    "objectValue": {
      "inner": "value"
    }
  },
  "proj:servers": {
    "value": "[{\"port\":80}]",
    "secret": false,
    "object": true,
    "objectValue": [
      {
        "port": 80
      }
    ]
  },
  "proj:token": {
    "secret": true,
    "object": false
  },
  "proj:tokens": {
    "secret": true,
    "object": true
  }
}
```

If the value is a map or list, `"object"` will be `true`. `"value"` will
contain the object as serialized JSON and a new `"objectValue"` property
will be available containing the value of the object.

If the object contains any secret values, `"secret"` will be `true`, and
just like with scalar values, the value will not be outputted unless
`--show-secrets` is specified.

**Accessing config values from Pulumi programs**

Map/list values are available to Pulumi programs as serialized JSON, so
the existing
`getObject`/`requireObject`/`getSecretObject`/`requireSecretObject`
functions can be used to retrieve such values, e.g.:

```typescript
import * as pulumi from "@pulumi/pulumi";

interface Server {
    port: number;
}

const config = new pulumi.Config();

const names = config.requireObject<string[]>("names");
for (const n of names) {
    console.log(n);
}

const servers = config.requireObject<Server[]>("servers");
for (const s of servers) {
    console.log(s.port);
}
```
2019-11-01 13:41:27 -07:00
CyrusNajmabadi df06b8fc9b
Add publishing to nuget support (#3416) 2019-10-29 20:14:49 -07:00
CyrusNajmabadi 394c91d7f6
Add **preview** .NET Core support for pulumi. (#3399) 2019-10-25 16:59:50 -07:00
Mikhail Shilkov 90e111ba73 Override all options in ProgramTest.With (#3264) 2019-10-07 14:11:36 +03:00