Commit graph

24 commits

Author SHA1 Message Date
Pat Gavlin 2dcf3806bd
[automation-api] Exclude tests from test_fast. (#7986)
The Automation API tests take long enough that thay don't really fit
into `test_fast`. These changes move those tests to `test_all`.
2021-09-16 17:33:33 -07:00
Anton Tayanovskyy ba70b3fdba
Attempt to avoid triple-building projects in the solution (#7638) 2021-07-28 20:31:11 -04:00
Anton Tayanovskyy 3aa97a4b7d
Fanout build experiment (#7628)
* Experiment with gotestsum and test timings

* Fix to locating the helper script

* Fix the code for installing gotestsum

* Try alternative installation method

* Use go to compute test stats; Python fails parsing time values

* Try version without v

* Try with fixed gorelaser config

* Fix test time correlation

* Try a stable test stat sort finally

* Use more accurate test duration aggregation

* Include python and auto-api tests in the Go timing counts

* Bring back TESTPARALLELISM

* Fix test compilation

* Only top 100 slow tests

* Try to fracture build matrix to fan out tests

* Do not run Publish Test Results on unsuppored Mac

* Auto-create test-results-dir

* Fix new flaky test by polling for logs

* Try to move native tests to their own config

* Actually skip

* Do not fail on empty test-results folder

* Try again

* Try once more

* Integration test config is the crit path - make it smaller

* Squash underutilized test configurations

* Remove the test result summary box from PR - counts now incorrec

* Remove debugging step
2021-07-27 10:07:15 -04:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Paul Stack 3fad2e5329 Removing x namespace from go/python/nodejs automation packages (#6518) 2021-04-14 19:32:18 +01:00
Paul Stack 1d3c9edb6c
Ensure that make brew works as expected rather than passing empty version (#6566)
Fixes:#6565

As part of #6460, the logic for determing the version of the build was
moved to be a dependency on pulumictl.

Unfortunately, the homebrew installs use the "make dist" command to
build + install Pulumi to the user maching and as that would have a
dependency on pulumictl and it not existing on the user machine, it
would pass an empty version to the ldflag

This then manifested to the user as:

```
▶ pulumi version
warning: A new version of Pulumi is available. To upgrade from version '0.0.0' to '2.22.0', run
   $ brew upgrade pulumi
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
```

We are able to mitigate this behaviour by bringing back the get-version
script and using that script as part of the make brew installation

We can see that the versions are the same between the 2 different
installation techniques

```
make dist <------- uses pulumict
DIST:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=2.24.0-alpha.1616029310+787eb70a" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
DIST:
BUILD:
```

```
make brew <----- uses the legacy script
▶ make brew
BREW:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=v2.24.0-alpha.1616029310+g787eb70a2" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
BREW:
```

A full post mortem will be carried out to ensure we mitigate these
types of errors going forward and that we are able to better test
these types of situations
2021-03-18 02:07:02 +00:00
Paul Stack c48ba37fcf
Migrate the version calculation to use pulumictl (#6460) 2021-03-10 19:03:29 +00:00
evanboyle ec98494e5a add auto tests to Makefile and fix up for CI 2020-08-23 21:36:21 -07:00
Lee Briggs 8c314ec39a
Add brew make targets to sdk makefiles 2020-05-13 20:42:01 -07:00
Pat Gavlin 9b1b9cca24
Makefile: properly encode dependencies (#4541)
This allows for the use of `make -j`, which speeds up a full build
dramatically at the cost of rather incomprehensible logs.
2020-05-04 14:26:52 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls (#3750)

* Switch away from native grpc impl. (#3728)

* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)

* Only retry as long as we get unavailable back.  Anything else continues. (#3769)

* Handle all errors for now. (#3781)


* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes (#4288)

Adjust C# generation

* Replace IDeployment with a sealed class (#4318)

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty (#4320)

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen (#4379)
2020-04-14 09:30:25 +01:00
evanboyle fa47d867fd fix go sdk makefile to run tests 2020-03-19 14:10:48 -07:00
evanboyle bc4e5bf31c run common tests as a part of go sdk make 2020-03-18 17:32:27 -07:00
evanboyle 67f6b72caa Revert "move sdk/go/pulumi-language-go -> pkg/cmd/pulumi-language-go"
This reverts commit 7ea547a14d.
2020-03-18 17:14:41 -07:00
evanboyle 30df499838 move pkg/version -> sdk/go/common/version 2020-03-18 15:25:25 -07:00
evanboyle 7ea547a14d move sdk/go/pulumi-language-go -> pkg/cmd/pulumi-language-go 2020-03-18 12:49:56 -07:00
Pat Gavlin f168bdc1c2
Redesign the Go SDK resource/input/output system. (#3506)
The redesign is focused around providing better static typings and
improved ease-of-use for the Go SDK. Most of the redesign revolves
around three pivots:
- Strongly-typed inputs, especially for nested types
- Struct-based resource and invoke APIs
- Ease-of-use of Apply

1. Strongly-typed inputs

Input is the type of a generic input value for a Pulumi resource.
This type is used in conjunction with Output to provide polymorphism
over strongly-typed input values.

The intended pattern for nested Pulumi value types is to define an
input interface and a plain, input, and output variant of the value
type that implement the input interface.

For example, given a nested Pulumi value type with the following shape:

```
type Nested struct {
    Foo int
    Bar string
}
```

We would define the following:

```
var nestedType = reflect.TypeOf((*Nested)(nil)).Elem()

type NestedInput interface {
    pulumi.Input

    ToNestedOutput() NestedOutput
    ToNestedOutputWithContext(context.Context) NestedOutput
}

type Nested struct {
    Foo int `pulumi:"foo"`
    Bar string `pulumi:"bar"`
}

type NestedInputValue struct {
    Foo pulumi.IntInput `pulumi:"foo"`
    Bar pulumi.StringInput `pulumi:"bar"`
}

func (NestedInputValue) ElementType() reflect.Type {
    return nestedType
}

func (v NestedInputValue) ToNestedOutput() NestedOutput {
    return pulumi.ToOutput(v).(NestedOutput)
}

func (v NestedInputValue) ToNestedOutputWithContext(ctx context.Context) NestedOutput {
    return pulumi.ToOutputWithContext(ctx, v).(NestedOutput)
}

type NestedOutput struct { *pulumi.OutputState }

func (NestedOutput) ElementType() reflect.Type {
    return nestedType
}

func (o NestedOutput) ToNestedOutput() NestedOutput {
    return o
}

func (o NestedOutput) ToNestedOutputWithContext(ctx context.Context) NestedOutput {
    return o
}

func (o NestedOutput) Foo() pulumi.IntOutput {
    return o.Apply(func (v Nested) int {
        return v.Foo
    }).(pulumi.IntOutput)
}

func (o NestedOutput) Bar() pulumi.StringOutput {
    return o.Apply(func (v Nested) string {
        return v.Bar
    }).(pulumi.StringOutput)
}
```

The SDK provides input and output types for primitives, arrays, and
maps.

2. Struct-based APIs

Instead of providing expected output properties in the input map passed
to {Read,Register}Resource and returning the outputs as a map, the user
now passes a pointer to a struct that implements one of the Resource
interfaces and has appropriately typed and tagged fields that represent
its output properties.

For example, given a custom resource with an int-typed output "foo" and
a string-typed output "bar", we would define the following
CustomResource type:

```
type MyResource struct {
    pulumi.CustomResourceState

    Foo pulumi.IntOutput    `pulumi:"foo"`
    Bar pulumi.StringOutput `pulumi:"bar"`
}
```

And invoke RegisterResource like so:

```
var resource MyResource
err := ctx.RegisterResource(tok, name, props, &resource, opts...)
```

Invoke arguments and results are also provided via structs, but use
plain-old Go types for their fields:

```
type MyInvokeArgs struct {
    Foo int `pulumi:"foo"`
}

type MyInvokeResult struct {
    Bar string `pulumi:"bar"`
}

var result MyInvokeResult
err := ctx.Invoke(tok, MyInvokeArgs{Foo: 42}, &result, opts...)
```

3. Ease-of-use of Apply

All `Apply` methods now accept an interface{} as the callback type.
The provided callback value must have one of the following signatures:

	func (v T) U
	func (v T) (U, error)
	func (ctx context.Context, v T) U
	func (ctx context.Context, v T) (U, error)

T must be assignable from the ElementType of the Output. If U is a type
that has a registered Output type, the result of the Apply will be the
corresponding Output type. Otherwise, the result of the Apply will be
AnyOutput.

Fixes https://github.com/pulumi/pulumi/issues/2149.
Fixes https://github.com/pulumi/pulumi/issues/3488.
Fixes https://github.com/pulumi/pulumi/issues/3487.
Fixes https://github.com/pulumi/pulumi-aws/issues/248.
Fixes https://github.com/pulumi/pulumi/issues/3492.
Fixes https://github.com/pulumi/pulumi/issues/3491.
Fixes https://github.com/pulumi/pulumi/issues/3562.
2020-01-18 10:08:37 -05:00
CyrusNajmabadi df06b8fc9b
Add publishing to nuget support (#3416) 2019-10-29 20:14:49 -07:00
Matt Ellis 6c7092ff65 Pass -count=1 to disable result caching on go 1.10 and above 2018-11-08 14:11:52 -08:00
Matt Ellis 992b048dbf Adopt golangci-lint and address issues
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
2018-11-08 14:11:47 -08:00
joeduffy 162157c1a7 Add a Dockerfile for the Pulumi CLI
This introduces a Dockerfile for the Pulumi CLI. This makes it
easier to develop and test the engine in a self-contained environment,
in addition to being suitable for running the actual CLI itself.

For instance,

    $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up

will run the Pulumi program mounted under the /app volume. This will
be used in some upcoming CI/CD scenarios.

This uses multi-stage builds, and Debian Stretch as the base, for
relatively fast and lean build times and resulting images. We are
intentional about restoring dep packages independent of the actual
source code so that we don't end up needlessly re-depping, which can
consume quite a bit of time. After fixing
https://github.com/pulumi/pulumi/issues/1986, we should explore an
Alpine base image option.

I made the decision to keep this image scoped to just the Go builds.
Therefore, none of the actual SDK packages themselves are built, just
the engine, CLI, and language plugins for Node.js, Python, and Go.
It's possible to create a mega-container that has all of these full
environments so that we can rebuild them too, but for now I figured
it was better to rely on package management for them.

Another alternative would have been to install released binaries,
rather than building them. To keep the useful flow for development,
however, I decided to go the build route for now. If we build at the
same hashes, the resulting binaries "should" be ~identical anyhow.

I've created a pulumi/pulumi Docker Hub repo that we can publish this
into. For now, there is no CI publishing of the image.

This fixes pulumi/pulumi#1991.
2018-09-29 11:48:21 -07:00
Thomas Schersach 62463ab3bc Added dist target for make, will help with Homebrew (#1731)
* Added dist target for make, will help with Homebrew

* Try to install go dependencies before building

* Make sure dep ensure is called before trying to build SDKs

* Removed dep ensure from dist initial step
2018-08-08 13:00:42 -07:00
joeduffy cf1cb2d61f Get lint clean 2018-06-09 17:23:12 -07:00
joeduffy 5a71ab9d12 Add Makefile machinery for Go provider 2018-06-09 16:16:35 -07:00