Commit graph

1627 commits

Author SHA1 Message Date
Pat Gavlin 3a89fd8262 [sdk/go] retract v3.10.1
This version introduced a critical bug in resource argument marshaling
that caused empty properties to be marshaled as null values for any
resource that depends on other resources.
2021-08-16 18:25:08 -05:00
Anton Tayanovskyy 4d4642c23d
Remove threading from mock tests to fix AIO waits on diff event loop (#7666)
* Remove threading from mock tests to fix AIO waits on diff event loop

* Add reference to the proper fix
2021-08-16 18:53:15 -04:00
PND b5ee840b16
[sdk/nodejs] Prevent Pulumi from overriding tsconfig.json options. (#7068) 2021-08-15 18:58:43 -07:00
Anton Tayanovskyy 5069a8fca8
Fixes 5642 for Go: allow DependsOn accepting inputs and outputs (#7584)
* Work in progress, first passing tests

* Test unknown dep prop similar to Python

* Test fixes

* Fix lint

* Nit fix

* CHANGELOG

* Add ResourceInputArray and simplify the API

* Adopt urnSet
2021-08-13 11:13:23 -04:00
Anton Tayanovskyy cd885bded5
Re-introduce the fix for 7734 and mitigate CI hangs (#7746)
* Revert the revert

* Fix exception bleed from one test suite to another

* Fix typos
2021-08-13 11:07:13 -04:00
Luke Hoban ebb0e6aaed
Allow Python dynamic provider resources to be constructed outside of __main__ (#7755)
The underlying library `dill` that we use for serializing dynamic providers into Pulumi state for Python dynamic providers serializes classes differently depending on whether they are in `__main__` or in another module.  We need the by-value serialization to be applied in all cases.

https://github.com/uqfoundation/dill/issues/424 is tracking adding the ability into `dill` to specify this by-value serialization explicitly, but until then, we will temporarily re-write the `__module__` of a provder class prior to serialization, so that `dill` behaves as we need for the dynamic provider use case.

Fixes #7453.
2021-08-12 20:02:17 -07:00
Pat Gavlin ecb98b66fd
[sdk/python] Transitive component dependencies. (#7732)
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.

Part of #7542.
2021-08-11 21:52:16 -05:00
Pat Gavlin 2d70324b56
[sdk/go] Transitive component dependencies. (#7737)
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.

Part of #7542.
2021-08-11 21:51:23 -05:00
Anton Tayanovskyy ea333681ab
Revert "Fix hangs in core SDK when monitor unavailable (#7734)" (#7744)
This reverts commit e567b4762f.
2021-08-11 17:13:37 -04:00
Justin Van Patten 0b782ea884
[sdk/python] Fix pulumi.property's default value handling (#7736)
We were incorrectly setting the name of the attribute rather than the intended default value.
2021-08-11 09:15:46 -07:00
Anton Tayanovskyy e567b4762f
Fix hangs in core SDK when monitor unavailable (#7734)
* Fix hangs in core SDK when monitor unavailable

* merged
2021-08-11 10:50:27 -04:00
Pat Gavlin 64696b42b8
[sdk/providers] Fix update previews (#7560)
Do not return the inputs as the state for update previews that use an
unconfigured provider. Returning the inputs as the state allows the
language SDKs to incorrectly treat unknown properties as known (because
we can't call `Update` on an unconfigured provider, we can't know which
properties are unknown). Users can re-enable the existing behavior by
setting the `PULUMI_LEGACY_PROVIDER_PREVIEW` environment variable to a
truthy value (e.g. `1`, `true`, etc.).

Most users will be unaffected by these changes. The most common programs
that may be affected are those that combine the creation of a managed
Kubernetes cluster with the deployment of applications to that cluster. These
programs generally need to configure a k8s provider instance by constructing
a kubeconfig from the output of the managed k8s cluster. Any changes to the
cluster that cause the kubeconfig to be unknown then cause the provider to
go unconfigured at runtime. Prior to these changes, resources managed by the
k8s provider would have some known outputs in this scenario, as the engine
would treat the resource's input values as its output values. After these changes,
the resource's outputs will be treated as unknown. The most frequent affect
that this has is that applies/stack outputs that depend on the outputs of
a k8s resource managed by a provider with an unknown kubeconfig will not
run/be displayed as `output`s during previews, respectively.

We might be able to improve on this by taking advantage of schema
information and filling in unknown values for properties that do not
exist in the inputs.

Fixes #7521.

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Luke Hoban <luke@pulumi.com>
2021-08-10 19:44:15 -05:00
Horace Lee a92a005d68
Use ESlint instead of TSlint (#7719)
Migrated TSlint configs to ESlint ones using [tslint-to-eslint-config](https://github.com/typescript-eslint/tslint-to-eslint-config) tool, and refined the configs to better match the current coding style.

Changes:
- [member-delimiter-style](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/docs/rules/member-delimiter-style.md#options) that as suggested default for type definition to be  with `semicolon`
- Indentation fixes that is enforced by [eslint-indent](https://eslint.org/docs/rules/indent#options)
- Added dependencies for ESlint with Typescript
- Removed TSlint
2021-08-10 11:31:59 -07:00
Evan Boyle 7b7f03ea55
free prepResource in finally (#7707) 2021-08-04 13:19:43 -07:00
Evan Boyle f4153770eb
improve error message when flag passed to -s is not fully qualified (#7708) 2021-08-04 12:55:56 -07:00
Evan Boyle 8b918e544e
[sdk/nodejs] Account for outstanding async work done in prepareResource (#7704) 2021-08-04 10:27:38 -07:00
Nikhil Benesch 442cdf5743
python: Use Sequence rather than List in Resource fields (#7700)
Similar to #5282, but for core SDK types. The tl;dr is that because
Sequence[T] is covariant, constructing resources becomes much more
ergonomic.

Fix #7693.
2021-08-03 14:03:42 -07:00
yarinm 013036064f
Add force flag for RemoveStack (#7523) 2021-08-02 12:54:46 -07:00
Vivek Lakshmanan 3b009c1266
Assertion fix in config key generation (#6902) 2021-08-02 12:22:09 -07:00
jetvova ec5bafc3e2
Go install command is given proper DOTNET_VERSION (#7695) 2021-08-02 20:59:28 +03:00
svangordon-fruit 29fa23d6d9
Accept git remotes with periods in hostname (#7386)
Fix `cloudSourceControlSSHRegex` so that it will match git remotes with periods or hyphens in the hostname. However, `azureSourceControlSSHRegex` _does_ match hostnames with a period. As a result, `TryGetVCSInfo` will treat these sorts of remotes as Azure source control remotes and drop the first group. This causes updates to have an incorrect `"vcs.kind"` field. For example, an update with a Git remote of `github.foo.acme.com` will have a `"vcs.kind"` field of `foo.acme.com`. This occurs if a user is using a self-hosted GH enterprise instance.
2021-08-01 10:47:44 -07:00
Anton Tayanovskyy 13b6cc5eea
Fix reflecting list type annotations (#7681)
* Fix reflecting list type annotations

* Fix typo and add a test case

* Typo
2021-07-30 09:35:16 -04:00
Justin Van Patten 74ab9e0869
[sdk/python] Handle unknown results from methods (#7677)
This fixes handling of unknown results from methods in the Python SDK. Node.js and Go were already handling this appropriately.
2021-07-29 11:11:52 -07:00
Anton Tayanovskyy 5788befcf4
Fixing WaitGroup related panics in the Go SDK (#7661)
* Add tests reproducing the panics

* Add v2 of the test

* Use tailor-made workGroup to mimic sync.WaitGroup

* Lint

* CHANGELOG
2021-07-29 12:39:28 -04:00
Anton Tayanovskyy ba70b3fdba
Attempt to avoid triple-building projects in the solution (#7638) 2021-07-28 20:31:11 -04:00
Anton Tayanovskyy bc96ac799d
Inputty depends_on support for Python (#7559)
* Inputty depends_on support for Python

* Fix circular import

* Add a test, fix bugs found by test

* Fix typo

* Minimize loss of dep information in presence of unks to match Node behavior

* Add CHANGELOG entr

* Update sdk/python/lib/pulumi/output.py

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Update sdk/python/lib/test/test_resource.py

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Fix _is_prompt

* Enhance tests, found out a gap against Node impl

* Add non-listy case

* Internal functional combinators

* Do not use all/deep await when merging

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-07-28 09:49:07 -04:00
Anton Tayanovskyy 148df112d7
Revert flag parse ordering change (#7640)
* Revert flag parse ordering change

* Add a simple unit test

* CHANGELOG
2021-07-27 12:42:52 -04:00
Anton Tayanovskyy 3aa97a4b7d
Fanout build experiment (#7628)
* Experiment with gotestsum and test timings

* Fix to locating the helper script

* Fix the code for installing gotestsum

* Try alternative installation method

* Use go to compute test stats; Python fails parsing time values

* Try version without v

* Try with fixed gorelaser config

* Fix test time correlation

* Try a stable test stat sort finally

* Use more accurate test duration aggregation

* Include python and auto-api tests in the Go timing counts

* Bring back TESTPARALLELISM

* Fix test compilation

* Only top 100 slow tests

* Try to fracture build matrix to fan out tests

* Do not run Publish Test Results on unsuppored Mac

* Auto-create test-results-dir

* Fix new flaky test by polling for logs

* Try to move native tests to their own config

* Actually skip

* Do not fail on empty test-results folder

* Try again

* Try once more

* Integration test config is the crit path - make it smaller

* Squash underutilized test configurations

* Remove the test result summary box from PR - counts now incorrec

* Remove debugging step
2021-07-27 10:07:15 -04:00
Justin Van Patten a05c3a4e9b
Set the package on DependencyProviderResource (#7630)
When initializing `DependencyProviderResource`, pass the package to the base constructor instead of an empty string s.t. the provider is usable when its package is read.
2021-07-27 06:50:24 -07:00
Anton Tayanovskyy babedf5171
Ignore logflow args in Node providers (#7644)
* Ignore logflow args in Node providers

* Add unit test, handle --tracing

* Add missing files

* Add CHANGELOG

* Lint
2021-07-26 19:52:59 -04:00
Anton Tayanovskyy fbf44bfa39
Fix new flaky test by polling for logs (#7627) 2021-07-23 18:49:18 -04:00
Justin Van Patten e8bd8e5e1f
Rehydrate provider resources in Construct (#7624)
Previously, any provider resource passed to multi-lang components would be instantiated as a `DependencyProviderResource` inside `Construct`, which prevents the component from being able to easily access the provider's state as an instance of of the provider (e.g. `*aws.Provider`).

This change attempts to rehydrate the provider resource in the same way that resource references are rehydrated, if it's been registered, s.t. the specific provider resource type is instantiated with its state. Otherwise falling back to returning `DependencyProviderResource`.
2021-07-23 14:10:06 -07:00
Anton Tayanovskyy 697cf3161e
Timeout of 1min for the slow task (#7619) 2021-07-22 19:11:39 -04:00
Chae SeungWoo bd208df180
[sdk/go] Add stack output helpers for numeric types (#7410) 2021-07-22 15:48:47 -07:00
Evan Boyle a659b3565c
reenable concurrent update test (#7609) 2021-07-22 10:06:44 -07:00
Anton Tayanovskyy b0f51a6b2c
Fixes for C# concurrency bugs in 7492 (#7529)
* Reproduce the issue in a failing test

* Fix

* Tentative fix

* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Update sdk/dotnet/Pulumi/Deployment/Deployment.Runner.cs

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Do not allocate TaskCompletionSource when not needed

* Update sdk/dotnet/Pulumi/Deployment/Deployment.Runner.cs

Co-authored-by: Josh Studt <32800478+orionstudt@users.noreply.github.com>

* Fix warning

* Cache delegate

* Simplify with named tuples

* Test early exception termination

* Test logging

* Remove the smelly method of suppressing engine exceptions

* Update sdk/dotnet/Pulumi/Deployment/TaskMonitoringHelper.cs

Co-authored-by: Josh Studt <32800478+orionstudt@users.noreply.github.com>

* Fix typo; check in xml docs

* Try CI again

* Add CHANGELOG entry

* Dedup exceptions before reporting

* Lock access to _exceptions list

* Fix typos

* Version of HandleExceptionsAsync that accepts N exceptions

* Do not aggregate exceptions prematurely

* Rename private members

* Formatting

* Summary markers

* Short-circuit return

* Stylistic fixes

* Strengthen test

* Check that we have only 1 exception

* Remove defensive clause about AggregateException from the test

* Simplify TerminatesEarly test

* Remove EmptyStack

* Notes on the regression nature of the WorksUnderStress test

* Remove race condition repro as it is a poor repro, impossible to trigger from user code

* Brace style

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Josh Studt <32800478+orionstudt@users.noreply.github.com>
2021-07-22 12:49:14 -04:00
Justin Van Patten 30311405ca
[sdk/go] Fix methods panic when marshaling self (#7604) 2021-07-22 05:28:46 -07:00
Sean Fausett 12217bd0dc
Fix async await warnings (#7537)
* Revert remove redundant async await
* Fix resharper code issues
* Update changelog
2021-07-21 18:44:10 -04:00
Pat Gavlin ece9f2fb30
[sdk/{go,dotnet] Unmarshal invalid assets. (#7579)
The two more strongly-typed Pulumi SDKs curently fail with an error
during unmarshaling when attempting to marshal a value that is not an
asset into an asset-typed location (e.g. an asset-typed resource
output property). While this behavior is reasonable on its face, it
gives rise to practical challenges when dealing with TF-provider-backed
resources that have asset-typed properties. When such a resource is
refreshed, the values of its asset-typed properties are replaced with
non-asset values, as the TF bridge can't currently create a resonable
stand-in asset value.

For example, consider an S3 bucket object:

```
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("my-bucket");
new aws.s3.BucketObject("my-object", {
    source: new pulumi.FileAsset("some/file"),
});
```

Upon creation, the value of the input property `source` will be a file
asset backed by the path `some/file`. The bridge will propagate this
value to the `source` output property; this propagation is safe because
the resource was just created and so the output property must have the
value that was passed by the program.

Now, let some actor apply out-of-band changes to the contents of the
bucket object s.t. the `source` property changes when the object is
refreshed. In that case, the `source` property will be a string value
which the bridge is unable to interpret as an asset. The next time the
Pulumi program is run, the Go or .NET SDK will attempt to deserialize
the string into an asset-typed property and will fail.

With these changes, the deserialization would not fail, and would
instead create an asset or archive value that will fail to marshal if
passed to another resource. Users can avoid these errors by not passing
asset or archive outputs to other resources/stack outputs.

These changes unblock users who are hitting
https://github.com/pulumi/pulumi-aws/issues/1521.
2021-07-21 13:40:36 -07:00
Sean St. Quentin 81f87b0c6a
Fix the Go Automation API's Target and Replace CLI options (#7426)
* Add missing Sprintf calls

* Fix target and replace options change

Co-authored-by: Komal <komal@pulumi.com>
2021-07-20 17:43:01 -07:00
Hossam Barakat 3fabf45542
Set PULUMI_CONFIG_PASSPHRASE for Automation Tests (#7502) 2021-07-20 17:18:30 -07:00
Justin Van Patten 68458bfab3
[sdk/python] Support for implementing methods in provider (#7555) 2021-07-19 14:58:55 -07:00
Justin Van Patten 8c4a56acb7
[sdk/dotnet] Clean obj & bin dirs when building (#7562) 2021-07-19 11:22:08 -07:00
Luke Hoban 0bcca3883e
[sdk] Wait on remote component dependencies (#7541)
When a resource `dependsOn` a remote component, we were not correctly waiting on it, because we were skipping over waiting on comoponents, and only waiting on their custom resource children.  For remote components, we do not know the children, but waiting on the remote component will always wait on all children.

Co-authored-by: Mike Metral <1112768+metral@users.noreply.github.com>
2021-07-16 16:11:34 -07:00
Nikhil Benesch c782a298a1
Make Output[T] covariant in Python SDK (#7483)
In short, this allows subtyping to correctly "propagate" through an Output;
if Cow is a subtype of Animal, this commit makes it so Output[Cow] is
treated as a subtype of Output[Animal].

Without this change, users of the Python SDK occasionally contend with a
confusing error message that is resolved by adding a call to
`.apply(lambda x: x)` to satisfy mypy.

Touches #3767.
Fix #6843.
2021-07-16 12:30:04 -07:00
PND c6062ea1d5
[sdk/nodejs] Fix a bug in closure serialization. (#6999)
Co-authored-by: Komal <komal@pulumi.com>
Co-authored-by: Luke Hoban <luke@pulumi.com>
2021-07-15 18:20:09 -07:00
Chris Smith 0f0b9dd272
Add ContinuationToken to ListStacksResponse (#7510) 2021-07-13 14:01:39 -07:00
Pat Gavlin a74ea205fc
Add a TODO. (#7511)
Just what it says on the tin. This code should be made more precise once
7434 is fixed.
2021-07-13 12:44:39 -07:00
James Nugent 4d136339b0
Fix marshaling prompt slices/maps with input elems (#7508)
This fixes #7504, and is the other half of the interim fix corresponding
to #7359. The case listed in #7509 no longer reproduces after this fix
is applied.
2021-07-13 12:14:55 -07:00
Anton Tayanovskyy f211969059
Reject unknowns in context.go when not in preview (#7457)
* Reject unknowns in context.go when not in preview

* Fix test compilation

* Lint
2021-07-09 14:30:35 -04:00