Commit graph

470 commits

Author SHA1 Message Date
Ian Wahbe 054b3f9edc
Iwahbe/7881/thread replace on chages through sdks (#7967)
* Thread replaceOnChanges through the Go SDK

* Add replaceOnChanges to the .NET SDK

* Update CHANGELOG_PENDING.md

* Fix null error

* Update CHANGELOG_PENDING.md

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-09-15 14:29:13 -07:00
Justin Van Patten ed4b53d3ae
Add monitor feature for output values (#7870) 2021-09-15 14:16:00 -07:00
Justin Van Patten 0c0684af5c
Initial support for (un)marshaling output values (#7861)
This change expands the definition of `resource.Output` in the Go SDK with additional information about the output, i.e. dependencies and secretness, and adds support in the core Go RPC code for (un)marshaling output values.

Output values are marshaled as special objects ala archives, assets, and resource refs and are unmarshaled as `resource.Output` values.

Subsequent PRs will add:
 - A monitor feature for output values, which will initially be disabled by default but available to turn on via an envvar
 - Support for (un)marshaling output values in each language SDKs
 - A way for providers to indicate support for receiving output values
 - E2E tests
 - Turn the monitor feature on by default (w/ env var to disable) (Note: the current plan is to initially scope this to only be used when marshaling inputs to a multi-language component)
2021-09-13 09:05:31 -07:00
Benjamin Schiborr 77867f9ce4
feat: Improve error messages for (un)marshalling (#7936)
Currently whenever an issue occurs in `UnmarshalProperties` and
`MarshalProperties` the offending property is hidden and very difficult
to track down.

This commit changes the behavior. For `Assets` and `Archives` the error
message now includes the URI and for other properties it includes the
key of the `PropertyMap`.
2021-09-10 13:18:08 -07:00
Anton Tayanovskyy 27b1404d9e
Fix lint (#7915)
* Fix lint

* Fix lint of pkg folder
2021-09-07 16:41:17 -04:00
Vivek Lakshmanan 233b396f29
Merge pull request #7877 from pulumi/vl/FixAuto
[auto/go] Fix stack config loading
2021-08-31 11:32:05 -07:00
Ian Wahbe 27fc39c00a Fix spelling of environment 2021-08-31 00:54:07 -07:00
Ian Wahbe 59f88030b7 Allow failure to cleanup 2021-08-31 00:25:50 -07:00
Ian Wahbe 703104e412 Skip failing test on windows 2021-08-30 22:53:16 -07:00
Ian Wahbe 70833d3a3c Allow windows to find commands without .exe 2021-08-30 21:36:03 -07:00
Vivek Lakshmanan 9435d9ae9a Update test 2021-08-30 21:26:09 -07:00
Vivek Lakshmanan 6162004df2 Fix stack config loading 2021-08-30 20:56:08 -07:00
Ian Wahbe 8150ef0880 Improve CI error messages
Failures unique to windows require a more informative error message to
diagnose.
2021-08-30 12:55:30 -07:00
Ian Wahbe 878ab50044 Merge branch 'master' into iwahbe/2715/add-pulumi-about-command 2021-08-27 04:50:50 -04:00
Pat Gavlin 98f73cf1ed
Start in on developer documentation. (#7839)
Developer documentation is written in Markdown and can be built into
HTML, PDF, etc. using Sphinx. Diagrams are written in PlantUML and
rendered as SVGs. All developer docs live in the `developer-docs` folder
under the root of the repository.
2021-08-25 15:18:13 -07:00
Anton Tayanovskyy b667ce5d4f
Tighten up auto test (#7830)
* Tighten up a test to not fail for wrong reasons, use python helpers

* Use local in-source Python SDK for the test
2021-08-24 14:48:09 -04:00
Ian Wahbe f75ddfc01d Refactor data into get and display components
This allows us to give the output in json.
2021-08-23 00:48:22 -07:00
Pat Gavlin 8f9a13a76f
[sdk/go] Fix a typo in marshaling. (#7768)
A few identically-typed variables got confused with the changes in #7737.
The confusion caused empty property values to be included in resources
that had any dependencies on other resources, which confused the
unmarshaling code for Go multi-language components. These changes fix
the typo and restore the original behavior, which is to omit empty
property values.

Co-authored-by: Emiliza Gutierrez <emiliza@pulumi.com>
2021-08-16 19:01:20 -05:00
Anton Tayanovskyy 5069a8fca8
Fixes 5642 for Go: allow DependsOn accepting inputs and outputs (#7584)
* Work in progress, first passing tests

* Test unknown dep prop similar to Python

* Test fixes

* Fix lint

* Nit fix

* CHANGELOG

* Add ResourceInputArray and simplify the API

* Adopt urnSet
2021-08-13 11:13:23 -04:00
Pat Gavlin 2d70324b56
[sdk/go] Transitive component dependencies. (#7737)
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.

Part of #7542.
2021-08-11 21:51:23 -05:00
Pat Gavlin 64696b42b8
[sdk/providers] Fix update previews (#7560)
Do not return the inputs as the state for update previews that use an
unconfigured provider. Returning the inputs as the state allows the
language SDKs to incorrectly treat unknown properties as known (because
we can't call `Update` on an unconfigured provider, we can't know which
properties are unknown). Users can re-enable the existing behavior by
setting the `PULUMI_LEGACY_PROVIDER_PREVIEW` environment variable to a
truthy value (e.g. `1`, `true`, etc.).

Most users will be unaffected by these changes. The most common programs
that may be affected are those that combine the creation of a managed
Kubernetes cluster with the deployment of applications to that cluster. These
programs generally need to configure a k8s provider instance by constructing
a kubeconfig from the output of the managed k8s cluster. Any changes to the
cluster that cause the kubeconfig to be unknown then cause the provider to
go unconfigured at runtime. Prior to these changes, resources managed by the
k8s provider would have some known outputs in this scenario, as the engine
would treat the resource's input values as its output values. After these changes,
the resource's outputs will be treated as unknown. The most frequent affect
that this has is that applies/stack outputs that depend on the outputs of
a k8s resource managed by a provider with an unknown kubeconfig will not
run/be displayed as `output`s during previews, respectively.

We might be able to improve on this by taking advantage of schema
information and filling in unknown values for properties that do not
exist in the inputs.

Fixes #7521.

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Luke Hoban <luke@pulumi.com>
2021-08-10 19:44:15 -05:00
Evan Boyle f4153770eb
improve error message when flag passed to -s is not fully qualified (#7708) 2021-08-04 12:55:56 -07:00
yarinm 013036064f
Add force flag for RemoveStack (#7523) 2021-08-02 12:54:46 -07:00
Vivek Lakshmanan 3b009c1266
Assertion fix in config key generation (#6902) 2021-08-02 12:22:09 -07:00
svangordon-fruit 29fa23d6d9
Accept git remotes with periods in hostname (#7386)
Fix `cloudSourceControlSSHRegex` so that it will match git remotes with periods or hyphens in the hostname. However, `azureSourceControlSSHRegex` _does_ match hostnames with a period. As a result, `TryGetVCSInfo` will treat these sorts of remotes as Azure source control remotes and drop the first group. This causes updates to have an incorrect `"vcs.kind"` field. For example, an update with a Git remote of `github.foo.acme.com` will have a `"vcs.kind"` field of `foo.acme.com`. This occurs if a user is using a self-hosted GH enterprise instance.
2021-08-01 10:47:44 -07:00
Anton Tayanovskyy 5788befcf4
Fixing WaitGroup related panics in the Go SDK (#7661)
* Add tests reproducing the panics

* Add v2 of the test

* Use tailor-made workGroup to mimic sync.WaitGroup

* Lint

* CHANGELOG
2021-07-29 12:39:28 -04:00
Anton Tayanovskyy ba70b3fdba
Attempt to avoid triple-building projects in the solution (#7638) 2021-07-28 20:31:11 -04:00
Anton Tayanovskyy 148df112d7
Revert flag parse ordering change (#7640)
* Revert flag parse ordering change

* Add a simple unit test

* CHANGELOG
2021-07-27 12:42:52 -04:00
Anton Tayanovskyy 3aa97a4b7d
Fanout build experiment (#7628)
* Experiment with gotestsum and test timings

* Fix to locating the helper script

* Fix the code for installing gotestsum

* Try alternative installation method

* Use go to compute test stats; Python fails parsing time values

* Try version without v

* Try with fixed gorelaser config

* Fix test time correlation

* Try a stable test stat sort finally

* Use more accurate test duration aggregation

* Include python and auto-api tests in the Go timing counts

* Bring back TESTPARALLELISM

* Fix test compilation

* Only top 100 slow tests

* Try to fracture build matrix to fan out tests

* Do not run Publish Test Results on unsuppored Mac

* Auto-create test-results-dir

* Fix new flaky test by polling for logs

* Try to move native tests to their own config

* Actually skip

* Do not fail on empty test-results folder

* Try again

* Try once more

* Integration test config is the crit path - make it smaller

* Squash underutilized test configurations

* Remove the test result summary box from PR - counts now incorrec

* Remove debugging step
2021-07-27 10:07:15 -04:00
Justin Van Patten e8bd8e5e1f
Rehydrate provider resources in Construct (#7624)
Previously, any provider resource passed to multi-lang components would be instantiated as a `DependencyProviderResource` inside `Construct`, which prevents the component from being able to easily access the provider's state as an instance of of the provider (e.g. `*aws.Provider`).

This change attempts to rehydrate the provider resource in the same way that resource references are rehydrated, if it's been registered, s.t. the specific provider resource type is instantiated with its state. Otherwise falling back to returning `DependencyProviderResource`.
2021-07-23 14:10:06 -07:00
Chae SeungWoo bd208df180
[sdk/go] Add stack output helpers for numeric types (#7410) 2021-07-22 15:48:47 -07:00
Evan Boyle a659b3565c
reenable concurrent update test (#7609) 2021-07-22 10:06:44 -07:00
Justin Van Patten 30311405ca
[sdk/go] Fix methods panic when marshaling self (#7604) 2021-07-22 05:28:46 -07:00
Pat Gavlin ece9f2fb30
[sdk/{go,dotnet] Unmarshal invalid assets. (#7579)
The two more strongly-typed Pulumi SDKs curently fail with an error
during unmarshaling when attempting to marshal a value that is not an
asset into an asset-typed location (e.g. an asset-typed resource
output property). While this behavior is reasonable on its face, it
gives rise to practical challenges when dealing with TF-provider-backed
resources that have asset-typed properties. When such a resource is
refreshed, the values of its asset-typed properties are replaced with
non-asset values, as the TF bridge can't currently create a resonable
stand-in asset value.

For example, consider an S3 bucket object:

```
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("my-bucket");
new aws.s3.BucketObject("my-object", {
    source: new pulumi.FileAsset("some/file"),
});
```

Upon creation, the value of the input property `source` will be a file
asset backed by the path `some/file`. The bridge will propagate this
value to the `source` output property; this propagation is safe because
the resource was just created and so the output property must have the
value that was passed by the program.

Now, let some actor apply out-of-band changes to the contents of the
bucket object s.t. the `source` property changes when the object is
refreshed. In that case, the `source` property will be a string value
which the bridge is unable to interpret as an asset. The next time the
Pulumi program is run, the Go or .NET SDK will attempt to deserialize
the string into an asset-typed property and will fail.

With these changes, the deserialization would not fail, and would
instead create an asset or archive value that will fail to marshal if
passed to another resource. Users can avoid these errors by not passing
asset or archive outputs to other resources/stack outputs.

These changes unblock users who are hitting
https://github.com/pulumi/pulumi-aws/issues/1521.
2021-07-21 13:40:36 -07:00
Sean St. Quentin 81f87b0c6a
Fix the Go Automation API's Target and Replace CLI options (#7426)
* Add missing Sprintf calls

* Fix target and replace options change

Co-authored-by: Komal <komal@pulumi.com>
2021-07-20 17:43:01 -07:00
Chris Smith 0f0b9dd272
Add ContinuationToken to ListStacksResponse (#7510) 2021-07-13 14:01:39 -07:00
Pat Gavlin a74ea205fc
Add a TODO. (#7511)
Just what it says on the tin. This code should be made more precise once
7434 is fixed.
2021-07-13 12:44:39 -07:00
James Nugent 4d136339b0
Fix marshaling prompt slices/maps with input elems (#7508)
This fixes #7504, and is the other half of the interim fix corresponding
to #7359. The case listed in #7509 no longer reproduces after this fix
is applied.
2021-07-13 12:14:55 -07:00
Anton Tayanovskyy f211969059
Reject unknowns in context.go when not in preview (#7457)
* Reject unknowns in context.go when not in preview

* Fix test compilation

* Lint
2021-07-09 14:30:35 -04:00
Anton Tayanovskyy f1d9095374
Add a helper record to name fields in a very long tuple (#7458)
* Add a helper record to name fields in a very long tuple

* Trim the comment
2021-07-08 13:34:28 -04:00
Justin Van Patten eaf78edfef
[sdk/go] Support for calling methods (#7437) 2021-07-07 17:28:21 -07:00
Anton Tayanovskyy 36d11261c6
Quick fix for provider flag ordering (#7412)
* specify plugin arguments before other flags

* Ignore these pesky flags in python provider.main

Co-authored-by: evanboyle <evan@pulumi.com>
2021-07-07 11:07:04 -04:00
Justin Van Patten 824fb397f9
[sdk/go] Support for implementing methods in provider (#7379) 2021-07-01 15:46:16 -07:00
Luke Hoban eb32039013
Add replaceOnChanges resource option (#7226)
Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.

Fixes #6753.

Co-authored-by: Levi Blackstone <levi@pulumi.com>
2021-07-01 13:32:08 -06:00
Justin Van Patten 84b574f0df
Initial support for resource methods (authoring from Node.js, calling from Python) (#7363)
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
2021-06-30 07:48:56 -07:00
Pat Gavlin 398fb2f852
Preliminary fix for #7359. (#7369)
These changes contain a preliminary fix for #7359 in the Go SDK. The fix
handles input values that are nested one level deep within maps and
arrays, but does not handle other cases of nested input types.
2021-06-28 16:04:21 -07:00
Evan Boyle 634e97cd55
Include config secret info in Construct calls (#7358) 2021-06-24 15:38:01 -07:00
Levi Blackstone 98d9430486
[sdk/go] Add Contains method to PropertyPath (#7354) 2021-06-24 08:49:09 -06:00
Pat Gavlin 2cc89defbc
Read passphrase from the terminal when rotating. (#7347)
Rotating a passphrase requires that the old passphrase is available via
one of the `PULUMI_CONFIG_PASSPHRASE` or `PULUMI_CONFIG_PASSPHRASE_FILE`
environment variables. This confuses `readPassphrase` when reading a new
passphrase, since that function checks the aforementioned environment
variables prior to reading from the console. The overall effect is that
it is impossible to rotate the passphrase for a stack using the
passphrase provider. These changes fix this by always reading from the
console when rotating a passphrase.
2021-06-22 11:13:57 -07:00
James Nugent d4180d9f61
Print actionable information on assertion failure (#7344)
This commit moves from simply asserting on the assignability of types to
using `Assertf` to print the types in question. This provides more
information to a user whose code is panicking because of
non-assignability.

Ultimately it would likely be better to surface this via error messages
instead of via panic, but this at least improves the debuggability in
the meantime.
2021-06-21 11:30:08 -06:00