Commit graph

1706 commits

Author SHA1 Message Date
Komal 9a30f36c93
[auto/python] - Fix deserialization of event (#8375)
Fixes the deserialization for the `metadata` property on `ResourcePreEvent`, `ResOutputsEvent` and `ResOpFailedEvent`.
2021-11-08 13:05:04 -08:00
Anton Tayanovskyy e37892ac4a
Fix 8322 (#8339)
* Fix 8322

* Untabify

* Untabify again

* Yet More untabify

* More untabify

* Final untabify

* Add CHANGELOG_PENDING

* Apply suggestions from code review

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* PR feedback

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-11-08 10:45:26 -05:00
Anton Tayanovskyy 99fdad0ed9
Fix python dependencies issues in GHA (#8361) 2021-11-06 00:55:17 -07:00
Fraser Waters b58c39476f
Fix cmdutil.PrintTable to handle ansi escapes and non-byte glyphs (#8344)
Fixes two bugs in how padding was calculated in PrintTable.

Firstly we remove all ANSI escape codes from the string before measuring
how wide it is. Secondly we measure glyph count (using rivo/uniseg) not
byte or rune count of the string.

Together these fix the padding/alignment issues I saw when using
PrintTable with plan output. They also slightly change the layout of
"pulumi stack", for example the below is printed with current master and
has 6 characters of space for padding between SecurityGroup and
web-secgrp:

```
Current stack resources (4):
    TYPE                                        NAME
    pulumi:pulumi:Stack                         aws-cs-webserver-test
    ├─ aws:ec2/securityGroup:SecurityGroup      web-secgrp
    ├─ aws:ec2/instance:Instance                web-server-www
    └─ pulumi:providers:aws                     default_4_25_0
```

While printed with this commit you only get 2 characters of space for
padding (which is correct, the column gap is set to "  "):
```
Current stack resources (4):
    TYPE                                    NAME
    pulumi:pulumi:Stack                     aws-cs-webserver-test
    ├─ aws:ec2/securityGroup:SecurityGroup  web-secgrp
    ├─ aws:ec2/instance:Instance            web-server-www
    └─ pulumi:providers:aws                 default_4_25_0
```
2021-11-04 10:06:20 +00:00
Justin Van Patten f989e1cc9d
Update pkg -> sdk dependency (#8346) 2021-11-03 11:25:20 -07:00
Emiliza Gutierrez d243efae19
Revert "[python/sdk] - Remove python 3.6 support (#8161)" (#8332)
This reverts commit 895ae970ac.
2021-11-02 10:19:01 -07:00
Ian Wahbe e629bc32d5
Expand dependencies when marshaling output values (#8301)
* Expand dependencies when marshaling output values

[sdk/python]

* Update CHANGELOG_PENDING.md

* Use existing code

* Fix lint
2021-11-01 11:10:27 -07:00
Fraser Waters d39a14432f
Don't throw on type mismatches in the dotnet sdk (#8286)
* Don't throw on type mismatches in the dotnet sdk

Fixes #7329

The converter will no longer throw if resource providers return data
that does not match the expected type declared in the dotnet sdk.
Instead a warning will be logged for the resource and the value will be
set to `default(T)`.
2021-10-29 17:35:17 +01:00
Fraser Waters e71fd81fe8
Type inference for out var differs by compiler (#8312)
While our CI seems happy with this line of code on master, my machine
with the 3.1 sdk is resolving it to non-nullable type and complaining
about the "= null" line later.

```
LocalWorkspace.cs(389,27): error CS8600: Converting null literal or
possible null value to non-nullable type.
[/workspaces/pulumi/sdk/dotnet/Pulumi.Automation/Pulumi.Automation.csproj]
```

Quick fix to just not use "var" here.
2021-10-29 16:29:45 +01:00
Ian Wahbe e38876f7af
[sdk/go] Respect default parent in go aliases. (#8288)
* Respect default parent in go aliases.

* Update changelog

* Handle empty parrents correctly

* Allow specifying no parent

* clarify variable name

* Improve `Unparent` ergonomics

* Take t0yv0's suggestion

* Adopt @t0yv0's tests and doc comments.

* Make NoParent,Parent,ParentURN mutually exclusive
2021-10-28 17:25:31 -07:00
Ian Wahbe 83e24765f3
.NET & python SDKs parity for bad pulumi versions (#8297)
* .NET & python SDKs parity for bad pulumi versions

They handle invalid Pulumi CLI version gracefully.

* Make python version property lazy

* Clarify .NET logic

* Add python test for validate_pulumi_version

* Add tests for invalid versions

* Fix python test

* Fix typo

* Fix tests

* Have _validate_pulumi_version handle parsing

* Modify python and .NET to parseAndValidate

* Modify typescript and go to parseAndValidate

* fix name
2021-10-27 20:54:23 -07:00
Komal 895ae970ac
[python/sdk] - Remove python 3.6 support (#8161) 2021-10-27 10:35:46 -07:00
Justin Van Patten fbf62399ca
[sdk/go] Fix regression marshaling assets/archives (#8290)
This change fixes a regression marshaling assets/archives that was introduced after adding support for marshaling output values.

For example, when setting an `Asset` on a field typed as `AssetOrAchiveInput`, in `marshalInput`, the input was being converted into an `AssetOrArchiveOutput` via the `ToAssetOrArchiveOutputWithContext`. Awaiting the output returns an `*asset` struct, which is itself an `AssetInput`, which causes infinite recursion when passed recursively to `marshalInput`.

The fix is to skip the `Input` checks in recursive calls, which is equivalent to the previous behavior before the regression was introduced.

Another issue was that when the input is converted to an output, this would result in `marshalInput` always returning an output value, even if the original passed-in value was not an output. To address this, if the output's value is known, not a secret, and has no dependencies, we can return the value itself rather than wrapping it as an output.
2021-10-27 08:44:42 -07:00
Ian Wahbe c8e117a37d
Handle invalid Pulumi CLI version (#8289)
* Handle invalid Pulumi CLI version

* Add clarifying comments and improve error message.

* Update go version of `validatePulumiVersion`
2021-10-26 16:20:45 -07:00
Fraser Waters a199ba8bb5
Fix race condition in TaskMonitoringHelper (#8294)
* Fix race condition in TaskMonitoringHelper

Fixes #8163

TaskMonitoringHelper was using two seperate trackers for Idle and
FirstException and then calling WhenAny on both to see which state
happened first. This was racy as you could end up completing a task with
an exception but getting the idle tracker fire first, resulting in
TaskMonitoringHelper thinking no exception had happened.

I've combined the two trackers into TaskMonitoringHelper now. At each
task completion we check for exceptions and then idleness.

* Add changelog
2021-10-26 22:37:47 +01:00
Carlos Tadeu Panato Junior 1c0a7e4907
reformat error message string (#8284)
Signed-off-by: Carlos Panato <ctadeu@gmail.com>
2021-10-26 14:55:25 -06:00
Fraser Waters b39c81e4cb
Improve error message and add test for #1440 (#8268)
* Improve error message and add test for #1440

* lint

* lint

* Use %w instead of errors.Wrap
2021-10-26 16:55:52 +01:00
Liam White 6e3e17ef28
Enable workaround for Yarn workspaces for inline functions (#8215)
See https://github.com/pulumi/pulumi/issues/2661\#issuecomment-939531284 for context.

Bascially, yarn workspaces rely on symlinks for monorepo-local packages.

I think this also resolves https://github.com/pulumi/pulumi/issues/2980

Signed-off-by: Liam White <liam@tetrate.io>
2021-10-25 10:13:09 -07:00
Josh Studt caf5fcd525
[auto/dotnet] - pulumi state delete & unprotect functionality (#8202)
Co-authored-by: Komal <komal@pulumi.com>
2021-10-18 16:26:45 -07:00
Ian Wahbe e07d96af9e
Pin grpc version (#8214) 2021-10-13 13:44:55 -07:00
iamjekyun 1bb85ca98c
Only prune the trailing spaces in Python automation result (#8160) 2021-10-11 10:26:22 -07:00
Ian Wahbe 20287afdd7
Iwahbe/8000/uprgade go versions (#8171)
* Upgrade the pulumi cli to go1.17.1

* Run go mod tidy

* Update CHANGELOG_PENDING.md

* Update pkg/go.mod for go1.17

* Update sdk and tests
2021-10-11 09:47:08 -07:00
Komal a9a62bd761
Determine secretness by checking raw string for the secret sentinel (#8179) 2021-10-08 14:43:10 -07:00
Anton Tayanovskyy 32695f7022
Minimally extend .NET SDK to support 5758 (#8142)
* Minimally extend .NET SDK to support 5758

* Retract CodegenUtilities

* Add CHANGELOG_PENDING
2021-10-07 09:51:24 -04:00
Komal b6c6108edd
[sdk/python] - Python 3.10 compatibility (#8130) 2021-10-05 11:43:52 -07:00
alexander-minchin 43f73f2cc1
Fix typo in Output.apply docstring (#8133) 2021-10-04 18:06:43 -07:00
Jeremy Dartigalongue 25f396bf49
Fix Go automation API --target / --replace args (#8109) 2021-10-04 11:55:23 -07:00
Anton Tayanovskyy 836b410911
Disable ConcurrentUpdateError test (#7953)
* Better logging from ConcurrentUpdateError test

* Disable the test
2021-10-04 13:31:31 -04:00
Pat Gavlin 4eec3b7e1b
[sdk/python] Add a smoke test for output deps (#8113)
Ensure that output property dependencies are deserialized and translated
properly.

Fixes #7444.

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-10-02 09:06:11 -07:00
Praneet Loke 8f89ab6e3a
[codegen/docs] Generate a package tree that can be serialized as JSON (#8102)
* Add an initialize func. Add a new func that generates the package tree
2021-09-30 17:35:44 -07:00
Pat Gavlin 5f9d211cae
[testing] Stack context for value generation (#8108)
These changes add a context type to `resource/testing` that can be used to
generate property values that are valid with respect to certain rules. The
context represents a Pulumi stack, and contains a project name, stack name,
and a list of resources.

- URNs generated using a context will always use the context's project
  and stack name
- Resource references generated using a context will always refer to
  resources in the context's resource list
- Output values generated using a context will always pull dependencies
  from the context's resource list
2021-09-30 14:39:09 -07:00
Paul Stack 09a8cc7079
[cli] Add the ability to control auto-refresh of stacks by Pulumi.yaml (#8071)
Co-authored-by: Komal Ali <komal@pulumi.com>
2021-09-29 12:43:48 +03:00
Ian Wahbe 9df8d3a028
Bold in-progress diffs diffrently (#7918)
* Initial take on coloring in-progress diffs

* Update CHANGELOG_PENDING.md

* Add high and low color levels

* Don't print functions

* Change colorprogress to bold

* Fix @komalali's nits.

* Restore background colors comment
2021-09-28 15:16:09 -07:00
Justin Van Patten ea44ae3d0f
[sdk/go] Fix unmarshaling known output values in provider (#8082)
`ConstructInputs.CopyTo` can error when falling back to the old marshaling behavior when it's lacking an input type registration for a _known_ output value. In such cases, it tries to unmarshal a known output value, it can't unmarshal it because of a missing switch case. This change adds the missing case.
2021-09-27 18:46:30 -07:00
Justin Van Patten 9deb5ca0ec
[sdk/go] Marshal output values (#7958)
This change adds support for marshaling outputs as output values in the Go SDK.
2021-09-27 09:01:40 -07:00
Justin Van Patten 3027d01f25
Enable output values by default (#8014)
* Enable output values by default

Enable output values by default in the resource monitor and change the polarity of the envvar from `PULUMI_ENABLE_OUTPUT_VALUES` to `PULUMI_DISABLE_OUTPUT_VALUES`.

* Marshal unknown as unknown string when `!KeepOutputValues`

Marshal all unknown output values as `resource.MakeComputed(resource.NewStringProperty(""))` when not keeping output values, which is consistent with what the SDKs do.

Otherwise, when `v.OutputValue().Element` is nil, `resource.MakeComputed(v.OutputValue().Element)` will be marshaled as a null value rather than as an unknown sentinel.

* Add MarshalOptions.DontSkipOutputs and use where needed

Before we expanded the meaning of `resource.Output`, `MarshalProperties` always skipped output values:

```go
if v.IsOutput() {
    logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```

As part of expanding the meaning of `resource.Output`, I'd adjusted `MarshalProperties` to only skip output values when the value was unknown and when not keeping output values:

```go
if v.IsOutput() && !v.OutputValue().Known && !opts.KeepOutputValues {
    logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```

However, this doesn't work the way we want when marshaling properties that include unknown output values to a provider that does not accept outputs. In that case, `opts.KeepOutputValues` will be `false` because we want the marshaler to fall back to returning non-output-values (e.g. unknown sentinel value for unknown output values), but instead of getting the intended fallback values, the unknown output values are skipped (not what we want).

I suspect we may be able to delete the output value skipping in `MarshalProperties` altogether (it's odd that it is skipping `resource.Output` but not `resource.Computed`), but to avoid any unintended side effects of doing that, instead, this commit introduces a new `MarshalOptions.DontSkipOutputs` option that can be set to `true` to opt-in to not skipping output values when marshaling. The check in `MarshalProperties` now looks like this:

```go
if !opts.DontSkipOutputs && v.IsOutput() && !v.OutputValue().Known {
    logging.V(9).Infof("Skipping output property for RPC[%s]: %v", opts.Label, key)
}
```

`opts.DontSkipOutputs` is set to `true` when marshaling properties for calls to a provider's `Construct` and `Call`.

* [sdk/nodejs] Deserialize output values

This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK.

* [sdk/python] Deserialize output values

This commit adds support for deserializing output values, which is needed in some cases when serialized inputs are returned as outputs in the SDK.
2021-09-24 08:57:04 -07:00
Justin Van Patten 1e09626bc7
[sdk/{nodejs,python}] Fix errors when testing remote components with mocks (#8053)
v3.13.0 introduces support for serializing outputs in inputs as special output value objects in the Node.js and Python SDKs when serializing inputs for remote components and method calls. This functionality is currently disabled by default in the engine (setting the `PULUMI_ENABLE_OUTPUT_VALUES` envvar to a truthy value enables it).

However, unit testing remote components with mocks results in errors being raised in v3.13.0, related to the new output value support. This is due to the mock monitor implementation saying it supports all features (which now includes output values), so the SDK serializers are serializing outputs as output values, which the mock monitor can't handle correctly.

This change addresses the issue by updating the mock monitor implementation in the Node.js and Python SDKs to indicate the specific features that are supported, excluding support for output values. New tests with mocks fail before the change and pass after.
2021-09-24 06:08:13 -07:00
Komal 212ac89366
[auto/python] - Fix a bug in printing stack. (#8032) 2021-09-22 16:49:03 -07:00
Pat Gavlin 134d7cb818
[apitype] Add a JSON schema for deployments. (#8002)
This schema can be used to validate the contents of a Pulumi deployment.
If a deployment validates against this schema, it should be considered
syntactically valid, though it may contain certain classes of semantic
errors (e.g. references to unknown resources in dependency lists,
dependency cycles, etc.).

This schema is not yet used for validation in practice and may not be
entirely accurate.

These changes also add this schema (and the schemas on which it depends)
to the developer documentation. jsonschema2md.go has been updated to
support multi-file schemas.
2021-09-21 21:37:06 -07:00
Komal 3d42198991
Clean up diagnostic messages in event log (#7998)
* Don't escape characters in event log.

* Respect NO_COLOR
2021-09-21 17:22:39 -07:00
Pat Gavlin cbdecf2cd5
[testing] Add rapid generators for PropertyValues. (#8009)
And use those generators to test property value serialization and
deserialization paths.
2021-09-21 15:02:10 -07:00
Ian Wahbe 2e5fedff54
Switch from golint to revive (#8010) 2021-09-21 10:00:44 -07:00
Justin Van Patten df50c4492e
[sdk/python] Fix serializing output values within dicts (#7996)
Pass along the `keep_output_values` param when serializing values within dicts.
2021-09-17 20:53:29 -07:00
Justin Van Patten d812d16329
[sdk/go] Unmarshal output values in component provider (#7975)
This adds support for unmarshaling output values in the CopyTo helper used when copying inputs to an args struct
2021-09-17 19:46:06 -07:00
Komal 452e3f76f1
Fix python lint issues (#7988) 2021-09-16 17:42:35 -07:00
Pat Gavlin 2dcf3806bd
[automation-api] Exclude tests from test_fast. (#7986)
The Automation API tests take long enough that thay don't really fit
into `test_fast`. These changes move those tests to `test_all`.
2021-09-16 17:33:33 -07:00
Justin Van Patten 4f3f366695
[sdk/python] Marshal output values (#7926)
This change adds support for marshaling outputs as output values in the Python SDK.
2021-09-15 21:49:23 -07:00
Justin Van Patten 1a4f36e97b
[sdk/go] Add RegisterInputType and register built-in types (#7928)
This change adds a `RegisterInputType` function (similar to the existing `RegisterOutputType`) that is used to register an input interface's type and its associated input type, and adds registrations for the built-in input types.

This will be used when copying inputs to an args struct for multi-lang components. When a field is typed as the input interface (e.g. `StringMapInput`) and doesn't need to be an `Output`, we can use this to lookup the non-Output type that implements the interface (e.g. `StringMap`) so it can be instantiated.

A subsequent change will update the Go SDK codegen to produce input type registrations for a provider's input types.
2021-09-15 21:12:49 -07:00
Justin Van Patten 0b9e41e8c4
[sdk/nodejs] Marshal output values (#7925)
This change adds support for marshaling outputs as output values in the Node.js SDK.
2021-09-15 18:25:26 -07:00
Ian Wahbe 054b3f9edc
Iwahbe/7881/thread replace on chages through sdks (#7967)
* Thread replaceOnChanges through the Go SDK

* Add replaceOnChanges to the .NET SDK

* Update CHANGELOG_PENDING.md

* Fix null error

* Update CHANGELOG_PENDING.md

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-09-15 14:29:13 -07:00