Commit graph

406 commits

Author SHA1 Message Date
Ian Wahbe d7ee074a68 Include package host version in about 2021-08-20 19:38:51 -07:00
Jan Češpivo e67334db1d
Added support for custom naming of dynamic provider resource (#7633)
Now there is not possible to change a name of dynamic provider resource without copying a code of the `pulumi.sdk.python.lib.pulumi.dynamic.dynamic.Resource` and changing the hard-coded name `"pulumi-python:dynamic:Resource"`. I successfully uses this proposal to make it possible.

Usage:
```python
class CustomResource(
    Resource, name="my-custom-provider:CustomResource"
):
   ...
```

Co-authored-by: Pat Gavlin <pgavlin@gmail.com>
2021-08-17 14:15:53 -07:00
Anton Tayanovskyy 4d4642c23d
Remove threading from mock tests to fix AIO waits on diff event loop (#7666)
* Remove threading from mock tests to fix AIO waits on diff event loop

* Add reference to the proper fix
2021-08-16 18:53:15 -04:00
Anton Tayanovskyy cd885bded5
Re-introduce the fix for 7734 and mitigate CI hangs (#7746)
* Revert the revert

* Fix exception bleed from one test suite to another

* Fix typos
2021-08-13 11:07:13 -04:00
Luke Hoban ebb0e6aaed
Allow Python dynamic provider resources to be constructed outside of __main__ (#7755)
The underlying library `dill` that we use for serializing dynamic providers into Pulumi state for Python dynamic providers serializes classes differently depending on whether they are in `__main__` or in another module.  We need the by-value serialization to be applied in all cases.

https://github.com/uqfoundation/dill/issues/424 is tracking adding the ability into `dill` to specify this by-value serialization explicitly, but until then, we will temporarily re-write the `__module__` of a provder class prior to serialization, so that `dill` behaves as we need for the dynamic provider use case.

Fixes #7453.
2021-08-12 20:02:17 -07:00
Pat Gavlin ecb98b66fd
[sdk/python] Transitive component dependencies. (#7732)
Implement Node/.NET-style dependency semantics for component resources.
Depending on a component implicitly depends on all of the component's
children. The exact set of children depends on exactly when the
component resource is observed.

Part of #7542.
2021-08-11 21:52:16 -05:00
Anton Tayanovskyy ea333681ab
Revert "Fix hangs in core SDK when monitor unavailable (#7734)" (#7744)
This reverts commit e567b4762f.
2021-08-11 17:13:37 -04:00
Justin Van Patten 0b782ea884
[sdk/python] Fix pulumi.property's default value handling (#7736)
We were incorrectly setting the name of the attribute rather than the intended default value.
2021-08-11 09:15:46 -07:00
Anton Tayanovskyy e567b4762f
Fix hangs in core SDK when monitor unavailable (#7734)
* Fix hangs in core SDK when monitor unavailable

* merged
2021-08-11 10:50:27 -04:00
Nikhil Benesch 442cdf5743
python: Use Sequence rather than List in Resource fields (#7700)
Similar to #5282, but for core SDK types. The tl;dr is that because
Sequence[T] is covariant, constructing resources becomes much more
ergonomic.

Fix #7693.
2021-08-03 14:03:42 -07:00
Anton Tayanovskyy 13b6cc5eea
Fix reflecting list type annotations (#7681)
* Fix reflecting list type annotations

* Fix typo and add a test case

* Typo
2021-07-30 09:35:16 -04:00
Justin Van Patten 74ab9e0869
[sdk/python] Handle unknown results from methods (#7677)
This fixes handling of unknown results from methods in the Python SDK. Node.js and Go were already handling this appropriately.
2021-07-29 11:11:52 -07:00
Anton Tayanovskyy ba70b3fdba
Attempt to avoid triple-building projects in the solution (#7638) 2021-07-28 20:31:11 -04:00
Anton Tayanovskyy bc96ac799d
Inputty depends_on support for Python (#7559)
* Inputty depends_on support for Python

* Fix circular import

* Add a test, fix bugs found by test

* Fix typo

* Minimize loss of dep information in presence of unks to match Node behavior

* Add CHANGELOG entr

* Update sdk/python/lib/pulumi/output.py

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Update sdk/python/lib/test/test_resource.py

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Fix _is_prompt

* Enhance tests, found out a gap against Node impl

* Add non-listy case

* Internal functional combinators

* Do not use all/deep await when merging

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-07-28 09:49:07 -04:00
Anton Tayanovskyy 3aa97a4b7d
Fanout build experiment (#7628)
* Experiment with gotestsum and test timings

* Fix to locating the helper script

* Fix the code for installing gotestsum

* Try alternative installation method

* Use go to compute test stats; Python fails parsing time values

* Try version without v

* Try with fixed gorelaser config

* Fix test time correlation

* Try a stable test stat sort finally

* Use more accurate test duration aggregation

* Include python and auto-api tests in the Go timing counts

* Bring back TESTPARALLELISM

* Fix test compilation

* Only top 100 slow tests

* Try to fracture build matrix to fan out tests

* Do not run Publish Test Results on unsuppored Mac

* Auto-create test-results-dir

* Fix new flaky test by polling for logs

* Try to move native tests to their own config

* Actually skip

* Do not fail on empty test-results folder

* Try again

* Try once more

* Integration test config is the crit path - make it smaller

* Squash underutilized test configurations

* Remove the test result summary box from PR - counts now incorrec

* Remove debugging step
2021-07-27 10:07:15 -04:00
Justin Van Patten a05c3a4e9b
Set the package on DependencyProviderResource (#7630)
When initializing `DependencyProviderResource`, pass the package to the base constructor instead of an empty string s.t. the provider is usable when its package is read.
2021-07-27 06:50:24 -07:00
Justin Van Patten e8bd8e5e1f
Rehydrate provider resources in Construct (#7624)
Previously, any provider resource passed to multi-lang components would be instantiated as a `DependencyProviderResource` inside `Construct`, which prevents the component from being able to easily access the provider's state as an instance of of the provider (e.g. `*aws.Provider`).

This change attempts to rehydrate the provider resource in the same way that resource references are rehydrated, if it's been registered, s.t. the specific provider resource type is instantiated with its state. Otherwise falling back to returning `DependencyProviderResource`.
2021-07-23 14:10:06 -07:00
Justin Van Patten 68458bfab3
[sdk/python] Support for implementing methods in provider (#7555) 2021-07-19 14:58:55 -07:00
Nikhil Benesch c782a298a1
Make Output[T] covariant in Python SDK (#7483)
In short, this allows subtyping to correctly "propagate" through an Output;
if Cow is a subtype of Animal, this commit makes it so Output[Cow] is
treated as a subtype of Output[Animal].

Without this change, users of the Python SDK occasionally contend with a
confusing error message that is resolved by adding a call to
`.apply(lambda x: x)` to satisfy mypy.

Touches #3767.
Fix #6843.
2021-07-16 12:30:04 -07:00
Anton Tayanovskyy 36d11261c6
Quick fix for provider flag ordering (#7412)
* specify plugin arguments before other flags

* Ignore these pesky flags in python provider.main

Co-authored-by: evanboyle <evan@pulumi.com>
2021-07-07 11:07:04 -04:00
Pat Gavlin 713b901b0c
[sdk/python] Translate property dep keys. (#7443)
The map of property dependencies the Python SDK receives from the
engine uses `camelCase` property keys, but the SDK performs lookups
using `snake_case` property keys. Fix this by translating the map's keys
from `camelCase` to `snake_case` prior to performing any lookups.
2021-07-07 20:01:41 +10:00
Luke Hoban eb32039013
Add replaceOnChanges resource option (#7226)
Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.

Fixes #6753.

Co-authored-by: Levi Blackstone <levi@pulumi.com>
2021-07-01 13:32:08 -06:00
Justin Van Patten 84b574f0df
Initial support for resource methods (authoring from Node.js, calling from Python) (#7363)
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
2021-06-30 07:48:56 -07:00
Justin Van Patten e966c65f69
Fix pylint issues (#7389) 2021-06-29 21:35:25 -07:00
Evan Boyle 634e97cd55
Include config secret info in Construct calls (#7358) 2021-06-24 15:38:01 -07:00
Anton Tayanovskyy b16d50085b
Fix hanging deployments and improve errmsgs for programs with incorrect typings for output values (#7049)
* Do not hang but propagate exception when it happens in resolve_outputs

* Add an integration test for the issue

* Better error message

* Add CHANGELOG_PENDING entry

* Update sdk/python/lib/pulumi/runtime/rpc.py

Co-authored-by: Justin Van Patten <jvp@justinvp.com>

* Address PR feedback and tighten path param typing

* Given Windows builder is failing, allow 2x time for the test

* Give some more time to the Windows runner

* Attempt to solve differently

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-06-23 11:27:17 -04:00
Justin Van Patten 48bbc28d9e
[sdk/go] Specify known when creating outputs for construct (#7343)
If an input is an unknown value or contains unknowns, specify `false` for `known` when creating the output.
2021-06-21 09:51:41 -07:00
Luke Hoban 965d23ba2a
[sdk/python] Ensure Output objects are not iterable (#7288)
Although `Output` objects can never correct support iteration, Python will see the implementation of `__getitem__` and try to iterate the object, leading to an infinite loop.  To prevent this, we need to explicitly implement `__iter__` and make it return a `TypeError` to prevent iteration (and offer a useful error message).

Fixes #5028.
2021-06-15 09:25:24 +10:00
Justin Van Patten 63e129a386
[sdk/python] Reduce log.debug calls for improved performance (#7295)
`log.debug` messages are part of an update and get sent to the service. Reducing the number of `log.debug` calls can significantly improve the performance of a program. This commit changes the debug logging in the Python SDK to be similar to the debug logging in the Node.js SDK.

 - An `excessive_debug_output` variable has been added (that can be set to `True` during debugging), but otherwise will avoid a large number of `log.debug` calls.
- Some unhelpful/redundant `log.debug` calls have been deleted.
2021-06-14 10:24:01 -07:00
Paul Stack 57347188fb
[sdk/python] Fix linting errors due to missing types files (#7257) 2021-06-09 19:14:59 +03:00
Luke Hoban 26e252f241
Ensure Output.from_input({}) returns {} instead of [] (#7254)
Fixes #7252.
2021-06-09 19:48:10 +10:00
Luke Hoban bd6410e2fb
[sdk/python] Avoid exponential complexity for from_input/all (#7175)
These mutually recursive functions unintentionally had exponential complexity in nesting depth of objects, arg types and most likely arrays.

Remove the exponential complexity by avoiding direct recursion of from_input on itself, and relying on mutual recursion with all alone to reduce nested substructure.

Also simplify the implementation to aid readability.

Fixes pulumi/pulumi-kubernetes#1597.
Fixes pulumi/pulumi-kubernetes#1425.
Fixes pulumi/pulumi-kubernetes#1372.
Fixes #3987.
2021-06-01 13:11:22 +10:00
Justin Van Patten d6b7762102
Temporarily disable config secrets warning (#7129)
Temporarily disable the new config secret warning to avoid unactionable warnings from provider `config` modules. We'll re-enable the warning when we've addressed that issue.
2021-05-24 16:06:27 -07:00
Justin Van Patten f973fbcf58
[sdk/nodejs|python] Add GetSchema support to providers (#6892) 2021-05-19 07:11:18 -07:00
Justin Van Patten 480173a57f
[sdk/python] Warn when a secret config is read as a non-secret (#7078) 2021-05-18 15:00:30 -07:00
Justin Van Patten a61e79eb0d
[sdk/nodejs] Warn when a secret config is read as a non-secret (#6896) 2021-05-18 09:48:08 -07:00
Anton Tayanovskyy 493bac4c18
Make virtualenv paths relative to root when main points elsewhere (#6966)
* Propagate workspace.Project metadata to plugin init

* Get to a working fix

* Propagate Root via plugin context

* Propagate root instead of yaml path

* Revert out unnecessary parameter propagation

* Root is now always absolute at this point; simplify code and docs

* Drop python conditional and propagate unused -root to all lang hosts

* Add tests that fail before and pass after

* Lint

* Add changelog entry
2021-05-14 13:41:55 -04:00
Komal 17fae620af
Skip flaky python test (#7035) 2021-05-12 09:29:13 -07:00
Anton Tayanovskyy 2288f8b7fb
Fix issue with lazy_import affecting pulumi-eks (#7024)
* Fix issue with lazy_import in presense of modules-as-attrs

* Temporary inline _lazy_import into generated code for smooth updates

* Add CHANGELOG entry

* Rename _lazy_import to lazy_import

* Comments on short-circuit behavior

* Comment fix
2021-05-11 22:48:08 -04:00
Komal 3bac1d68b7
Move comment in python SDK (#7028) 2021-05-11 17:23:18 -07:00
Komal 3ef2648f45
Use test-org from env var (#7016) 2021-05-11 08:41:21 -07:00
Anton Tayanovskyy d55327db50
Fix tests that used to be skipped on missing async lib (#6941) 2021-05-10 13:27:20 -04:00
Komal 59992cff97
[automation/python] - Export ProjectBackend class (#6984) 2021-05-06 19:43:31 -07:00
Justin Van Patten 3746028c38
[sdk/python] Don't error when dict input value has a mismatched type (#6949)
Pulumi 3.0 raises an error when a dict value is passed as an input but the type annotation does not accept a dict. Unfortunately, this prevents historical cases where a dict value is allowed but the type annotation doesn't match. We need to fix the type annotations, but in the meantime, we should not raise an error in the SDK for such cases as it breaks existing programs.
2021-05-03 08:47:55 -07:00
Justin Van Patten 81db03ea2a
Fix Python test file names (#6947)
I'd run these tests individually locally, but they need to start with `test_` to be picked up by pytest.
2021-04-30 20:22:25 -07:00
Justin Van Patten 8574d40dda
[sdk/python] Fix type-related regression on Python 3.6 (#6942)
Pulumi 3.0 uses type annotations for input values to determine whether dict keys should be translated from snake_case to camelCase or not. This additional inspection of types did not work correctly on Python 3.6 due to some missing functionality on that version of Python which we need to provide an implementation for. Specifically, when inspecting `Union` args to determine whether or not a value is intended to be an input class or user-defined dict. To address the issue, this change improves how we get the args for `Union` types when running on Python 3.6 to behave the same way as later versions of Python (for our purposes). Existing tests fail on Python 3.6 before this change, and pass after.
2021-04-30 11:50:42 -07:00
Justin Van Patten 69d50ced27
[sdk/python] Address issues when using resource subclasses (#6890)
We were only looking at the current resource class's type/name metadata for camelCase <=> snake_case property name translations which prevented it from working correctly when using a subclass of a resource. This change addresses this by looking at metadata of the current class and any base classes.

Additionally, to help resolve forward references when getting type hints, we'd pass along the current resource class's globals, which doesn't work correctly when using a subclass of a resource. This change also addresses this, by using the globals of the current class and any base classes.
2021-04-30 10:07:23 -07:00
Komal 1ed3445ed4
[automation/*] - Optionally skip Automation API version check (#6882)
Co-authored-by: James Nugent <jen20@apple.com>
2021-04-27 20:54:27 -07:00
Ville Penttinen daa6045381
[automation/*] Add support for getting stack outputs using Workspace (#6859) 2021-04-26 16:32:30 -07:00
Komal 33f403e7c5
Fix python mock args (#6863) 2021-04-26 13:22:10 -07:00