Commit graph

195 commits

Author SHA1 Message Date
Justin Van Patten d0ba9fbdcd
[sdk/python] Add support for Sequence (#5282)
We currently emit array types as `List[T]` for Python, but `List[T]` is invariant, which causes type checkers like mypy to produce errors when values like `["foo", "bar"]` are passed as args typed as `List[pulumi.Input[str]]` (since `Input[str]` is an alias for `Union[T, Awaitable[T], Output[T]]`. To address this, we should move to using [`Sequence[T]`](https://docs.python.org/3/library/typing.html#typing.Sequence) which is covariant, and does not have this problem.

We actually already do this for `Dict` vs. `Mapping`, emitting map types as `Mapping[str, T]` rather than `Dict[str, T]` because `Mapping[str, T]` is covariant for the value. This change makes us consistent for array types.

These are the SDK changes necessary to support `Sequence[T]`.
2020-09-08 22:22:35 -07:00
Pat Gavlin 2585b86aa4
Initial support for remote component construction. (#5280)
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.

Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.

The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.

These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).

An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.

This is the core of #2430.
2020-09-07 19:33:55 -07:00
Justin Van Patten 69fbd70330
Python: Ignore internal properties when unmarshaling (#5251) 2020-08-28 18:26:16 -07:00
Justin Van Patten 16d226d667
Avoid raising unexpected type errors with Any (#5238)
Avoid raising an `AssertionError` due to unexpected types when a type is annotated as `Any`.
2020-08-27 12:19:46 -07:00
Justin Van Patten cd9fae599d
Python SDK changes to support input/output classes (#5033)
Python SDK changes to support strongly-typed input/output "dataclasses".
2020-08-19 01:15:56 -07:00
Luke Hoban 5066ae3227
Fix support for CheckFailures in Python Dynamic Providers (#5139)
Fixes #5138.
2020-08-10 09:28:25 -07:00
Komal f364311064
Remove known_types decorators (#4980) 2020-07-13 12:27:00 -07:00
Komal bdc86e002a
Change how submodules are imported to work with Intellisense (#4948) 2020-07-08 18:34:59 -07:00
Komal ed752bc384
[codegen/python] Don't use __all__ (#4873) 2020-06-24 11:41:03 -07:00
Justin Van Patten 6bad3a3620
Exclude grpcio v1.30.0, which breaks Pulumi Python programs (#4883) 2020-06-24 08:11:05 -07:00
Justin Van Patten 9b0169be35
Fix pylint(no-member) when accessing resource.id (#4813)
Pylint currently reports `E1101: Instance of 'Bucket' has no 'id' member (no-member)` on lines in Pulumi Python programs like:

```python
pulumi.export('bucket_name', bucket.id)
```

Here's a description of this message from http://pylint-messages.wikidot.com/messages:e1101:

> Used when an object (variable, function, …) is accessed for a non-existent member.
>
> False positives: This message may report object members that are created dynamically, but exist at the time they are accessed.

This appears to be a false positive case: `id` isn't set in the constructor (it's set later in `register_resource`) and Pylint isn't able to figure this out statically. `urn` has the same problem. (Oddly, Pylint doesn't complain when accessing other resource output properties).

This change refactors `register_resource` so that `id` and `urn` can be assigned in the resource's constructor, so that Pylint can see it being assigned. The change also does the same with `read_resource`.
2020-06-12 12:41:56 -07:00
Luke Hoban 678a5a6d3e
Export the CustomTimeouts class (#4747)
This class was available in the pulumi.resource module, but was not exported from the core `pulumi` module as intended for all public APIs at this level.
2020-06-03 15:36:44 -07:00
Justin Van Patten 6b0a845cc1
Support publishing Python policy packs (#4644)
Adds support for publishing Python policy packs to the service, and downloading/using such policy packs when applied to stacks in an organization.
2020-05-22 15:01:15 -07:00
Justin Van Patten af3d4b890d
Allow pulumi.export calls from unit tests (#4670)
The previous attempt to allow this didn't actually allow it, so this is
take two. As part of the previous attempt, I thought after tweaking the
test I had observed the test failing, and then succeeding after making
the product changes, but I must have been mistaken.

It turns out that our existing mocks tests weren't running at all
because of a missing `__init__.py` file. Once the missing `__init__.py`
is added, the tests run, but other tests ("test mode" tests) fail
because the code that creates the mocks and resources will run during
test discovery, and setting the mocks modifies global state.

To address the test issue, I've moved the mocks tests into their own
`test_with_mocks` package that can be run separately from other tests.

And addressed the original issue, by creating a root Stack resource if
one isn't already present when the mocks are set.
2020-05-20 09:54:40 -07:00
solidDoWant b7fb3bae22
Fixed ResourceOption deepcopy bug in stack_reference.py (#4553) 2020-05-08 12:01:59 -07:00
Justin Van Patten 16926504c1
Allow pulumi.export calls from unit tests (#4518)
This change allows importing modules with calls to `pulumi.export` in unit tests. Previously, you'd have to structure the Python program in a way that avoids the `pulumi.export` from being called from unit tests.
2020-04-29 10:03:20 -07:00
Komal c7866108e1
Merge branch 'master' into komalali/increase-grpc-message-size 2020-04-24 17:54:32 -07:00
Luke Hoban cfa57914a1
[sdk/python] Improve ResourceOptions.merge type (#4484)
* [sdk/python] Improve `ResoruceOptions.merge` type

The implementation correctly handles `None` inputs, so the type should allow these as well.

* Add CHANGELOG
2020-04-24 17:45:11 -07:00
komal 28d69e963e add tests for node and python 2020-04-23 11:16:36 -07:00
Komal Ali 9cf635ad5a pr changes 2020-04-23 11:16:36 -07:00
Komal Ali de41fecfdc fix python server options 2020-04-23 11:16:36 -07:00
Komal Ali 64f3c4a02a update sdks with new max message size 2020-04-23 11:16:36 -07:00
Mikhail Shilkov 6d32d575e0
Enable features in mock monitor (#4272) 2020-04-03 08:33:40 +02:00
Mikhail Shilkov 0bce094dc1
Fix python mock's call (#4274)
Fix python mock's call
2020-04-03 07:28:52 +02:00
Justin Van Patten dd104a00a7
Propagate secretness correctly in Python apply (#4273)
* Propagate secretness correctly in Python `apply`

* Improve `apply` test coverage

* Update CHANGELOG.md
2020-04-02 13:01:29 -07:00
Justin Van Patten e6be38e285
PaC: Add initial config support for policy packs (#4233)
The initial config represents any config that was specified programmatically to the Policy Pack, for Policy Packs that support programmatic configuration like AWSGuard.
2020-03-30 12:52:05 -07:00
Matthieu Berthomé 7c75d4fe5b
ephemeral kwargs for log methods (#4197)
Co-authored-by: rienafairefr <rienafairefr@gmail.com>
2020-03-26 14:23:43 -07:00
Luke Hoban 215f5af505
Clarify documentation on Output.from_input (#4178) 2020-03-25 15:28:22 -07:00
Justin Van Patten 24e804bbe8
Support for running Python policy packs (#4057)
These changes enable running policy packs written in Python.
2020-03-18 16:15:57 -07:00
Luke Hoban fdd9a57ff8
Implement GetSchema in dynamic providers (#4083)
It's not entirely clear why gRPC doesn't already report these cleanly as unimplemented, but for now we'll explicitly implement them to avoid any spurious warnings.

Fixes #4028.
2020-03-15 15:36:08 -07:00
Luke Hoban 9da774e180
Fix Python mocks (#4074)
The original version of this code caused inconsistencies in the event
loop associated with a given thread. These changes elimintate the event
loop shenanigans the mocks were trying to play by updating _sync_await
to create an event loop if none exists in the current thread.

It's possible that this will cause problems if the tests run on a
different thread than the original program, as the tests are likely to
end up waiting on outputs created by the program, which is not supported
in Python.

Also adds test coverage of the mocking/testing support in Python.
2020-03-12 21:09:47 -07:00
Mikhail Shilkov 8f1534c895
Mention mocks in the missing project error (#4041) 2020-03-09 16:23:49 +01:00
Justin Van Patten 80f6c61310
Initial support for configuring policies (#4015) 2020-03-08 14:11:55 -07:00
Charly Román 560f1f130e Python SDK fix type annotations for Output.all and Output.concat 2020-03-03 22:00:37 -06:00
James Nugent 60eeff3265
Merge pull request #3990 from pulumi/jen20/main-dockerfile-for-proto
Update gRPC library used in Go SDKs and regenerate code with latest tools
2020-02-29 12:46:46 +00:00
Pat Gavlin 682dced40b
Mock resource monitor (#3738)
These changes add support for mocking the resource monitor to the NodeJS
and Python SDKs. The proposed mock interface is a simplified version of
the standard resource monitor that allows an end-user to replace the
usual implementations of ReadResource/RegisterResource and Invoke with
their own. This can be used in unit tests to allow for precise control
of resource outputs and invoke results.
2020-02-28 17:22:50 -08:00
James Nugent a1b55192dd Regenerate protocol buffers code 2020-02-28 11:53:47 +00:00
Pat Gavlin ccde31b713
Add support for extracting schemas from providers. (#3984)
These changes add a new method to the resource provider gRPC interface,
`GetSchema`, that allows consumers of these providers to extract
JSON-serialized schema information for the provider's types, resources,
and functions.
2020-02-27 16:10:47 -08:00
Levi Blackstone 0931c0d419
Fix missing module import on Windows platform (#3983)
#3895 added a module import that apparently doesn't
exist on Windows. Check the platform before importing,
and skip this import for Windows.
2020-02-26 16:08:49 -07:00
Lee Zen 33b232adc4
Fix a regression for CustomTimeouts in Python SDK (#3964)
* Fix a regression for CustomTimeouts in Python SDK
* Accept dict as well as CustomTimeouts object
2020-02-22 19:32:06 -08:00
Lee Zen 54d344f7c1
Serialize CustomTimeouts in Python SDK (#3920)
* Translate CustomTimeouts properly for RPC serialization
* Include integration test for custom timeouts
2020-02-13 10:40:56 -08:00
Levi Blackstone 7efb88de3e
Allow oversize protocol buffers (#3895)
Set an option to increase the memory limit on protobuf
parsing so that we can handle larger gRPC payloads.

Co-authored-by: Evan Boyle <EvanBoyle@users.noreply.github.com>
2020-02-10 15:30:42 -07:00
Justin Van Patten 3bf9067bac
Expose options, parent, deps, and provider config to policies (#3862) 2020-02-07 16:11:34 -08:00
Pat Gavlin 4a201a7dfd
Do not busy wait in Python. (#3892)
Instead, keep a stack of outstanding RPCs and await each in turn. This
allows the main loop to block instead of spin.

Fixes #3759.
2020-02-07 12:10:59 -08:00
Luke Hoban 5bf490228a
Use \n for provider protocol even on Windows (#3855)
The provider plugin protocol is to write a port number followed by `\n`.  We must guarantee we do that even on Windows, so must avoid Python `print` statements which implicitly rewrite newlines to platform specific character sequences.

Fixes #3807.
2020-02-06 11:34:05 -08:00
Evan Boyle 106154f1fc
fix python sdk stack ref regression (#3798) 2020-01-24 09:36:47 -08:00
James Forcier bf84a3482b Add py.typed to Python package for PEP 561 compliance (#3704)
PEP 561 specifies that packages which contain either inline type hints
or type stubs should indicate their support for type hints via
including a file named `py.typed` in the root of the package. Since
Pulumi already includes inline type hints, adding `py.typed` to the
Python SDK simply allows these hints to be used by mypy.
2020-01-17 15:01:36 -08:00
Evan Boyle 972fc44dd7
Enable mypy (#3758) 2020-01-17 14:45:08 -08:00
clstokes 0a3191905c Fixed 'transacational' misspelling. 2020-01-13 10:45:07 -08:00
CyrusNajmabadi f4fc00ad0e
Output.apply should lift resources from inner Outputs to the top level output. (#3663) 2019-12-17 14:11:45 -08:00