Commit graph

733 commits

Author SHA1 Message Date
Matt Ellis 67f6d4d7e5 Update baseline for a previously failing test
Change is 3.6.2 of typescript have caused their code generation to no
longer emit a call to `this` inside an arrow function, so this test is
no longer causing an error to be thrown.

For now, just accept the baseline, but I'll file an issue so we can
actually get a real failing test here.
2019-08-28 13:18:28 -07:00
Matt Ellis e9fc96d4dc Update baselines for Typescript 3.6.2
TypeScript has changed the way the generate some code, and so we must
update our baselines to adapt.
2019-08-28 13:17:47 -07:00
Joe Duffy 2b48611d9c
Don't encourage PULUMI_TEST_MODE (#3146)
We intend to replace PULUMI_TEST_MODE with better testing support
that doesn't suffer from all the pitfalls of our current approach.
Unfortunately, we don't yet have complete guidance or validation
that the new approaches will work for all existing end users. So,
until we do, we'll take a lighter touch approach here, and simply
not encourage new usage of PULUMI_TEST_MODE.

Issue #3045 will remain open to track a mroe permanent fix.
2019-08-26 18:49:13 -07:00
Matt Ellis e349f3c094 Fix stack reference issue when running on a pre -beta.3 CLI
In #3071 we made change to the built in provider for `StackReference`
to return additional data about the names of outputs which were
secrets.  The SDKs uses this information to decide if it should mark a
stack output as secret when returning it's value from `getOutput`.

When using an older CLI which does not report this data, we hit an
error:

```
 TypeError: Cannot read property 'outputs' of undefined
```

This was caused by a refactoring error where we extracted a private
helper out of the StackReference class, but neglected to change one
instance of `this` to `sr`. While we do have tests that exercise this
feature, we would only see this bug when using an older CLI, and we
don't have automated tests here that use the neweset `@pulumi/pulumi`
with an older `pulumi` CLI
2019-08-22 11:55:02 -07:00
CyrusNajmabadi 590114497e
Simplify merge code. (#3122) 2019-08-20 17:09:53 -07:00
Pat Gavlin 8745440c1b
Allow users to explicitly disable delete-before-replace. (#3118)
With these changes, a user may explicitly set `deleteBeforeReplace` to
`false` in order to disable DBR behavior for a particular resource. This
is the SDK + CLI escape hatch for cases where the changes in
https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable
behavior.
2019-08-20 15:51:02 -07:00
CyrusNajmabadi d0a72fc24d
Allow ResourceOptions.id to be an Input in python (to match JS sdk). (#3116) 2019-08-20 12:18:48 -07:00
CyrusNajmabadi 8290377cbd
Add some simple typechecking (#3109) 2019-08-19 13:19:16 -07:00
Matt Ellis 0bb4e6d70b Respond to PR feedback
Address post commit feedback from Cyrus on
pulumi/pulumi#3071
2019-08-15 12:42:51 -07:00
CyrusNajmabadi 20058f982d
Make 'ResourceOptions.merge' static so downstream clients don't have to manually handle None cases. (#3072) 2019-08-14 15:55:03 -07:00
Matt Ellis 9308246114 Do not taint all stack outputs as secrets if just one is
When using StackReference, if the stack you reference contains any
secret outputs, we have to mark the entire `outputs` member as a
secret output. This is because we only track secretness on a per
`Output<T>` basis.

For `getSecret` and friends, however, we know the name of the output
you are looking up and we can be smarter about if the returned
`Output<T>` should be treated as a secret or not.

This change augments the provider for StackReference such that it also
returns a list of top level stack output names who's values contain
secrets. In the language SDKs, we use this information, when present,
to decide if we should return an `Output<T>` that is marked as a
secret or not. Since the SDK and CLI are independent components, care
is taken to ensure that when the CLI does not return this information,
we behave as we did before (i.e. if any output is a secret, we treat
every output as a secret).

Fixes #2744
2019-08-13 16:11:38 -07:00
Pat Gavlin fdfef5f334
Update the NodeJS version compat checks. (#3083)
- Unify the 1.x.y series and the 0.17.z series
- Fix the check s.t. post-1.0, only the major versions are required to
  match
2019-08-13 15:40:25 -07:00
CyrusNajmabadi 1a698cbc9e
Fix crashes around secrets with 'undefined' value. (#3069) 2019-08-12 16:00:20 -07:00
CyrusNajmabadi de254aa389
Do not include 'test' package in our python library. (#3067) 2019-08-12 11:32:33 -07:00
Pat Gavlin 7fef102bc3
Check for valid PB types in serialize_property (#3060)
Just what it says on the tin. This allows us to return an incrementally
better error.

Fixes #2939.
2019-08-09 16:48:28 -07:00
Pat Gavlin 91af1a93c4
Fix a reentrancy issue in _sync_await. (#3056)
_sync_await was not reentrant with respect to _run_once: the latter
captures the length of the ready list before it iterates it, and the
former drains the ready list by reentering _run_once. Fix this by
tracking the length of the list before pumping the event loop and then
pushing cancelled handles on to the list as necessary after pumping the
loop.

These changes also fix an issue with `export`ing awaitables.

Fixes #3038.
2019-08-08 19:51:11 -07:00
CyrusNajmabadi 24e2c6f791
Workaround intermittent issue on some machines where Object.values can't be found. (#3054) 2019-08-08 12:11:46 -07:00
Pat Gavlin 3c03ee3bdd
Fix invoke error reporting. (#3048)
- Report all failures
- Use appropriate member functions to access failure details

Fixes https://github.com/pulumi/pulumi-terraform/issues/396
2019-08-08 09:14:36 -07:00
CyrusNajmabadi a13a04b193
Move resource-options check down to core layer. (#3043) 2019-08-06 17:39:34 -07:00
Alex Clemmer 1da23d6a1e Fix typo in PP script 2019-08-06 17:35:52 -07:00
Pat Gavlin 62189e6053
Harden asset and archive deserialization. (#3042)
- Ensure that type assertions are guarded, and that incorrectly-typed
  properties return errors rather than panicking
- Expand the asset/archive tests in the Node SDK to ensure that eventual
  archives and assets serialize and deserialize correctly

Fixes #2836.
Fixes #3016.
2019-08-06 16:32:05 -07:00
CyrusNajmabadi 60206e716b
Fix spellin (#3041) 2019-08-06 16:16:33 -07:00
CyrusNajmabadi c846015643
Add tests (#3031) 2019-08-05 21:53:39 -07:00
Alex Clemmer 6360cba588 Improve "project not found" error messages
Fixes pulumi/pulumi-policy#36.
Fixes pulumi/pulumi-policy#37.
2019-08-05 14:14:20 -07:00
Pat Gavlin 2ab814fd09
Do not resolve missing outputs to inputs in preview. (#3014)
This can cause `apply`s to run on values that may change during an
update, which can lead to unexpected behavior.

Fixes #2433.
2019-08-05 12:44:04 -07:00
Luke Hoban 8da252270f
Fix tsconfig.json (#3028)
Missing files here lead to docs not being included.  In particular, docs for `interface CodePathOptions` were missing.
2019-08-05 11:55:55 -07:00
Pat Gavlin 06034fab40
Only await input properties once. (#3024)
These changes fix a bug in the Python runtime that would cause any
awaitable input properties passed to a resource that are missing
from the resource's output properties to be awaited twice. The fix is
straightforward: rather than roundtripping an input property through
serialize/deserialize, just deserialized the already-serialized input
property.

Fixes #2940.
2019-08-03 10:29:19 -07:00
Pat Gavlin 00bc1287af
Support synchronous invokes in Python 3.6. (#3025)
Python 3.6 does not expose _{enter,leave}_task. Detect this situation
and implement the equivalent operation as necessary.
2019-08-02 16:52:56 -07:00
Pat Gavlin 63eb7abb59
Make pulumi.runtime.invoke synchronous. (#3019)
These changes make the `pulumi.runtime.invoke` function invokable in a
synchronous manner. Because this function still needs to perform
asynchronous work under the covers--namely awaiting a provider URN and
ID if a provider instance is present in the `InvokeOptions`--this
requires some creativity. This creativity comes in the form of a helper
function, `_sync_await`, that performs a logical yield from the
currently running event, manually runs the event loop until the given
future completes, performs a logical resume back to the
currently executing event, and returns the result of the future.

The code in `_sync_await` is a bit scary, as it relies upon knowledge of
(and functions in) the internals of the `asyncio` package. The necessary
work performed in this function was derived from the implementations of
`task_step` (which pointed out the need to call `_{enter,leave}_task`)
and `BaseEventLoop.run_forever` (which illustrated how the event loop is
pumped). In addition to potential breaking changes to these internals,
the code may not work if a user has provided an alternative implementation
for `EventLoop`. That said, the code is a close enough copy of
`BaseEventLoop.run_forever` that it should be a reasonable solution.
2019-08-02 14:19:56 -07:00
Charlie Hills 654f5e2d35 Fixes python get_secret always returning None (#3022)
get_secret was looking up secrets by the fully resolved key, but should use the simple key.
2019-08-02 09:59:51 -07:00
CyrusNajmabadi d1376db975
Support arbitrary stack export values in python. (#3015) 2019-08-01 20:00:07 -07:00
Luke Hoban aac25eabc4
Add requireOutput to StackReference (#3007)
Provides an additional helper function to read outputs from a stack reference in case it is known that the stack output must be present. This is similar to the design for config.get and config.require.

Fixes #2343.
2019-08-01 11:27:32 -07:00
Luke Hoban 3d4c01abeb
Add Output.concat to Python (#3006)
Fixes #2366
2019-07-31 11:35:05 -07:00
Pat Gavlin 1af7449f1a
Remove references to pulumi/glog. (#3009)
This package's flags conflict with those in google/glog. Replace all
references to this package with references to
pulumi/pulumi/pkg/util/logging, and change that package to explicitly
call `flag.CommandLine.Parse` with an empty slice.

This should make it much easier to consume these packages in downstream
repos that have direct or indirect dependencies on google/glog.
2019-07-31 13:23:33 -05:00
Pat Gavlin 67ec74bdc5
Pass ignoreChanges to providers. (#3005)
These changes add support for passing `ignoreChanges` paths to resource
providers. This is intended to accommodate providers that perform diffs
between resource inputs and resource state (e.g. all Terraform-based
providers, the k8s provider when using API server dry-runs). These paths
are specified using the same syntax as the paths used in detailed diffs.

In addition to passing these paths to providers, the existing support
for `ignoreChanges` in inputs has been extended to accept paths rather
than top-level keys. It is an error to specify a path that is missing
one or more component in the old or new inputs.

Fixes #2936, #2663.
2019-07-31 11:39:07 -05:00
CyrusNajmabadi 7bdd590586
Add deprecation warnings. (#3004) 2019-07-30 15:51:44 -07:00
CyrusNajmabadi 93f0bd708d
Add helper function for merging ResourceOptions (#2988) 2019-07-29 12:01:10 -07:00
James Nugent 7f6a6501ef Depend on pulumi fork of glog.
This removes the need for a replace directive in every downstream `go.mod`.
2019-07-25 16:10:53 -05:00
CyrusNajmabadi 237f8d2222
Add python aliases support. (#2974) 2019-07-25 11:21:06 -07:00
Chris Smith 208dfc29b4
Update references to pulumi.io (#2979)
* Remove pulumi.io reference in tests

* Remove pulumi.io references in Dockerfiles

* Remove pulumi.io references in CONTRIBUTING.md

* Update README.md's

* Use correct logo
2019-07-25 09:58:12 -07:00
Luke Hoban fa4da84669
Add get_object functions to Python SDK (#2959)
Fixes #2320
2019-07-20 07:04:18 -07:00
CyrusNajmabadi d47473e462
Remove unnecessary exports (#2956) 2019-07-19 12:49:48 -07:00
Pat Gavlin b0fc599e02 Restore the old pickler in a try/finally (#2954)
Just what it says on the tin.
2019-07-19 12:08:15 -07:00
Alex Clemmer 784c39e22c Implement Omit like TypeScript upstream does 2019-07-19 11:14:31 -07:00
Luke Hoban 3768e5c690
Python Dynamic Providers (#2900)
Dynamic providers in Python.

This PR uses [dill](https://pypi.org/project/dill/) for code serialization, along with a customization to help ensure deterministic serialization results.

One notable limitation - which I believe is a general requirement of Python - is that any serialization of Python functions must serialize byte code, and byte code is not safely versioned across Python versions.  So any resource created with Python `3.x.y` can only be updated by exactly the same version of Python.  This is very constraining, but it's not clear there is any other option within the realm of what "dynamic providers" are as a feature.  It is plausible that we could ensure that updates which only update the serialized provider can avoid calling the dynamic provider operations, so that version updates could still be accomplished.  We can explore this separately.

```py
from pulumi import ComponentResource, export, Input, Output
from pulumi.dynamic import Resource, ResourceProvider, CreateResult, UpdateResult
from typing import Optional
from github import Github, GithubObject

auth = "<auth token>"
g = Github(auth)

class GithubLabelArgs(object):
    owner: Input[str]
    repo: Input[str]
    name: Input[str]
    color: Input[str]
    description: Optional[Input[str]]
    def __init__(self, owner, repo, name, color, description=None):
        self.owner = owner
        self.repo = repo
        self.name = name
        self.color = color
        self.description = description

class GithubLabelProvider(ResourceProvider):
    def create(self, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).create_label(
            name=props["name"],
            color=props["color"],
            description=props.get("description", GithubObject.NotSet))
        return CreateResult(l.name, {**props, **l.raw_data}) 
    def update(self, id, _olds, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.edit(name=props["name"],
               color=props["color"],
               description=props.get("description", GithubObject.NotSet))
        return UpdateResult({**props, **l.raw_data})
    def delete(self, id, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.delete()

class GithubLabel(Resource):
    name: Output[str]
    color: Output[str]
    url: Output[str]
    description: Output[str]
    def __init__(self, name, args: GithubLabelArgs, opts = None):
        full_args = {'url':None, 'description':None, 'name':None, 'color':None, **vars(args)}
        super().__init__(GithubLabelProvider(), name, full_args, opts)

label = GithubLabel("foo", GithubLabelArgs("lukehoban", "todo", "mylabel", "d94f0b"))

export("label_color", label.color)
export("label_url", label.url)
```


Fixes https://github.com/pulumi/pulumi/issues/2902.
2019-07-19 10:18:25 -07:00
CyrusNajmabadi 70b3883942
Add an option to allow callers to specify they would like make invoke calls asynchronously. (#2947) 2019-07-18 17:19:41 -07:00
CyrusNajmabadi e558296afa
Add a helper function to easily convert async functions to sync functions (#2943) 2019-07-16 12:12:21 -07:00
Luke Hoban fc38d7d4d9
Handle mixed versions of Resources in parent hierarchy (#2942)
For new properties added to `Resource`, we need to make sure to handle cases where these are undefined as they may not be available on versions of `Resource` that come from older SDK versions, which could me side-by-side in a single Pulumi program execution.

Fixes #2938
2019-07-16 11:15:26 -07:00
Alex Clemmer 3bc86055c2 Add pulumi-analyzer-package scripts in publishing pipeline 2019-07-16 00:58:33 -07:00
Alex Clemmer 3fc03167c5 Implement cmd/run-policy-pack
This command will cause `pulumi policy publish` to behave in much the
same way `pulumi up` does -- if the policy program is in TypeScript, we
will use ts-node to attempt to compile in-process before executing, and
fall back to plain-old node.

We accomplish this by moving `cmd/run/run.ts` into a generic helper
package, `runtime/run.ts`, which slightly generalizes the use cases
supported (notably, allowing us to exec some program outside of the
context of a Pulumi stack).

This new package is then called by both `cmd/run/index.ts` and
`cmd/run-policy-pack/index.ts`.
2019-07-16 00:58:33 -07:00