Commit graph

560 commits

Author SHA1 Message Date
Matt Ellis
0bb4e6d70b Respond to PR feedback
Address post commit feedback from Cyrus on
pulumi/pulumi#3071
2019-08-15 12:42:51 -07:00
Matt Ellis
9308246114 Do not taint all stack outputs as secrets if just one is
When using StackReference, if the stack you reference contains any
secret outputs, we have to mark the entire `outputs` member as a
secret output. This is because we only track secretness on a per
`Output<T>` basis.

For `getSecret` and friends, however, we know the name of the output
you are looking up and we can be smarter about if the returned
`Output<T>` should be treated as a secret or not.

This change augments the provider for StackReference such that it also
returns a list of top level stack output names who's values contain
secrets. In the language SDKs, we use this information, when present,
to decide if we should return an `Output<T>` that is marked as a
secret or not. Since the SDK and CLI are independent components, care
is taken to ensure that when the CLI does not return this information,
we behave as we did before (i.e. if any output is a secret, we treat
every output as a secret).

Fixes #2744
2019-08-13 16:11:38 -07:00
Pat Gavlin
fdfef5f334
Update the NodeJS version compat checks. (#3083)
- Unify the 1.x.y series and the 0.17.z series
- Fix the check s.t. post-1.0, only the major versions are required to
  match
2019-08-13 15:40:25 -07:00
CyrusNajmabadi
1a698cbc9e
Fix crashes around secrets with 'undefined' value. (#3069) 2019-08-12 16:00:20 -07:00
CyrusNajmabadi
24e2c6f791
Workaround intermittent issue on some machines where Object.values can't be found. (#3054) 2019-08-08 12:11:46 -07:00
Pat Gavlin
3c03ee3bdd
Fix invoke error reporting. (#3048)
- Report all failures
- Use appropriate member functions to access failure details

Fixes https://github.com/pulumi/pulumi-terraform/issues/396
2019-08-08 09:14:36 -07:00
Alex Clemmer
1da23d6a1e Fix typo in PP script 2019-08-06 17:35:52 -07:00
Pat Gavlin
62189e6053
Harden asset and archive deserialization. (#3042)
- Ensure that type assertions are guarded, and that incorrectly-typed
  properties return errors rather than panicking
- Expand the asset/archive tests in the Node SDK to ensure that eventual
  archives and assets serialize and deserialize correctly

Fixes #2836.
Fixes #3016.
2019-08-06 16:32:05 -07:00
CyrusNajmabadi
c846015643
Add tests (#3031) 2019-08-05 21:53:39 -07:00
Alex Clemmer
6360cba588 Improve "project not found" error messages
Fixes pulumi/pulumi-policy#36.
Fixes pulumi/pulumi-policy#37.
2019-08-05 14:14:20 -07:00
Pat Gavlin
2ab814fd09
Do not resolve missing outputs to inputs in preview. (#3014)
This can cause `apply`s to run on values that may change during an
update, which can lead to unexpected behavior.

Fixes #2433.
2019-08-05 12:44:04 -07:00
Luke Hoban
8da252270f
Fix tsconfig.json (#3028)
Missing files here lead to docs not being included.  In particular, docs for `interface CodePathOptions` were missing.
2019-08-05 11:55:55 -07:00
Luke Hoban
aac25eabc4
Add requireOutput to StackReference (#3007)
Provides an additional helper function to read outputs from a stack reference in case it is known that the stack output must be present. This is similar to the design for config.get and config.require.

Fixes #2343.
2019-08-01 11:27:32 -07:00
Pat Gavlin
1af7449f1a
Remove references to pulumi/glog. (#3009)
This package's flags conflict with those in google/glog. Replace all
references to this package with references to
pulumi/pulumi/pkg/util/logging, and change that package to explicitly
call `flag.CommandLine.Parse` with an empty slice.

This should make it much easier to consume these packages in downstream
repos that have direct or indirect dependencies on google/glog.
2019-07-31 13:23:33 -05:00
Pat Gavlin
67ec74bdc5
Pass ignoreChanges to providers. (#3005)
These changes add support for passing `ignoreChanges` paths to resource
providers. This is intended to accommodate providers that perform diffs
between resource inputs and resource state (e.g. all Terraform-based
providers, the k8s provider when using API server dry-runs). These paths
are specified using the same syntax as the paths used in detailed diffs.

In addition to passing these paths to providers, the existing support
for `ignoreChanges` in inputs has been extended to accept paths rather
than top-level keys. It is an error to specify a path that is missing
one or more component in the old or new inputs.

Fixes #2936, #2663.
2019-07-31 11:39:07 -05:00
CyrusNajmabadi
7bdd590586
Add deprecation warnings. (#3004) 2019-07-30 15:51:44 -07:00
CyrusNajmabadi
93f0bd708d
Add helper function for merging ResourceOptions (#2988) 2019-07-29 12:01:10 -07:00
James Nugent
7f6a6501ef Depend on pulumi fork of glog.
This removes the need for a replace directive in every downstream `go.mod`.
2019-07-25 16:10:53 -05:00
Chris Smith
208dfc29b4
Update references to pulumi.io (#2979)
* Remove pulumi.io reference in tests

* Remove pulumi.io references in Dockerfiles

* Remove pulumi.io references in CONTRIBUTING.md

* Update README.md's

* Use correct logo
2019-07-25 09:58:12 -07:00
CyrusNajmabadi
d47473e462
Remove unnecessary exports (#2956) 2019-07-19 12:49:48 -07:00
Alex Clemmer
784c39e22c Implement Omit like TypeScript upstream does 2019-07-19 11:14:31 -07:00
CyrusNajmabadi
70b3883942
Add an option to allow callers to specify they would like make invoke calls asynchronously. (#2947) 2019-07-18 17:19:41 -07:00
CyrusNajmabadi
e558296afa
Add a helper function to easily convert async functions to sync functions (#2943) 2019-07-16 12:12:21 -07:00
Luke Hoban
fc38d7d4d9
Handle mixed versions of Resources in parent hierarchy (#2942)
For new properties added to `Resource`, we need to make sure to handle cases where these are undefined as they may not be available on versions of `Resource` that come from older SDK versions, which could me side-by-side in a single Pulumi program execution.

Fixes #2938
2019-07-16 11:15:26 -07:00
Alex Clemmer
3bc86055c2 Add pulumi-analyzer-package scripts in publishing pipeline 2019-07-16 00:58:33 -07:00
Alex Clemmer
3fc03167c5 Implement cmd/run-policy-pack
This command will cause `pulumi policy publish` to behave in much the
same way `pulumi up` does -- if the policy program is in TypeScript, we
will use ts-node to attempt to compile in-process before executing, and
fall back to plain-old node.

We accomplish this by moving `cmd/run/run.ts` into a generic helper
package, `runtime/run.ts`, which slightly generalizes the use cases
supported (notably, allowing us to exec some program outside of the
context of a Pulumi stack).

This new package is then called by both `cmd/run/index.ts` and
`cmd/run-policy-pack/index.ts`.
2019-07-16 00:58:33 -07:00
Alex Clemmer
cd86475009 Add GetAnalyzerInfo to Analyzer gRPC definition
`GetAnalyzerInfo` is an RPC command that allows the Pulumi CLI to ask an
`Analyzer` for metadata about the resource policies it defines.

This is useful because the Pulumi service needs to be able to render
metadata about a policy pack after the user runs a `pulumi policy
publish`. Since we don't execute code on the server, the first step of
`policy publish` is to interrogate the policy pack program about what
policies it has, so that this metadata can be sent to the server, along
with a packed version of those policies.
2019-07-16 00:58:33 -07:00
Paul Stack
02ffff8840
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers

* Plumbing the CustomTimeouts through to the engine and adding test to show this

* Change the provider proto to include individual timeouts

* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface

* Change how the CustomTimeouts are sent across RPC

These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest

```
req=&pulumirpc.RegisterResourceRequest{
           Type:                    "aws:s3/bucket:Bucket",
           Name:                    "my-bucket",
           Parent:                  "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
           Custom:                  true,
           Object:                  &structpb.Struct{},
           Protect:                 false,
           Dependencies:            nil,
           Provider:                "",
           PropertyDependencies:    {},
           DeleteBeforeReplace:     false,
           Version:                 "",
           IgnoreChanges:           nil,
           AcceptSecrets:           true,
           AdditionalSecretOutputs: nil,
           Aliases:                 nil,
           CustomTimeouts:          &pulumirpc.RegisterResourceRequest_CustomTimeouts{
               Create:               300,
               Update:               400,
               Delete:               500,
               XXX_NoUnkeyedLiteral: struct {}{},
               XXX_unrecognized:     nil,
               XXX_sizecache:        0,
           },
           XXX_NoUnkeyedLiteral: struct {}{},
           XXX_unrecognized:     nil,
           XXX_sizecache:        0,
       }
```

* Changing the design to use strings

* CHANGELOG entry to include the CustomTimeouts work

* Changing custom timeouts to be passed around the engine as converted value

We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-16 00:26:28 +03:00
Pat Gavlin
e1a52693dc
Add support for importing existing resources. (#2893)
A resource can be imported by setting the `import` property in the
resource options bag when instantiating a resource. In order to
successfully import a resource, its desired configuration (i.e. its
inputs) must not differ from its actual configuration (i.e. its state)
as calculated by the resource's provider.

There are a few interesting state transitions hiding here when importing
a resource:
1. No prior resource exists in the checkpoint file. In this case, the
   resource is simply imported.
2. An external resource exists in the checkpoint file. In this case, the
   resource is imported and the old external state is discarded.
3. A non-external resource exists in the checkpoint file and its ID is
   different from the ID to import. In this case, the new resource is
   imported and the old resource is deleted.
4. A non-external resource exists in the checkpoint file, but the ID is
   the same as the ID to import. In this case, the import ID is ignored
   and the resource is treated as it would be in all cases except for
   changes that would replace the resource. In that case, the step
   generator issues an error that indicates that the import ID should be
   removed: were we to move forward with the replace, the new state of
   the stack would fall under case (3), which is almost certainly not
   what the user intends.

Fixes #1662.
2019-07-12 11:12:01 -07:00
CyrusNajmabadi
73b4100899
Support the exported value of a stack being a Promise/Output itself (#2916) 2019-07-09 17:22:35 -07:00
Matt Ellis
83e4bbdebb Lock to an earlier version of gRPC
The most recently released version of gRPC has a `index.d.ts` file in
it that does not work when complied with noImplicitAny. Until a fix
can be made upstream, lock to an earlier version so that we can build
without turning off noImplicitAny.
2019-07-09 14:39:53 -07:00
Matt Ellis
858517a7ca Correctly push secretness up during deserialization in runtimes
There current RPC model for Pulumi allows secret values to be deeply
embedded in lists or maps, however at the language level, since we
track secrets via `Output<T>` we need to ensure that during
deserialization, if a list or a map contains a secret, we need to
instead treat it as if the entire list or map was a secret.

We have logic in the language runtimes to do this as part of
serialization. There were a few issues this commit addresses:

- We were not promoting secretness across arrays in either Node or
  Python
- For Python, our promotion logic was buggy and caused it to behave in
  a manner where if any value was secret, the output values of the
  object would be corrupted, because we'd incorrectly treat the
  outputs as a secret who's value was a map, instead of a map of
  values (some of which may be secret).

This caused very confusing behavior, because it would appear that a
resource creation call just did not set various output properties when
one or more of them ended up containing a secret.
2019-07-09 10:40:27 -07:00
Pat Gavlin
6e5c4a38d8
Defer all diffs to resource providers. (#2849)
Thse changes make a subtle but critical adjustment to the process the
Pulumi engine uses to determine whether or not a difference exists
between a resource's actual and desired states, and adjusts the way this
difference is calculated and displayed accordingly.

Today, the Pulumi engine get the first chance to decide whether or not
there is a difference between a resource's actual and desired states. It
does this by comparing the current set of inputs for a resource (i.e.
the inputs from the running Pulumi program) with the last set of inputs
used to update the resource. If there is no difference between the old
and new inputs, the engine decides that no change is necessary without
consulting the resource's provider. Only if there are changes does the
engine consult the resource's provider for more information about the
difference. This can be problematic for a number of reasons:

- Not all providers do input-input comparison; some do input-state
  comparison
- Not all providers are able to update the last deployed set of inputs
  when performing a refresh
- Some providers--either intentionally or due to bugs--may see changes
  in resources whose inputs have not changed

All of these situations are confusing at the very least, and the first
is problematic with respect to correctness. Furthermore, the display
code only renders diffs it observes rather than rendering the diffs
observed by the provider, which can obscure the actual changes detected
at runtime.

These changes address both of these issues:
- Rather than comparing the current inputs against the last inputs
  before calling a resource provider's Diff function, the engine calls
  the Diff function in all cases.
- Providers may now return a list of properties that differ between the
  requested and actual state and the way in which they differ. This
  information will then be used by the CLI to render the diff
  appropriately. A provider may also indicate that a particular diff is
  between old and new inputs rather than old state and new inputs.

Fixes #2453.
2019-07-01 12:34:19 -07:00
CyrusNajmabadi
504478c641
Move to latest version of package (#2889) 2019-06-29 15:51:20 -07:00
Matt Ellis
881db4d72a Correctly flow secretness across POJO serliazation for stack outputs
Our logic to export a resource as a stack output transforms the
resource into a plain old object by eliding internal fields and then
just serializing the resource as a POJO.

The custom serialization logic we used here unwrapped an Output
without care to see if it held a secret. Now, when it does, we
continue to return an Output as the thing to be serialized and that
output is marked as a secret.

Fixes #2862
2019-06-26 15:16:07 -07:00
CyrusNajmabadi
c47ae413c3
Lock to a specific version of the read-package-tree library. (#2882) 2019-06-26 13:46:00 -07:00
CyrusNajmabadi
9823035de4
Dedupe aliases in the sdk as well (#2864) 2019-06-23 09:49:04 -07:00
CyrusNajmabadi
867abac947
Make it possible with aliases to say 'I had no parent before' (#2853) 2019-06-20 15:53:33 -07:00
Matt Ellis
eb3a7d0a7a Fix up some spelling errors
@keen99 pointed out that newer versions of golangci-lint were failing
due to some spelling errors. This change fixes them up.  We have also
now have a work item to track moving to a newer golangci-lint tool in
the future.

Fixes #2841
2019-06-18 15:30:25 -07:00
CyrusNajmabadi
55d2f82170
Make private field internal (#2840) 2019-06-18 10:54:14 -07:00
Matt Ellis
e3151c1528
Merge pull request #2837 from pulumi/ellismg/update-node-version
Update supported versions list
2019-06-17 13:27:58 -07:00
CyrusNajmabadi
11a19a4990
Make it possible to get a StackReference output promptly (#2824) 2019-06-17 12:25:56 -07:00
Matt Ellis
ad7397b310 Update supported versions list
We no longer support Node 6.X (it went end of life at the end of
April, we stopped testing on it, and now we use features that are not
in Node 6 like `Object.values`.

Note that we support 8+, and add an `engines` section to our
package.json so folks who try to install on older versions of node see
warnings.
2019-06-17 12:11:38 -07:00
Alex Clemmer
43ef3acb64 Remove policy ID from the analyzer protocol 2019-06-13 17:39:30 -07:00
Alex Clemmer
346861b9d6 Update Analyzer PB for policy 2019-06-13 16:04:13 -07:00
CyrusNajmabadi
54cbda80c1
Store information about a CustomResource's provider in __providers. (#2816) 2019-06-11 16:57:37 -07:00
CyrusNajmabadi
55bb3b2486
Simplify API for passing providers to a ComponentResource. (#2609) 2019-06-06 16:20:12 -07:00
Alex Clemmer
02788b9b32 Implement listResourceOutputs in the Node.js SDK
This commit will expose the new `Invoke` routine that lists resource
outputs through the Node.js SDK.

This API is implemented via a new API, `EnumerablePromise`, which is a
collection of simple query primitives built onto the `Promise` API. The
query model is lazy and LINQ-like, and generally intended to make
`Promise` simpler to deal with in query scenarios. See #2601 for more
details.

Fixes #2600.
2019-06-03 14:56:49 -07:00
Alex Clemmer
df5273e549 Add __pulumiType to CustomResource class
Running `pulumi query` over state resources will require the ability to
filter resources by type.

This commit begins the process of making this possible simply, using the
TypeScript user-defined type guards feature. This commit changes the
`CustomResource` constructor to record the `t` argument in the
`__pulumiType` field, which acts as a "sentinel" value in the resource
base, which deriving classes need only check to ensure they are of that
type. For example:

    aws.s3.Bucket.isInstance(someOb)

would check the `aws.s3.Bucket.__pulumiType` field, and make sure that
the type field lines up.
2019-06-03 14:56:49 -07:00
Matt Ellis
c201d92380 Use server information from NodeJS host for fetching plugins 2019-06-03 09:31:18 -07:00