Commit graph

504 commits

Author SHA1 Message Date
Aylei faebaa0dc5
[ciutil] Correct system name of Jenkins CI (#5891)
Signed-off-by: Aylei <rayingecho@gmail.com>
2020-12-08 18:36:31 +00:00
Luke Hoban df8f01dba7
Revert #5857 (#5880)
Fixes #5879.
2020-12-07 10:06:41 -08:00
Joe Duffy a5c30f1547
Properly resize arrays when adding (#5872)
* Properly resize arrays when adding

The current logic attempts to update the array but, because
append may need to allocate a new array with adequate space,
the code can currently leave dest referring to the old,
under-sized array. The solution is to use the set(dest)
logic that already exists and is used for the IsNull case.

Added a test case that would fail before this fix and now passes.

This fixes pulumi/pulumi#5871.

* Add CHANGELOG entry
2020-12-06 21:23:50 +00:00
Justin Van Patten 5ab051cd97
Implement GetRequiredPlugins for Python (#5787)
Implement GetRequiredPlugins for Python, which determines the plugins
required by the program.

Also, if the `virtualenv` runtime option is set, and the specified
virtual directory is missing or empty, automatically create it and
install dependencies into it.
2020-12-03 19:22:16 -08:00
Luke Hoban 9e955241fc
Make credentials.json file writes concurrency safe (#5857)
Running `pulumi` operations in parallel could occasionally result in truncating the `~/.pulumi/credentials.json` file and reading that truncated file from another process before the content could be written.

Instead, use `os.Rename` to atomically replace the file contents.

Concurrent `pulumi` operations could still compete for who gets to write the file first, and could lead to surprising results in some extreme cases.   But we should not see the corrupted file contents any longer.

Fixes #3877.
2020-12-03 15:07:55 -08:00
Levi Blackstone aec5ada15d
[sdk/go]: Add support for getResource to Go SDK (#5823)
- Clean up resourceOptions and associated methods
- Add URN field and setter function
- Invoke getResource if URN set
2020-12-01 11:03:47 -07:00
Pat Gavlin 807b09d6a6
Fix a few issues with resource references. (#5804)
- Differentiate between resource references that have no ID (i.e. because
  the referenced resource is not a CustomResource) and resource references
  that have IDs that are not known. This is necessary for proper
  backwards-compatible serialization of resource references.
- Fix the key that stores a resource reference's package version in the
  .NET, NodeJS, and Python SDKs.
- Ensure that the resource monitor's marshalling/unmarshalling  of inputs
  and outputs to/from calls to `Construct` retain resource references as
  appropriate.
- Fix serialization behavior for resources -> resource references in the
  Go SDK: if a resource's ID is unknown, it should still be serialized
  as a resource reference, albeit a reference with an unknown ID.
2020-11-23 11:15:10 -08:00
Luke Hoban 4ecd8f9f56
Add PULUMI_BACKEND_URL env var (#5789)
The PULUMI_BACKEND_URL env var allows specifying the backend to use instead of deferring to the project or the ~/.pulumi/credentials.json file to decide on the "current" backend.  This allows for using Pulumi without a dependence on this piece of global filesystem state, so that each `pulumi` invocation can control the exact backend it want's to operate on, without having to do stateful `pulumi login`/`pulumi logout` operations.

This is especially useful for automation scenarios like Automation API generally (and effectively solves https://github.com/pulumi/pulumi/issues/5591), or https://github.com/pulumi/pulumi-kubernetes-operator/issues/83 specifically.

This also makes things like efe7a599e6/dist/actions/entrypoint.sh (L10) less necessary, and possible to accomplish for any containerized `pulumi` execution without the need for this logic to be embedded in bash scripts wrapping the CLI.
2020-11-22 15:27:59 -08:00
Justin Van Patten 9a707c4e03
Improve the output when installing plugins (#5808)
Two improvements:

1. Don't display "[resource plugin <foo>] installing" if the plugin is already installed.

2. Close the plugin download progress bar before displaying any subsequent output, and only show output of `npm install` when there is an error.
2020-11-21 08:02:05 -08:00
Vivek Lakshmanan 302f869dc5 Drop unnecessary prefix in test output since we rely on go's testing to do the right thing 2020-11-17 23:24:08 -08:00
Justin Van Patten 594da1e95c
Fix plugin install failures on Windows (#5759)
When installing a plugin, previous versions of Pulumi extracted the
plugin tarball to a temp directory and then renamed the temp directory
to the final plugin directory. This was done to prevent concurrent
installs: if a process fails to rename the temp dir because the final
dir already exists, it means another process already installed the
plugin. Unfortunately, on Windows the rename operation often fails due
to aggressive virus scanners opening files in the temp dir.

In order to provide reliable plugin installs on Windows, we now extract
the tarball directly into the final directory, and use file locks to
prevent concurrent installs from toppling over one another.

During install, a lock file is created in the plugin cache directory
with the same name as the plugin's final directory but suffixed with
`.lock`. The process that obtains the lock is responsible for extracting
the tarball. Before it does that, it cleans up any previous temp
directories of failed installs of previous versions of Pulumi. Then it
creates an empty `.partial` file next to the `.lock` file. The
`.partial` file indicates an installation is in-progress. The `.partial`
file is deleted when installation is complete, indicating the plugin was
successfully installed. If a failure occurs during installation, the
`.partial` file will remain indicating the plugin wasn't fully
installed. The next time the plugin is installed, the old installation
directory will be removed and replaced with a fresh install.

This is the same approach Go uses for installing modules in its
module cache.
2020-11-16 09:44:29 -08:00
Vivek Lakshmanan 9d6e380680
Merge pull request #5741 from pulumi/vl/GoVersion
Pulumi-language-go and pulumi new now checks go version is at least 1.14.0
2020-11-12 15:40:57 -08:00
Pat Gavlin 1e0c9efdd7
Respect provider config secretness. (#5742)
Just what it says on the tin. This is implemented by changing the
`GetPackageConfig` method of `ConfigSource` to return a `PropertyMap`
and ensuring that any secret config is represented by a `Secret`.
2020-11-12 12:18:12 -08:00
Vivek Lakshmanan b2e2562427 Fix test 2020-11-12 09:20:33 -08:00
Vivek Lakshmanan bcb9d0f4c8 Refactor to add version check to pulumi new as well 2020-11-12 09:20:28 -08:00
Vivek Lakshmanan f3b41e038e Pulumi-language-go now checks go version is atleast 1.14.0 2020-11-11 15:39:34 -08:00
Evan Boyle 98a048b716
Turn Automation API unmarshaling error into AutoError for Go SDK preview ops (#5715) 2020-11-08 18:53:05 -08:00
Komal 00f8966a9a
[codegen/nodejs] - Fix typescript enum issues (#5696)
Co-authored-by: Levi Blackstone <levi@pulumi.com>
2020-11-06 09:29:49 -08:00
RyanW02 90fbe4bc8f
Add boolean values to Go SDK (#5647) (#5687)
* Add boolean values to Go SDK (#5647)

* Update changelog
2020-11-05 13:33:58 -08:00
Pat Gavlin 44376738f6
Add resource modules. (#5645)
This is necessary due to the way we've factored the libraries imported
by users into modules. The primary alternative is to ensure that each
child module imports the root module for a package and registers itself
with that package where necessary to prevent circular dependencies. This
simplifies the core SDKs slightly at the cost of greater complications
in the generated SDKs; the approach taken by these changes seems like a
more maintainable option.

Contributes to #2430.

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2020-11-04 10:24:41 -08:00
Komal f1f6a126bc
Support python 3.9 (#5669)
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: stack72 <public@paulstack.co.uk>
2020-11-03 16:28:45 -08:00
Justin Van Patten 855f14c053
Support remote components in Go (#5558) 2020-10-29 15:13:17 -07:00
Pat Gavlin 3d2e31289a
Add support for serialized resource references. (#5041)
Resources are serialized as their URN, ID, and package version. Each
Pulumi package is expected to register itself with the SDK. The package
will be invoked to construct appropriate instances of rehydrated
resources. Packages are distinguished by their name and their version.

This is the foundation of cross-process resources.

Related to #2430.

Co-authored-by: Mikhail Shilkov <github@mikhail.io>
Co-authored-by: Luke Hoban <luke@pulumi.com>
Co-authored-by: Levi Blackstone <levi@pulumi.com>
2020-10-27 10:12:12 -07:00
Pat Gavlin b516c8944f
Respect disableProviderPreview in providers. (#5612) 2020-10-22 21:25:05 +01:00
Pat Gavlin 89c71bb49e
[cli] Add an import command. (#4765)
Co-authored-by: stack72 <public@paulstack.co.uk>
2020-10-14 12:51:53 +01:00
Mikhail Shilkov 29850869ef
Pass the preview flag to Update (#5562) 2020-10-13 21:02:19 +02:00
Paul Stack 186e2f059e
Respect logging verbosity as part of pulumi plugin install (#5549)
Fixes: #4427

By default, there is no indepth logging:

```
pulumi plugin install resource azure-nextgen v0.2.1
[resource plugin azure-nextgen-0.2.1] installing
Downloading plugin: 17.82 MiB / 17.82 MiB [=========================] 100.00% 2s
Moving plugin... done.
```

We can increase the logging verosity get more detailed logging:
```
pulumi plugin install resource azure-nextgen v0.2.1 -v=1 --logtostderr
[resource plugin azure-nextgen-0.2.1] installing
I1010 19:30:58.472772    7128 plugins.go:201] azure-nextgen downloading from https://get.pulumi.com/releases/plugins
Downloading plugin: 0 B / 17.82 MiB [----------------------------------]   0.00%I1010 19:30:58.704168    7128 plugin_install.go:136] [resource plugin azure-nextgen-0.2.1] installing tarball ...
Downloading plugin: 17.82 MiB / 17.82 MiB [=========================] 100.00% 3s
Moving plugin...I1010 19:31:01.874427    7128 plugins.go:304] moving plugin from "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1.tmp884796935" to "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1"
 done.
```

The most verbose logging level will log the request and response headers
```
pulumi plugin install resource azure-nextgen v0.2.1 -v=9 --logtostderr
I1010 19:29:46.989150    7089 sink.go:146] defaultSink::Infoerr([resource plugin azure-nextgen-0.2.1] installing)
[resource plugin azure-nextgen-0.2.1] installing
I1010 19:29:46.989295    7089 plugins.go:591] SelectCompatiblePlugin(..., azure-nextgen): beginning
I1010 19:29:46.989300    7089 plugins.go:630] SelectCompatiblePlugin(..., azure-nextgen): failed to find match
I1010 19:29:46.989323    7089 plugins.go:201] azure-nextgen downloading from https://get.pulumi.com/releases/plugins
I1010 19:29:46.989333    7089 plugins.go:208] full plugin download url: https://get.pulumi.com/releases/plugins/pulumi-resource-azure-nextgen-v0.2.1-darwin-amd64.tar.gz
I1010 19:29:46.989360    7089 plugins.go:218] plugin install request headers: map[User-Agent:[pulumi-cli/1 (; darwin)]]
I1010 19:29:47.242941    7089 plugins.go:225] plugin install response headers: map[Accept-Ranges:[bytes] Age:[370098] Content-Disposition:[attachment; filename=pulumi-resource-azure-nextgen-v0.2.1-darwin-amd64.tar.gz] Content-Length:[18684382] Content-Type:[application/x-gzip] Date:[Tue, 06 Oct 2020 11:41:30 GMT] Etag:["518f2e7efd46fe5b7be9508dc785b9c9-4"] Last-Modified:[Sat, 03 Oct 2020 20:57:45 GMT] Server:[AmazonS3] Via:[1.1 198b7d1bb217783eef010e6636984c9f.cloudfront.net (CloudFront)] X-Amz-Cf-Id:[aaC3Q0WcecspsFbWROLxXnD6iF7sYoAfShQKYiS2xkFKAWlFRBu-1Q==] X-Amz-Cf-Pop:[MAN50-C2] X-Cache:[Hit from cloudfront]]
Downloading plugin: 0 B / 17.82 MiB [----------------------------------]   0.00%I1010 19:29:47.243189    7089 plugin_install.go:136] [resource plugin azure-nextgen-0.2.1] installing tarball ...
Downloading plugin: 17.82 MiB / 17.82 MiB [=========================] 100.00% 2s
Moving plugin...I1010 19:29:50.031099    7089 plugins.go:304] moving plugin from "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1.tmp960784076" to "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1"
 done.
```
2020-10-13 13:09:29 +01:00
Paul Stack 156aa9862e
Add the ability to specify privateKey contents to AutomationAPI (#5557)
Fixes: #5383
2020-10-12 19:51:26 +01:00
Paul Stack 64577f5b10
Protect against panic in finding go executable and finding resources (#5548)
Fixes: #5131
Fixes: #5016
2020-10-11 07:13:54 +01:00
Pat Gavlin 249140242e
Add support for provider-side preview. (#5443)
These changes add support for provider-side previews of create and
update operations, which allows resource providers to supply output
property values for resources that are being created or updated during a
preview.

If a plugin supports provider-side preview, its create/update methods
will be invoked during previews with the `preview` property set to true.
It is the responsibility of the provider to fill in any output
properties that are known before returning. It is a best practice for
providers to only fill in property values that are guaranteed to be
identical if the preview were instead an update (i.e. only those output
properties whose values can be conclusively determined without
actually performing the create/update operation should be populated).
Providers that support previews must accept unknown values in their
create and update methods.

If a plugin does not support provider-side preview, the inputs to a
create or update operation will be propagated to the outputs as they are
today.

Fixes #4992.
2020-10-09 13:13:55 -07:00
Evan Boyle b86e6ae93e
add plugin management utils to node Automation API (#5536) 2020-10-09 09:03:03 -07:00
Joe Duffy 12e5e46c73
Handle blank project names in 'new' (#5504)
The logic for validating prompted values in 'new' wasn't quite right,
leading to the possibility of creating Pulumi.yaml files with blank
project names.

This manifests in various ways and I've hit it a number of times
over the past few months because of the way we handle project/stack
name conflicts in 'new' -- which itself is a bit annoying too:

https://github.com/pulumi/pulumi/blob/master/pkg/cmd/pulumi/new.go#L206-L207

Because we substitue a default value of "", and because the prompting
logic assumed default values are always valid, we would skip validation
and therefore accept a blank Pulumi.yaml file.

This generates an invalid project which causes errors elsewhere, such as

    error: failed to load Pulumi project located at ".../Pulumi.yaml":
        project is missing a 'name' attribute

I hit this all the time with our getting started guide because I've
gone through it so many times and have leftover stacks from prior
run-throughs. I wouldn't be surprised if a lot of people hit this.

The solution here validates all values, including the default.

Note also that we failed to validate the value used by 'new --yes'
which meant you could bypass all validation by passing --yes, leading
to similar outcomes.

I've added a couple new tests for these cases. There is a risk we
depend on illegal default values somewhere which will now be rejected,
but that would seem strange, and assuming the tests pass, I would
assume that's not true. Let me know if that's wrong.

Fixes pulumi/pulumi#3255.
2020-10-05 13:40:24 -07:00
Mike Metral 08c8848375 feat(autoapi): expose EnvVars LocalWorkspaceOption to set in ctor 2020-10-02 09:01:28 -07:00
jetvova b4c2a3521c
Fixed issue #5014 (Missing Pulumi console tags on Windows) (#5406)
Co-authored-by: Paul Stack <public@paulstack.co.uk>
2020-09-25 15:04:18 +01:00
Paul Stack 7c258c77c3
change how secrets provider parameters are passed via automation api (#5402) 2020-09-17 19:57:27 +01:00
Levi Blackstone ab70dd3d64
[sdk/go] Add missing Version field to invokeOptions (#5401) 2020-09-17 12:21:50 -06:00
Paul Stack f1d0b31cda
Update the docs and examples for Automation API for private git repos (#5396) 2020-09-17 15:48:44 +01:00
Evan Boyle dd51072507
skip test (#5394) 2020-09-16 14:38:27 -07:00
Evan Boyle 0e3666cc36
Automation API - add recovery APIs (cancel/export/import) (#5369) 2020-09-15 14:20:58 -07:00
Mike Metral 4e6ea760db feat(autoapi): add GetPermalink for operation result 2020-09-15 10:49:02 -07:00
Evan Boyle 4744d9acd2
up concurrency to fix flaky test (#5372) 2020-09-15 12:24:39 +01:00
Evan Boyle c23ca46382
Automation API - add the ability to stream progress updates during up/refresh/destroy (#5367) 2020-09-14 18:56:04 -07:00
Evan Boyle c05ac500da
[Automation API] remove stack name validation and fqsn enforcement (#5337) 2020-09-14 17:45:07 -07:00
Pat Gavlin 855f1fd1cd
Revise host mode. (#5317)
* Revise host mode.

The current implementation of host mode uses a `pulumi host` command and
an ad-hoc communication protocol between the engine and client to
connect a language host after the host has begun listening. The most
significant disadvantages of this approach are the communication
protocol (which currently requires the use of stdout), the host-specific
command, and the difficulty of accommodating the typical program-bound
lifetime for an update.

These changes reimplement host mode by adding engine support for
connecting to an existing language runtime service rather than launching
a plugin. This capability is provided via an engine-specific language
runtime, `client`, which accepts the address of the existing languge
runtime service as a runtime option. The CLI exposes this runtime via
the `--client` flag to the `up` and `preview` commands, which similarly
accepts the address of an existing language runtime service as an
argument. These changes also adjust the automation API to consume the
new host mode implementation.
2020-09-14 17:40:17 -07:00
Justin Van Patten 46c7c327dd
Install plugin dependencies (#5353)
When installing a plugin, if it contains a `PulumiPlugin.yaml` file with a `runtime` value of `nodejs` or `python`, install dependencies for the plugin.

For Node.js, `npm install` is run (or `yarn install` if `PULUMI_PREFER_YARN` is set).

For Python, a virtual environment is created and deps installed into it.
2020-09-14 13:54:26 -07:00
Paul Stack aafe84d823
Add support for cloning private repos as part of automation API (#5333) 2020-09-14 20:24:57 +01:00
Paul Stack 9a74064d2b
Allow passing a non-default secrets provider to AutomationAPI (#5320) 2020-09-10 19:25:47 +01:00
Mike Metral 49152e6331 feat(autoapi): add Upsert methods for stacks 2020-09-10 09:49:49 -07:00
Mike Metral 51c235352c feat(autoapi): add IsSelectStack404Error and IsCreateStack409Error 2020-09-09 17:13:50 -07:00
Mike Metral 3f445be494 fix(autoapi): return autoError for local_workspace ops 2020-09-09 17:13:50 -07:00
Lee Briggs 0ac075ef8f
add initial pull-request workflow (#5276)
* add initial pull-request workflow

* run SDK test all

* add SDK tests

* fixup make targets

* add dist target

* revert back to 5 updates

* disable test

* add issue for test disabling
2020-09-09 13:37:03 -07:00
Pat Gavlin 2585b86aa4
Initial support for remote component construction. (#5280)
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.

Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.

The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.

These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).

An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.

This is the core of #2430.
2020-09-07 19:33:55 -07:00
Mike Metral 46da25492e feat(autoapi): add workspace scoped envvars to LocalWorkspace and Stack 2020-09-03 09:57:23 -07:00
Evan Boyle 9ef5c72f72
update go plugin acquisition to work for users who vendor dependencies (#5286) 2020-09-03 09:15:18 -07:00
Mike Metral dec6a6a49a refactor(autoapi-gitrepo): use Workspace in SetupFn callback 2020-09-02 19:15:40 -07:00
evanboyle d3cf745c59 First pass at Automation API README 2020-09-01 23:28:46 -07:00
Evan Boyle a0275ffbd1
check for nil resource and invoke options in go sdk (#5270) 2020-09-01 09:59:27 -07:00
Evan Boyle 741485d7d0
expose execution kind via update metadata (#5258) 2020-08-31 09:07:59 -07:00
evanboyle 5458e2f685 add expectnochanges to stack.refresh 2020-08-28 19:17:46 -07:00
evanboyle c23f26b335 docs and style updates 2020-08-28 19:10:39 -07:00
evanboyle b0d91478bf docs updates 2020-08-28 14:21:56 -07:00
evanboyle cf247c1245 update file headers 2020-08-27 10:43:23 -07:00
evanboyle ab03c53487 automation api godoc 2020-08-25 11:16:54 -07:00
evanboyle 463c7b6588 Merge branch 'master' of https://github.com/pulumi/pulumi into evan/auto 2020-08-24 17:51:40 -07:00
evanboyle 40701e6420 outputs should be map[string]interface{} 2020-08-24 06:51:49 -07:00
evanboyle ec98494e5a add auto tests to Makefile and fix up for CI 2020-08-23 21:36:21 -07:00
evanboyle 27db14791a auto error tests 2020-08-23 19:43:51 -07:00
evanboyle 47fb5a128d Add functional options for Up/Pre/Refresh/Destroy 2020-08-23 10:25:17 -07:00
evanboyle fa1590a67f propagate nested stack error 2020-08-22 12:01:55 -07:00
evanboyle 458fd59c24 Make Workspace and Stack commands contextful 2020-08-21 22:20:32 -07:00
evanboyle 4dacd506d3 propagate env vars and extensibility points from Workspace to Stack 2020-08-21 16:22:45 -07:00
evanboyle cc7cf7c476 Add Stack.Info 2020-08-21 15:26:58 -07:00
evanboyle 3531635294 Add NewStack helper methods and tests 2020-08-21 09:49:46 -07:00
evanboyle 7b58202938 Implement Stack based on Workspace interface 2020-08-20 19:37:39 -07:00
evanboyle 535d48ca72 Round out LocalWorkspace impl 2020-08-19 21:23:53 -07:00
evanboyle 93ff03a9d3 LocalWorkspace initial implementation 2020-08-19 11:13:42 -07:00
evanboyle 4a44b27b40 add automation api workspace interface 2020-08-18 11:07:03 -07:00
Lee Briggs c325e4f53b
Merge branch 'justin/testresource' into jar-extract 2020-08-10 18:14:43 -07:00
Lee Briggs 574a9151db
support jar files 2020-08-10 14:15:35 -07:00
Evan Boyle 996fd85e00
Add missing MapMap and ArrayArray types to Go SDK (#5092) 2020-08-03 11:37:28 -07:00
evanboyle 12d7216769 serve engine address via pulumi host command 2020-07-30 15:20:57 -07:00
evanboyle 9281e55e00 detect and throw errors for nested stacks 2020-07-29 19:01:51 -07:00
evanboyle 5d8aa34d09 run all commands in non-interactive mode 2020-07-29 11:22:46 -07:00
evanboyle 7e6a08dbe1 example update 2020-07-28 17:40:25 -07:00
evanboyle 34a86f54be godoc improvements 2020-07-28 17:32:38 -07:00
evanboyle 8a707e27e3 godoc pt. 1 2020-07-27 22:26:39 -07:00
evanboyle 4f653683ce validate project name 2020-07-27 18:51:29 -07:00
evanboyle 114e3c2aff provide defaults for pului.yaml for inline source 2020-07-27 18:22:01 -07:00
Lee Briggs bad67d3242
switch os/user with luser
We make several calls to `os/user`, which uses CGO and means
cross-compilation is not possible. This replaces `os/user` with the
`luser` package, which is a drop-in replacement which does not use `CGO`
2020-07-27 14:44:08 -07:00
evanboyle 45ed471a7d handle and detect runtime errors for inline source 2020-07-26 13:43:33 -07:00
evanboyle 41d6767e32 host error handling, inline concurrent update detection 2020-07-26 12:52:59 -07:00
evanboyle 15b8f03dc3 enable e2e up/pre/ref/dest/rm test for inline source 2020-07-25 15:08:10 -07:00
evanboyle ae76944b77 use ts seeds for random test names 2020-07-25 13:28:57 -07:00
evanboyle b21f4e0f7b inline preview 2020-07-24 10:54:18 -07:00
evanboyle b0468d4166 init inline prototype 2020-07-24 01:31:54 -07:00
evanboyle 0d3203c96d add ability to specify git URL instead of local source 2020-07-22 14:28:03 -07:00
evanboyle 6e7ce72db6 update error formatting 2020-07-21 18:44:53 -07:00
evanboyle e789ae9700 update test 2020-07-21 18:41:36 -07:00
evanboyle acc719e7c1 IsUnexpectedEngineError() 2020-07-21 18:39:35 -07:00
evanboyle be42f75d2c add some comments based on findings 2020-07-21 18:27:31 -07:00
evanboyle 0500aae0e0 IsRuntimeError() 2020-07-21 18:17:38 -07:00
evanboyle c360e4d3f1 Merge branch 'master' of https://github.com/pulumi/pulumi into evan/auto 2020-07-21 16:52:08 -07:00
evanboyle ff81dd8961 IsCompilationError 2020-07-21 16:21:47 -07:00
evanboyle f6bc545809 clean up test 2020-07-17 20:08:45 -07:00
evanboyle cf4e02d517 init error strategy 2020-07-17 20:04:37 -07:00
evanboyle 99d769b903 typo 2020-07-17 11:15:58 -07:00
evanboyle 5faf54cedd lint 2020-07-17 10:59:13 -07:00
evanboyle 5b9f1b60d4 tighten up the interface 2020-07-17 10:14:30 -07:00
Paul Stack 6d09fe32df
Add the ability to copy configs between stacks (#4971) 2020-07-17 11:14:10 +01:00
Pat Gavlin 418e2291a2
Protect against engine event mutation. (#5003)
Certain operations in `engine/diff` mutate engine events during display.
This mutation can occur concurrently with the serialization of the event
for persistence, which causes a panic in the CLI. These changes fix the
offending code and add code that copies each engine event before
persisteing it in order to guard against future issues.
2020-07-16 23:52:31 -07:00
evanboyle 96b9c74bda refresh, e2e test 2020-07-16 19:17:07 -07:00
evanboyle 7ac704125e pulumi stack rm 2020-07-16 18:50:34 -07:00
evanboyle 102dbad5ea destroy 2020-07-16 08:47:04 -07:00
evanboyle 30f0962529 preview 2020-07-16 00:34:13 -07:00
evanboyle c09a1cb351 capture last event from history for UpResult 2020-07-15 23:34:34 -07:00
evanboyle 13746f9c82 set config and secrets upon stack initialization 2020-07-14 23:35:26 -07:00
evanboyle ea6cd84dce add utils to round trip pulumi.yaml and pulumi.stack.yaml 2020-07-14 23:13:45 -07:00
evanboyle ae465a19c9 expose GetUser GetOutputs 2020-07-10 00:55:07 -07:00
evanboyle e08e362a85 init/select stack, improvements 2020-07-09 22:30:52 -07:00
Pat Gavlin 2119601023
Implement PropertyPath.Add. (#4976)
This function adds a property value to another property at a given path,
creating containing properties as required. If the property cannot be
added because of a mismatch between the value types required by the path
and the values present in the destination, the add will fail. If a value
already exists at the given path, the add will succeed.

Related to #1635.
2020-07-07 14:45:36 -07:00
evanboyle 2504db3cd2 prototype 2020-07-07 14:25:11 -07:00
Paul Stack 57b91d553e
Adding the ability to specify a version as a ResourceOption (#4945) 2020-07-07 20:20:01 +01:00
Levi Blackstone 8961f5b0ca
Rename RetrieveTemplate function to RetrieveGitFolder (#4912)
The new name better reflects the functionality for external
consumers of this package.
2020-06-26 15:01:51 -06:00
Evan Boyle 2351fc66cf
Go SDK interfaces should declare impl of Ptr types where appropriate (#4911) 2020-06-26 11:25:53 -07:00
Pat Gavlin 3e6e28af36
Fix lint errors. (#4856)
These errors appear when using golangci-lint 1.27.0.
2020-06-18 14:17:11 -07:00
Levi Blackstone 19a113de7a
Handle invalid UTF-8 characters before RPC calls (#4816)
Several users reported cases where error messages would
cause a panic if they contained accented characters. I wasn't
able to reproduce this failure locally, but tracked down the
panic to logging gRPC calls. The Message field is typed as
a string, which requires all of the characters to be valid UTF-8.

This change runs each log string through the strings.ToValidUTF8
function, which will replace any invalid characters with the
"unknown" character. This should prevent the the logger from
panicking.
2020-06-17 13:30:59 -06:00
Praneet Loke 2513709089
Use the GH webhook payload to parse PR details when CI system is GH Actions (#4817)
* Read the file and parse it. Also use the PR Head SHA instead of the GITHUB_REF if the build type is pull_request.

* Update changelog
2020-06-15 12:49:09 -07:00
Evan Boyle fa69a31c97
Set default namespace for go config for config-less get/try/require (#4802) 2020-06-12 16:35:06 -07:00
Evan Boyle 3d2df8992f
Use appropriate types for go config methods (#4800) 2020-06-10 18:24:05 -07:00
Justin Van Patten b77ec919d4
Install and use dependencies automatically for new Python projects (#4775)
Automatically create a virtual environment and install dependencies in it with `pulumi new` and `pulumi policy new` for Python templates.

This will save a new `virtualenv` runtime option in `Pulumi.yaml` (`PulumiPolicy.yaml` for policy packs):

```yaml
runtime:
  name: python
  options:
    virtualenv: venv
```

`virtualenv` is the path to a virtual environment that Pulumi will use when running `python` commands.

Existing projects are unaffected and can opt-in to using this by setting `virtualenv`, otherwise, they'll continue to work as-is.
2020-06-09 16:42:53 -07:00
Praneet Loke 407896730c
Add GitHub Actions detection and update the Pulumi CI escape hatch with more CI vars (#4758)
* Add GitHub Actions detection. Update the Pulumi CI escape hatch with more CI vars.
2020-06-08 17:24:12 -07:00
Lee Zen 0dac22c360
Escape the path to ensure we fetch plugins correctly (#4733) 2020-05-29 14:59:25 -07:00
Chris Smith f613b6917b
Download plugins from get.pulumi.com (#4692)
* Remove long-since deprecated --cloud-url flag

* Download plugins from get.pulumi.com

* Update CHANGELOG.md

* Use more straight forward TrimSuffix
2020-05-25 09:16:57 -07:00
Justin Van Patten 6b0a845cc1
Support publishing Python policy packs (#4644)
Adds support for publishing Python policy packs to the service, and downloading/using such policy packs when applied to stacks in an organization.
2020-05-22 15:01:15 -07:00
Pat Gavlin e677c7d6dc
[sdk/go] Add support for untagged outputs. (#4640)
With these changes, a resource struct may tag a field with the empty
string. If such a field is present, any resource outputs that were not
unmarshalled into other fields will be unmarshalled into this field,
which must be a `MapOutput`.

Fixes #4629.
2020-05-14 14:43:20 -06:00
Lee Briggs 8c314ec39a
Add brew make targets to sdk makefiles 2020-05-13 20:42:01 -07:00
Paul Stack 48f906e026
Allow pulumi stack export to decrypt secrets (#4046) 2020-05-11 19:16:30 +01:00
Levi Blackstone b445206101
Add ResourceOutput type to Go SDK (#4575) 2020-05-05 21:13:58 -06:00
Pat Gavlin 9b1b9cca24
Makefile: properly encode dependencies (#4541)
This allows for the use of `make -j`, which speeds up a full build
dramatically at the cost of rather incomprehensible logs.
2020-05-04 14:26:52 -07:00
Luke Hoban edf8bf5c30
Support map-typed inputs in Go SDK RegisterResource (#4521)
Adds support for RegisterResource to accept map-typed implementations if Input as well as the existing struct-typed implementations. Currently these must be fully untyped - but both map[string]pulumi.Input and map[string]interface{} are allowed. In the future, it's plausible that a mode where the data itself is a map, but the ElementType implementation returns a struct could be supported, with the struct used to provide type information over the untyped map.
2020-04-30 11:56:47 -07:00
Levi Blackstone d1ba572e1d
Fix infinite recursion bug for Go SDK (#4516)
gatherDependencySet was not checking for Resources
in the dependency set, which could lead to infinite
recursion.
2020-04-28 18:10:07 -06:00
Komal Ali 9cf635ad5a pr changes 2020-04-23 11:16:36 -07:00
Komal Ali cc96df7485 update channel options in go 2020-04-23 11:16:36 -07:00
Justin Van Patten 7f27618e2d
Avoid replace on second update with import applied (#4403)
After importing some resources, and running a second update with the
import still applied, an unexpected replace would occur. This wouldn't
happen for the vast majority of resources, but for some it would.

It turns out that the resources that trigger this are ones that use a
different format of identifier for the import input than they do for the
ID property.

Before this change, we would trigger an import-replacement when an
existing resource's ID property didn't match the import property, which
would be the case for the small set of resources where the input
identifier is different than the ID property.

To avoid this, we now store the `importID` in the statefile, and
compare that to the import property instead of comparing the ID.
2020-04-15 18:52:40 -07:00
Justin Van Patten 33119659e0
Treat config values that start with '0' as strings (#4393)
When setting structured config values using `--path`, we automatically
treat values that can be converted into an integer via `strconv.Atoi` as
an integer, rather than as a string.

However, this ends up converting values like "0123456" into the integer
123456, stripping the leading 0, which isn't desirable for values like
commit SHAs, etc., where you want to keep the 0 (and keep it a string).

This change makes it so that values starting with 0 are not implicitly
converted to an integer; instead such values will remain a string.
2020-04-14 12:40:22 -07:00
stack72 d1c76808af Small linting error after merging feature 2.0 branch 2020-04-14 09:55:18 +01:00
stack72 74df34be75 Re-enabling dockerfile latest, docs and homebrew publishing
These had previously been disabled due to the beta releases. Also
fixes an import path for finding an executable
2020-04-14 09:40:11 +01:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls (#3750)

* Switch away from native grpc impl. (#3728)

* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)

* Only retry as long as we get unavailable back.  Anything else continues. (#3769)

* Handle all errors for now. (#3781)


* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes (#4288)

Adjust C# generation

* Replace IDeployment with a sealed class (#4318)

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty (#4320)

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen (#4379)
2020-04-14 09:30:25 +01:00
Evan Boyle 4261b27a5e
fix propagation of secrets for resource inputs/outputs in go sdk (#4387) 2020-04-13 22:48:36 -07:00
Evan Boyle 9d2f40b686
fix unknown status for secrets wrapping unknowns (#4377)
* fix unknown status for secrets wrapping unknowns

* changelog

* add test for secret computed values
2020-04-13 22:47:08 +01:00
Evan Boyle 4bbddafe6c
update remote path for policy pack template tests (#4381) 2020-04-13 14:02:17 -07:00
Evan Boyle fe7877177c
fix stack reference helpers to handle nil values (#4370) 2020-04-12 08:44:08 -07:00
Evan Boyle 779c4144f9
Properly propagate unknowns in go SDK marshaling operations (#4369) 2020-04-11 22:20:03 -07:00
Evan Boyle c3b2439094
Automate execution of go mod download for pulumi new Go templates (#4353) 2020-04-09 16:16:10 -07:00
Evan Boyle cfea357cda
Add runtime.options.binary to Pulumi.yaml to make prebuilt go binaries opt-in (#4338) 2020-04-09 12:21:26 -07:00
Evan Boyle 7382de6b5f
Add Go SDK stack output helpers for String and ID (#4341) 2020-04-09 07:52:34 -07:00
Evan Boyle f50fd69c10
Define merge behavior for go resource options (#4316) 2020-04-07 14:19:33 -07:00
Evan Boyle ab659aa0c1
Reimplement getRequiredPlugins for go sdk (#4297) 2020-04-06 12:30:40 -07:00
Luke Hoban 9e37741916
[Go] Ensure Apply handles nil output values (#4268)
Fixes #4247.
2020-04-01 17:39:01 -07:00
Evan Boyle 8a8f424628
disable go plugin acquisition (#4257) 2020-03-31 23:00:35 -07:00
Luke Hoban dd9ed3c938
Support grpc-js plugin servers (#4253)
In order to support grpc-js gRPC servers, we need to slightly loosen
our checks on plugin readiness.  This change was already in the 2.x
branch, but porting back so that 1.x CLIs can also be compatible.

Fixes #4252.
2020-03-31 14:18:09 -07:00
stack72 a1af9289c3 Move BuildUtil class to sdk
This was still in pkg and wasn't accessible from the scripts location.
This code should be in the go/common location of sdk so that it can be
accessed from within our build scripts

Without this change, we were not able to get access to the Version number
for Python and none of our Python builds have been publishing binaries
2020-03-31 12:07:24 +01:00
Justin Van Patten e6be38e285
PaC: Add initial config support for policy packs (#4233)
The initial config represents any config that was specified programmatically to the Policy Pack, for Policy Packs that support programmatic configuration like AWSGuard.
2020-03-30 12:52:05 -07:00
Tasia Halim 4afaf37b3a fixed bug 2020-03-27 10:25:06 -07:00
Evan Boyle 80f1989600
Removing the need for vendoring (#4167) 2020-03-25 15:57:46 -07:00
Evan Boyle 2408d34c71
Add missing builtin MapArray to the Go SDK (#4144) 2020-03-20 16:51:33 -07:00
Evan Boyle c28c602247
capture std err when getting required plugins for go (#4141)
* capture std err when getting required plugins for go
2020-03-20 14:59:12 -07:00
evanboyle 163444bce6 disable logging test 2020-03-19 14:45:03 -07:00
evanboyle fa47d867fd fix go sdk makefile to run tests 2020-03-19 14:10:48 -07:00
evanboyle a2b368827f fix internal logging usage 2020-03-19 11:45:10 -07:00
evanboyle 31d8f79f35 fix klog initialization 2020-03-19 10:01:10 -07:00
evanboyle 4e44854308 replace glog with klog 2020-03-19 08:56:44 -07:00
evanboyle ec686bbaf6 Merge branch 'master' of https://github.com/pulumi/pulumi into evan/gomod 2020-03-18 17:57:58 -07:00
evanboyle 29dfd65632 fix resource test data path 2020-03-18 17:36:28 -07:00
evanboyle bc4e5bf31c run common tests as a part of go sdk make 2020-03-18 17:32:27 -07:00
evanboyle d3f5bbce48 go fmt 2020-03-18 17:27:02 -07:00
evanboyle 67f6b72caa Revert "move sdk/go/pulumi-language-go -> pkg/cmd/pulumi-language-go"
This reverts commit 7ea547a14d.
2020-03-18 17:14:41 -07:00
evanboyle 6933745342 move pkg/tools -> sdk/go/common/tools 2020-03-18 15:56:58 -07:00
evanboyle a4ec3ec81b move pkg/testing -> sdk/go/common/testing, leave behind pkg/testing/integration 2020-03-18 15:55:41 -07:00
evanboyle 0e9c5989bb move pkg/util/gitutil -> sdk/go/common/util/gitutil 2020-03-18 15:52:09 -07:00
evanboyle df722e974a move pkg/util/archive -> sdk/go/common/util/archive 2020-03-18 15:50:04 -07:00
evanboyle 30e1a5917e move pkg/util/fsutil -> sdk/go/common/util/fsutil 2020-03-18 15:48:38 -07:00
evanboyle 33e23bdf0c move pkg/util/retry -> sdk/go/common/util/retry 2020-03-18 15:47:01 -07:00
evanboyle c1440e48d4 move pkg/util/result -> sdk/go/common/util 2020-03-18 15:45:42 -07:00
evanboyle fa348ceb1b move pkg/util/ciutil -> sdk/go/common/util/ciutil 2020-03-18 15:43:31 -07:00
evanboyle c1d3a8524b move pkg/util/cmdutil -> sdk/go/common/util/cmdutil 2020-03-18 15:39:00 -07:00
evanboyle 7ff46cb4fa move pkg/util/rpcutil -> sdk/go/common/util/rpcutil 2020-03-18 15:37:13 -07:00
evanboyle c3f6ae2451 move pkg/util/logging -> sdk/go/common/util/logging 2020-03-18 15:34:58 -07:00
evanboyle cdfb8608c9 move pkg/encoding -> sdk/go/common/encoding 2020-03-18 15:28:19 -07:00
evanboyle 30df499838 move pkg/version -> sdk/go/common/version 2020-03-18 15:25:25 -07:00
evanboyle 8df534a71e move pkg/diag -> sdk/go/common/diag 2020-03-18 15:09:29 -07:00
evanboyle f754b486b8 move pkg/resource/config -> sdk/go/common/resource/config 2020-03-18 15:03:37 -07:00
evanboyle 67cb405c93 move pkg/apitype -> sdk/common/apitype 2020-03-18 15:00:30 -07:00
evanboyle 70f386a967 move pkg/tokens -> sdk/go/common/tokens 2020-03-18 14:49:56 -07:00
evanboyle 4fe4d48ec5 move pkg/util/mapper -> sdk/go/common/util/mapper 2020-03-18 14:47:37 -07:00
evanboyle 8a87090068 move pkg/util/httputil -> sdk/go/common/util/httputil 2020-03-18 14:45:00 -07:00
evanboyle fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
evanboyle 8fb3f428b0 move pkg/workspace -> sdk/go/common/workspace 2020-03-18 14:35:53 -07:00
evanboyle dfab571aac move pkg/resource/plugin -> sdk/go/common/resource/plugin 2020-03-18 14:26:24 -07:00
evanboyle fba783caf9 move pkg/resource -> sdk/go/common/resource, but leave nested resource packages 2020-03-18 13:36:19 -07:00
evanboyle 7ea547a14d move sdk/go/pulumi-language-go -> pkg/cmd/pulumi-language-go 2020-03-18 12:49:56 -07:00
Evan Boyle 8b46e71887
Go plugin acquisition (#4060)
These changes implement `GetRequiredPlugins` for Go using a registry
mechanism and an alternate entry point for `pulumi.Run`. Packages that
require plugins are expected to register themselves with the Pulumi SDK.
When `pulumi.Run` is used and the `PULUMI_PLUGINS` envvar is truthy, the
program will dump a JSON-encoded description of its required plugins to
stdout. The language host then uses this description to respond to
2020-03-18 12:41:45 -07:00
Levi Blackstone ee62f38ab9
Add ArrayMap builtin types to Go SDK (#4086)
Add support for maps of arrays of builtin types.
These types are of the form map[string][]builtin.
2020-03-16 20:41:08 -06:00
Levi Blackstone ad1c884aad
Fix lint issue (#4088)
Linter was complaining about possibly nil dereferences,
but these are already checked with an assertion.
2020-03-16 16:45:02 -06:00
Tasia Halim b6e5d2737d
Update Go logging to use LogArgs (#4078) 2020-03-13 13:30:05 -07:00
Tasia Halim ef6f0d4de4
Expose engine logging to Go SDK (#4069)
* expose engine logging to Go SDK

* added comments
2020-03-12 16:00:59 -07:00
Paul Stack 2a24470135
Merge pull request #4030 from pulumi/stack72/allow-go-run-windows
Windows requires applications to have exe extension
2020-03-09 17:36:10 +02:00
Evan Boyle 81b6afa3c7
Revert "GetRequiredPlugins for Go (#3830)" (#4034)
This reverts commit 3a2890c0cd.
2020-03-06 17:50:18 -08:00
Pat Gavlin 3a2890c0cd
GetRequiredPlugins for Go (#3830)
These changes implement `GetRequiredPlugins` for Go using a registry
mechanism and an alternate entry point for `pulumi.Run`. Packages that
require plugins are expected to register themselves with the Pulumi SDK.
When `pulumi.Run` is used and the `PULUMI_PLUGINS` envvar is truthy, the
program will dump a JSON-encoded description of its required plugins to
stdout. The language host then uses this description to respond to
`GetRequiredPlugins`.
2020-03-06 14:30:43 -08:00
Evan Boyle e75dca1207
fix reflect panic (#4027) 2020-03-06 11:01:21 -08:00
stack72 7f86842c68 Windows requires applications to have exe extension
We need to ensure that if the pulumi application is prebuild on
Windows then it will have the exe extension otherwise it's not
a valid windows program

https://github.com/golang/go/wiki/WindowsCrossCompiling
2020-03-06 19:58:57 +02:00
Evan Boyle 3b8ffd5731
Go SDK fix panic when dest is struct and input is ptr (#3986) 2020-03-03 09:07:06 -08:00
Tasia Halim c96271b7a3
Support transformations in Go (#3978)
* started transformations for go sdk

* added first basic test

* added second test with child

* added RegisterStackTransformation

* added a couple tests to lifecycle_test

* update CHANGELOG and test

* included TODO for #3846
2020-03-02 13:59:11 -08:00
Evan Boyle cd0025edb5
properly check for nil input or nil input pointer (#4010) 2020-03-02 12:48:30 -08:00
James Nugent 60eeff3265
Merge pull request #3990 from pulumi/jen20/main-dockerfile-for-proto
Update gRPC library used in Go SDKs and regenerate code with latest tools
2020-02-29 12:46:46 +00:00
Evan Boyle 3627f4ce70
ignore internal properties when unmarshaling (#3996) 2020-02-28 10:01:35 -08:00
James Nugent 652bc7ab75 Upgrade Go gRPC library in go.mod 2020-02-28 11:52:22 +00:00
James Nugent 666c5d5cdc Remove use of github.com/pkg/errors from Go SDK
In preparation for publishing a separate module of the Go SDK for Pulumi
on which providers can depend, we should reduce the dependency footprint
so as to cause end users as few issues as possible with transitive
dependency versioning.

This commit removes all use of `github.com/pkg/errors` from the Go SDK
to that end, replacing it with the standard `errors` package and `fmt`
for error formatting where appropriate. We use the new (as of Go 1.13)
"%w" syntax for wrapping errors, so this code is no longer compatible
with Go 1.12.
2020-02-26 16:27:07 +00:00
Evan Boyle 930adc0504
Add support for secrets in Go SDK (#3938) 2020-02-25 17:45:36 -08:00
Pat Gavlin 0aa208a306
Add comments to Go output resolution (#3975) 2020-02-25 13:26:43 -08:00
Evan Boyle cea807a244
Go aliases (#3853) 2020-02-06 12:02:13 -08:00
Evan Boyle 411f1a179a
Support stack references in Go SDK (#3829) 2020-02-06 10:00:46 -08:00
Pat Gavlin a64616a0f9 Fix three bugs in the Go SDK
- Do not set top-level properties that were not provided
- Check for nil after awaiting inputs
- Check for interface-typed values, which can be produced by `Any`
- Pass a settable value to `awaitInputs` in `Any`
2020-01-27 09:14:23 -08:00
Pat Gavlin 47b7eaf484
Make primitive input types implement pointer types (#3806)
For example, pulumi.String also implements pulumi.StringPtr. This is
consistent with the output of the code generator, and makes optional
inputs much more ergonomic.
2020-01-25 12:19:00 -08:00
Pat Gavlin 2f554d5cdc
Fix an assert in the Go SDK. (#3794)
This assert is not correct in the case of pointer input types, in
particular `pulumi.stringPtr`. Though these types are not assignable,
they are convertible.
2020-01-23 13:07:47 -08:00
Pat Gavlin f9548b3095
Add coverage for pulumi.Fprintf. (#3782)
Just what it says on the tin.
2020-01-21 13:27:46 -08:00
Pat Gavlin 4611cbe645
Add more test coverage for builtins. (#3775)
- Test builtin Apply methods
- Test type-specific ToOutput methods
- Test pointer, array, and map indexers
2020-01-21 09:52:57 -08:00
Pat Gavlin f168bdc1c2
Redesign the Go SDK resource/input/output system. (#3506)
The redesign is focused around providing better static typings and
improved ease-of-use for the Go SDK. Most of the redesign revolves
around three pivots:
- Strongly-typed inputs, especially for nested types
- Struct-based resource and invoke APIs
- Ease-of-use of Apply

1. Strongly-typed inputs

Input is the type of a generic input value for a Pulumi resource.
This type is used in conjunction with Output to provide polymorphism
over strongly-typed input values.

The intended pattern for nested Pulumi value types is to define an
input interface and a plain, input, and output variant of the value
type that implement the input interface.

For example, given a nested Pulumi value type with the following shape:

```
type Nested struct {
    Foo int
    Bar string
}
```

We would define the following:

```
var nestedType = reflect.TypeOf((*Nested)(nil)).Elem()

type NestedInput interface {
    pulumi.Input

    ToNestedOutput() NestedOutput
    ToNestedOutputWithContext(context.Context) NestedOutput
}

type Nested struct {
    Foo int `pulumi:"foo"`
    Bar string `pulumi:"bar"`
}

type NestedInputValue struct {
    Foo pulumi.IntInput `pulumi:"foo"`
    Bar pulumi.StringInput `pulumi:"bar"`
}

func (NestedInputValue) ElementType() reflect.Type {
    return nestedType
}

func (v NestedInputValue) ToNestedOutput() NestedOutput {
    return pulumi.ToOutput(v).(NestedOutput)
}

func (v NestedInputValue) ToNestedOutputWithContext(ctx context.Context) NestedOutput {
    return pulumi.ToOutputWithContext(ctx, v).(NestedOutput)
}

type NestedOutput struct { *pulumi.OutputState }

func (NestedOutput) ElementType() reflect.Type {
    return nestedType
}

func (o NestedOutput) ToNestedOutput() NestedOutput {
    return o
}

func (o NestedOutput) ToNestedOutputWithContext(ctx context.Context) NestedOutput {
    return o
}

func (o NestedOutput) Foo() pulumi.IntOutput {
    return o.Apply(func (v Nested) int {
        return v.Foo
    }).(pulumi.IntOutput)
}

func (o NestedOutput) Bar() pulumi.StringOutput {
    return o.Apply(func (v Nested) string {
        return v.Bar
    }).(pulumi.StringOutput)
}
```

The SDK provides input and output types for primitives, arrays, and
maps.

2. Struct-based APIs

Instead of providing expected output properties in the input map passed
to {Read,Register}Resource and returning the outputs as a map, the user
now passes a pointer to a struct that implements one of the Resource
interfaces and has appropriately typed and tagged fields that represent
its output properties.

For example, given a custom resource with an int-typed output "foo" and
a string-typed output "bar", we would define the following
CustomResource type:

```
type MyResource struct {
    pulumi.CustomResourceState

    Foo pulumi.IntOutput    `pulumi:"foo"`
    Bar pulumi.StringOutput `pulumi:"bar"`
}
```

And invoke RegisterResource like so:

```
var resource MyResource
err := ctx.RegisterResource(tok, name, props, &resource, opts...)
```

Invoke arguments and results are also provided via structs, but use
plain-old Go types for their fields:

```
type MyInvokeArgs struct {
    Foo int `pulumi:"foo"`
}

type MyInvokeResult struct {
    Bar string `pulumi:"bar"`
}

var result MyInvokeResult
err := ctx.Invoke(tok, MyInvokeArgs{Foo: 42}, &result, opts...)
```

3. Ease-of-use of Apply

All `Apply` methods now accept an interface{} as the callback type.
The provided callback value must have one of the following signatures:

	func (v T) U
	func (v T) (U, error)
	func (ctx context.Context, v T) U
	func (ctx context.Context, v T) (U, error)

T must be assignable from the ElementType of the Output. If U is a type
that has a registered Output type, the result of the Apply will be the
corresponding Output type. Otherwise, the result of the Apply will be
AnyOutput.

Fixes https://github.com/pulumi/pulumi/issues/2149.
Fixes https://github.com/pulumi/pulumi/issues/3488.
Fixes https://github.com/pulumi/pulumi/issues/3487.
Fixes https://github.com/pulumi/pulumi-aws/issues/248.
Fixes https://github.com/pulumi/pulumi/issues/3492.
Fixes https://github.com/pulumi/pulumi/issues/3491.
Fixes https://github.com/pulumi/pulumi/issues/3562.
2020-01-18 10:08:37 -05:00
Justin Van Patten 10a960ea4b
PaC: Support Config/getProject/getStack/isDryRun (#3612)
Add support for using `Config`, `getProject()`, `getStack()`, and
`isDryRun()` from Policy Packs.
2019-12-16 22:51:02 +00:00
Evan Boyle 1ca50d4b89
Propagate parent and providers for go SDK calls (#3563) 2019-11-26 13:23:34 -08:00
Evan Boyle c83e4f9ca6
Fix go SDK ReadResource (#3581) 2019-11-25 15:31:12 -08:00
Evan Boyle a47103b49d
fix go sdk delete before replace implementation (#3572) 2019-11-25 14:10:06 -08:00
Evan Boyle 8547ede659
Add Go support for config.*Object (#3526) 2019-11-18 18:53:27 -08:00
Evan Boyle 3ac8dd5285
Add support to the go sdk for IgnoreChanges (#3514) 2019-11-18 16:47:19 -08:00
Evan Boyle 5ae4149af5
Add support for "go run" style execution (#3503) 2019-11-14 09:25:55 -08:00
Pat Gavlin a7f61a59b0
Reimplement Output for Go. (#3496)
- Use a mutex + condition variable instead of a channel for
  synchronizaiton in order to allow multiple calls to resolve/reject
- Properly handle outputs that are resolved to other outputs, especially
  if those outputs are not of exactly type Output
- Remove the Value() methods that allowed prompt access to output values
- Add variants of `Apply` that take a context parameter
- Ensure that resource outputs properly incorporate their resource as
  a dependency
- Make `Output` a plain struct. Uninitialized outputs will be treated as
   resolved and unknown. This makes conversions between output
   types more ergonomic.

Contributes to #3492.
2019-11-12 14:20:06 -08:00
CyrusNajmabadi df06b8fc9b
Add publishing to nuget support (#3416) 2019-10-29 20:14:49 -07:00
Pat Gavlin 82204230e1
Improve tracing support. (#3238)
* Fix some tracing issues.

- Add endpoints for `startUpdate` and `postEngineEventsBatch` so that
  spans for these invocations have proper names
- Inject a tracing span when walking a plan so that resource operations
  are properly parented
- When handling gRPC calls, inject a tracing span into the call's
  metadata if no span is already present so that resource monitor and
  engine spans are properly parented
- Do not trace client gRPC invocations of the empty method so that these
  calls (which are used to determine server availability) do not muddy
  the trace. Note that I tried parenting these spans appropriately, but
  doing so broke the trace entirely.

With these changes, the only unparented span in a typical Pulumi
invocation is a single call to `getUser`. This span is unparented
because that call does not have a context available. Plumbing a context
into that particular call is surprisingly tricky, as it is often called
by other context-less functions.

* Make tracing support more flexible.

- Add support for writing trace data to a local file using Appdash
- Add support for viewing Appdash traces via the CLI
2019-09-16 14:16:43 -07:00
Pat Gavlin 2ab814fd09
Do not resolve missing outputs to inputs in preview. (#3014)
This can cause `apply`s to run on values that may change during an
update, which can lead to unexpected behavior.

Fixes #2433.
2019-08-05 12:44:04 -07:00
Pat Gavlin 1af7449f1a
Remove references to pulumi/glog. (#3009)
This package's flags conflict with those in google/glog. Replace all
references to this package with references to
pulumi/pulumi/pkg/util/logging, and change that package to explicitly
call `flag.CommandLine.Parse` with an empty slice.

This should make it much easier to consume these packages in downstream
repos that have direct or indirect dependencies on google/glog.
2019-07-31 13:23:33 -05:00
James Nugent 7f6a6501ef Depend on pulumi fork of glog.
This removes the need for a replace directive in every downstream `go.mod`.
2019-07-25 16:10:53 -05:00
Paul Stack 02ffff8840
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers

* Plumbing the CustomTimeouts through to the engine and adding test to show this

* Change the provider proto to include individual timeouts

* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface

* Change how the CustomTimeouts are sent across RPC

These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest

```
req=&pulumirpc.RegisterResourceRequest{
           Type:                    "aws:s3/bucket:Bucket",
           Name:                    "my-bucket",
           Parent:                  "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
           Custom:                  true,
           Object:                  &structpb.Struct{},
           Protect:                 false,
           Dependencies:            nil,
           Provider:                "",
           PropertyDependencies:    {},
           DeleteBeforeReplace:     false,
           Version:                 "",
           IgnoreChanges:           nil,
           AcceptSecrets:           true,
           AdditionalSecretOutputs: nil,
           Aliases:                 nil,
           CustomTimeouts:          &pulumirpc.RegisterResourceRequest_CustomTimeouts{
               Create:               300,
               Update:               400,
               Delete:               500,
               XXX_NoUnkeyedLiteral: struct {}{},
               XXX_unrecognized:     nil,
               XXX_sizecache:        0,
           },
           XXX_NoUnkeyedLiteral: struct {}{},
           XXX_unrecognized:     nil,
           XXX_sizecache:        0,
       }
```

* Changing the design to use strings

* CHANGELOG entry to include the CustomTimeouts work

* Changing custom timeouts to be passed around the engine as converted value

We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-16 00:26:28 +03:00
Pat Gavlin e1a52693dc
Add support for importing existing resources. (#2893)
A resource can be imported by setting the `import` property in the
resource options bag when instantiating a resource. In order to
successfully import a resource, its desired configuration (i.e. its
inputs) must not differ from its actual configuration (i.e. its state)
as calculated by the resource's provider.

There are a few interesting state transitions hiding here when importing
a resource:
1. No prior resource exists in the checkpoint file. In this case, the
   resource is simply imported.
2. An external resource exists in the checkpoint file. In this case, the
   resource is imported and the old external state is discarded.
3. A non-external resource exists in the checkpoint file and its ID is
   different from the ID to import. In this case, the new resource is
   imported and the old resource is deleted.
4. A non-external resource exists in the checkpoint file, but the ID is
   the same as the ID to import. In this case, the import ID is ignored
   and the resource is treated as it would be in all cases except for
   changes that would replace the resource. In that case, the step
   generator issues an error that indicates that the import ID should be
   removed: were we to move forward with the replace, the new state of
   the stack would fall under case (3), which is almost certainly not
   what the user intends.

Fixes #1662.
2019-07-12 11:12:01 -07:00
Justin Van Patten fedfc9b6b4
pulumi update => pulumi up (#2702)
We changed the `pulumi update` command to be `pulumi up` a while back
(`update` is an alias of `up`). This change just makes it so we refer to
the actual command, `pulumi up`, instead of the older `pulumi update`.
2019-05-06 14:00:18 -07:00
Pat Gavlin 6e90ab0341
Add support for explicit delete-before-replace (#2415)
These changes add a new flag to the various `ResourceOptions` types that
indicates that a resource should be deleted before it is replaced, even
if the provider does not require this behavior. The usual
delete-before-replace cascade semantics apply.

Fixes #1620.
2019-01-31 14:27:53 -08:00
Pat Gavlin 35c60d61eb
Follow up on #2369 (#2397)
- Add support for per-property dependencies to the Go SDK
- Add tests for first-class secret rejection in the checkpoint and RPC
  layers and language SDKs
2019-01-28 17:38:16 -08:00
Pat Gavlin 1ecdc83a33 Implement more precise delete-before-replace semantics. (#2369)
This implements the new algorithm for deciding which resources must be
deleted due to a delete-before-replace operation.

We need to compute the set of resources that may be replaced by a
change to the resource under consideration. We do this by taking the
complete set of transitive dependents on the resource under
consideration and removing any resources that would not be replaced by
changes to their dependencies. We determine whether or not a resource
may be replaced by substituting unknowns for input properties that may
change due to deletion of the resources their value depends on and
calling the resource provider's Diff method.

This is perhaps clearer when described by example. Consider the
following dependency graph:

  A
__|__
B   C
|  _|_
D  E F

In this graph, all of B, C, D, E, and F transitively depend on A. It may
be the case, however, that changes to the specific properties of any of
those resources R that would occur if a resource on the path to A were
deleted and recreated may not cause R to be replaced. For example, the
edge from B to A may be a simple dependsOn edge such that a change to
B does not actually influence any of B's input properties. In that case,
neither B nor D would need to be deleted before A could be deleted.

In order to make the above algorithm a reality, the resource monitor
interface has been updated to include a map that associates an input
property key with the list of resources that input property depends on.
Older clients of the resource monitor will leave this map empty, in
which case all input properties will be treated as depending on all
dependencies of the resource. This is probably overly conservative, but
it is less conservative than what we currently implement, and is
certainly correct.
2019-01-28 09:46:30 -08:00
Matt Ellis b5450d41af Use Infof instead of Infoln when we have format strings 2018-11-08 14:11:52 -08:00
Matt Ellis 6c7092ff65 Pass -count=1 to disable result caching on go 1.10 and above 2018-11-08 14:11:52 -08:00
Matt Ellis 992b048dbf Adopt golangci-lint and address issues
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
2018-11-08 14:11:47 -08:00