Commit graph

18 commits

Author SHA1 Message Date
Paul Stack dc7eb5e2d2
[cli] Add ability to consume arm64 plugins from pulumi cli (#6492)
Related: #4868

Also adds the arm64 build and deployment steps via goreleaser
2021-03-10 16:28:55 +00:00
Pat Gavlin eeff5257c3
Do not read TGZs into memory. (#5983)
* Do not read TGZs into memory.

This runs a serious risk of exhausting the memory on lower-end machines
(e.g. certain CI VMs), especially given the potential size of some
plugins.

* CHANGELOG

* fixes
2020-12-20 12:54:11 -08:00
Justin Van Patten 5ab051cd97
Implement GetRequiredPlugins for Python (#5787)
Implement GetRequiredPlugins for Python, which determines the plugins
required by the program.

Also, if the `virtualenv` runtime option is set, and the specified
virtual directory is missing or empty, automatically create it and
install dependencies into it.
2020-12-03 19:22:16 -08:00
Justin Van Patten 9a707c4e03
Improve the output when installing plugins (#5808)
Two improvements:

1. Don't display "[resource plugin <foo>] installing" if the plugin is already installed.

2. Close the plugin download progress bar before displaying any subsequent output, and only show output of `npm install` when there is an error.
2020-11-21 08:02:05 -08:00
Justin Van Patten 594da1e95c
Fix plugin install failures on Windows (#5759)
When installing a plugin, previous versions of Pulumi extracted the
plugin tarball to a temp directory and then renamed the temp directory
to the final plugin directory. This was done to prevent concurrent
installs: if a process fails to rename the temp dir because the final
dir already exists, it means another process already installed the
plugin. Unfortunately, on Windows the rename operation often fails due
to aggressive virus scanners opening files in the temp dir.

In order to provide reliable plugin installs on Windows, we now extract
the tarball directly into the final directory, and use file locks to
prevent concurrent installs from toppling over one another.

During install, a lock file is created in the plugin cache directory
with the same name as the plugin's final directory but suffixed with
`.lock`. The process that obtains the lock is responsible for extracting
the tarball. Before it does that, it cleans up any previous temp
directories of failed installs of previous versions of Pulumi. Then it
creates an empty `.partial` file next to the `.lock` file. The
`.partial` file indicates an installation is in-progress. The `.partial`
file is deleted when installation is complete, indicating the plugin was
successfully installed. If a failure occurs during installation, the
`.partial` file will remain indicating the plugin wasn't fully
installed. The next time the plugin is installed, the old installation
directory will be removed and replaced with a fresh install.

This is the same approach Go uses for installing modules in its
module cache.
2020-11-16 09:44:29 -08:00
Paul Stack 186e2f059e
Respect logging verbosity as part of pulumi plugin install (#5549)
Fixes: #4427

By default, there is no indepth logging:

```
pulumi plugin install resource azure-nextgen v0.2.1
[resource plugin azure-nextgen-0.2.1] installing
Downloading plugin: 17.82 MiB / 17.82 MiB [=========================] 100.00% 2s
Moving plugin... done.
```

We can increase the logging verosity get more detailed logging:
```
pulumi plugin install resource azure-nextgen v0.2.1 -v=1 --logtostderr
[resource plugin azure-nextgen-0.2.1] installing
I1010 19:30:58.472772    7128 plugins.go:201] azure-nextgen downloading from https://get.pulumi.com/releases/plugins
Downloading plugin: 0 B / 17.82 MiB [----------------------------------]   0.00%I1010 19:30:58.704168    7128 plugin_install.go:136] [resource plugin azure-nextgen-0.2.1] installing tarball ...
Downloading plugin: 17.82 MiB / 17.82 MiB [=========================] 100.00% 3s
Moving plugin...I1010 19:31:01.874427    7128 plugins.go:304] moving plugin from "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1.tmp884796935" to "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1"
 done.
```

The most verbose logging level will log the request and response headers
```
pulumi plugin install resource azure-nextgen v0.2.1 -v=9 --logtostderr
I1010 19:29:46.989150    7089 sink.go:146] defaultSink::Infoerr([resource plugin azure-nextgen-0.2.1] installing)
[resource plugin azure-nextgen-0.2.1] installing
I1010 19:29:46.989295    7089 plugins.go:591] SelectCompatiblePlugin(..., azure-nextgen): beginning
I1010 19:29:46.989300    7089 plugins.go:630] SelectCompatiblePlugin(..., azure-nextgen): failed to find match
I1010 19:29:46.989323    7089 plugins.go:201] azure-nextgen downloading from https://get.pulumi.com/releases/plugins
I1010 19:29:46.989333    7089 plugins.go:208] full plugin download url: https://get.pulumi.com/releases/plugins/pulumi-resource-azure-nextgen-v0.2.1-darwin-amd64.tar.gz
I1010 19:29:46.989360    7089 plugins.go:218] plugin install request headers: map[User-Agent:[pulumi-cli/1 (; darwin)]]
I1010 19:29:47.242941    7089 plugins.go:225] plugin install response headers: map[Accept-Ranges:[bytes] Age:[370098] Content-Disposition:[attachment; filename=pulumi-resource-azure-nextgen-v0.2.1-darwin-amd64.tar.gz] Content-Length:[18684382] Content-Type:[application/x-gzip] Date:[Tue, 06 Oct 2020 11:41:30 GMT] Etag:["518f2e7efd46fe5b7be9508dc785b9c9-4"] Last-Modified:[Sat, 03 Oct 2020 20:57:45 GMT] Server:[AmazonS3] Via:[1.1 198b7d1bb217783eef010e6636984c9f.cloudfront.net (CloudFront)] X-Amz-Cf-Id:[aaC3Q0WcecspsFbWROLxXnD6iF7sYoAfShQKYiS2xkFKAWlFRBu-1Q==] X-Amz-Cf-Pop:[MAN50-C2] X-Cache:[Hit from cloudfront]]
Downloading plugin: 0 B / 17.82 MiB [----------------------------------]   0.00%I1010 19:29:47.243189    7089 plugin_install.go:136] [resource plugin azure-nextgen-0.2.1] installing tarball ...
Downloading plugin: 17.82 MiB / 17.82 MiB [=========================] 100.00% 2s
Moving plugin...I1010 19:29:50.031099    7089 plugins.go:304] moving plugin from "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1.tmp960784076" to "/Users/myuser/.pulumi/plugins/resource-azure-nextgen-v0.2.1"
 done.
```
2020-10-13 13:09:29 +01:00
Justin Van Patten 46c7c327dd
Install plugin dependencies (#5353)
When installing a plugin, if it contains a `PulumiPlugin.yaml` file with a `runtime` value of `nodejs` or `python`, install dependencies for the plugin.

For Node.js, `npm install` is run (or `yarn install` if `PULUMI_PREFER_YARN` is set).

For Python, a virtual environment is created and deps installed into it.
2020-09-14 13:54:26 -07:00
Lee Zen 0dac22c360
Escape the path to ensure we fetch plugins correctly (#4733) 2020-05-29 14:59:25 -07:00
Chris Smith f613b6917b
Download plugins from get.pulumi.com (#4692)
* Remove long-since deprecated --cloud-url flag

* Download plugins from get.pulumi.com

* Update CHANGELOG.md

* Use more straight forward TrimSuffix
2020-05-25 09:16:57 -07:00
Justin Van Patten 6b0a845cc1
Support publishing Python policy packs (#4644)
Adds support for publishing Python policy packs to the service, and downloading/using such policy packs when applied to stacks in an organization.
2020-05-22 15:01:15 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls (#3750)

* Switch away from native grpc impl. (#3728)

* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)

* Only retry as long as we get unavailable back.  Anything else continues. (#3769)

* Handle all errors for now. (#3781)


* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes (#4288)

Adjust C# generation

* Replace IDeployment with a sealed class (#4318)

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty (#4320)

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen (#4379)
2020-04-14 09:30:25 +01:00
evanboyle df722e974a move pkg/util/archive -> sdk/go/common/util/archive 2020-03-18 15:50:04 -07:00
evanboyle c3f6ae2451 move pkg/util/logging -> sdk/go/common/util/logging 2020-03-18 15:34:58 -07:00
evanboyle 30df499838 move pkg/version -> sdk/go/common/version 2020-03-18 15:25:25 -07:00
evanboyle 8df534a71e move pkg/diag -> sdk/go/common/diag 2020-03-18 15:09:29 -07:00
evanboyle 8a87090068 move pkg/util/httputil -> sdk/go/common/util/httputil 2020-03-18 14:45:00 -07:00
evanboyle fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
evanboyle 8fb3f428b0 move pkg/workspace -> sdk/go/common/workspace 2020-03-18 14:35:53 -07:00
Renamed from pkg/workspace/plugins.go (Browse further)