This change is a simple perf optimization to speed up the process of listing plugins by excluding some metadata like size by default.
Our strategy for finding a plugin is to first look on the path, and then to iterate through all plugins in the plugin cache (a directory). We do this for each plugin that is loaded when NewProvider is called. Unfortunately, the codepath that gets all plugins is shared by pulumi plugin ls that needs to do things like display the total size of all plugins, the size of each plugin, and when the plugin was last installed/last used.
This means that any time a plugin is loaded, we are computing the size of all plugins by recursively enumerating all folder (including all of the node_modules directories of any installed node multi-lang plugins!). For my 5 gb of node plugins this translated to 10s of overhead each time a plugin was loaded.
This change is a very simple fix. pulumi plugin ls is the only code path that uses size, so we create a dedicated code path GetPluginsWithMetadata that populates that info, excluding from the result of GetPlugins by default.
In Go, resource types are modeled as pointers, but there were cases where the type was not being emitted as a pointer, leading to panics and marshaling errors in programs. Additionally, array and map values that are external references were being emitted as pointers, but only resources should be pointers (not types), regardless of whether the resource type is external or local.
* Send plugin install output to stderr
We currently send plugin install output to stdout. This interferes
with --json (#5747), automation API scenarios, and in general is bad
CLI hygiene. This change sends plugin output to stdout instead.
* Add a changelog entry
Temporarily disable the new config secret warning to avoid unactionable warnings from provider `config` modules. We'll re-enable the warning when we've addressed that issue.
Following pulumi/pulumi-terraform-bridge#347, properties are generated
for all provider config matching the inputs. Unfortunately this does not
work for complex values and non-string primitives generally (not only in
bridged providers) since values are JSON serialized.
While a proper solution to this is designed, it's sufficient for now to
stop generating non-string properties, which this commit does.
This commit adds a fallback for the README definition in the generated
setup.py files for Python SDKs, thus allowing editable installs of
packages which not yet been built.
Co-authored-by: Luke Hoban <luke@pulumi.com>
The Pulumi Go SDK does not currently await all outstanding asynchronous
work associated with a Pulumi program. Because all relevant asynchronous
work is created via the Pulumi SDK, we can track this asynchronous work
and ensure that it has all completed prior to returning from
`Context.Run`.
This is complicated by the fact that many of the existing APIs that are
able to create `Output`s--`NewOutput`, `ToOutput`, `Any`, `ToSecret`,
and `All`--do not have a `*Context` parameter, and so have no
straightforward way to associate themselves with a `*Context`. To address
this, these changes add new versions of each of these APIs as methods on
`*Context`.
Despite these new methods, most Pulumi programs should work without
changes: the bulk of `Output`s are created by the SDK itself as part of
resource registration, and for `Any` and `All`, we can pick up the
context from any `Output`s present in the arguments. The only programs
that should require changes are those that create outputs from whole
cloth using `NewOutput`, `ToOutput`, or `ToSecret` and create unawaited
async work rooted at those outputs.
On an implementation level, these changes track asynchronous work using
a `sync.WaitGroup` associated with each `*Context`. This `WaitGroup` is
passed to each output associated with the context. The SDK increments
this `WaitGroup`'s count prior to starting any asynchronous work and
decrements it once the work (including any callbacks triggered by the
work) is complete.
This fixes the Go portion of #3991.
* Propagate workspace.Project metadata to plugin init
* Get to a working fix
* Propagate Root via plugin context
* Propagate root instead of yaml path
* Revert out unnecessary parameter propagation
* Root is now always absolute at this point; simplify code and docs
* Drop python conditional and propagate unused -root to all lang hosts
* Add tests that fail before and pass after
* Lint
* Add changelog entry
The Pulumi .NET SDK does not currently await all outstanding asynchronous
work associated with a Pulumi program. Because all relevant asynchronous
work is created via the Pulumi SDK, we can track this asynchronous work
and ensure that it has all completed prior to returning from
`Deployment.RunAsync`.
The implementation here is simpler than that in #6983, and re-uses the
existing support for tracking outstanding RPCs. If this proves to
negatively impact performance (which is a very real possibility for
programs that create many `Output` instances), we can simplify this
using a semaphore and a counter (essentially Go's `sync.WaitGroup`).
This fixes the .NET portion of #3991.
