This command converts an appdash trace into a pprof file for use with
`go tool pprof`. Spans are converted into stacks by sampling each root
span at a given rate and recording the stack of subspans at each sample.
These changes also replace the conditional addition of experimental and
debug commands with conditional visibility. Experimental and debug
commands will always be available, but will be hidden unless the
appropraite environment variables are set.
Co-authored-by: Levi Blackstone <levi@pulumi.com>
* Add trace proxying to fix sub-process trace collection when tracing to files
* Better func naming in test
* Avoid dealing with Windows path nightmare
* On Windows it is go.exe of course
* Rename operation to component to better align with existing trace output
This change is a simple perf optimization to speed up the process of listing plugins by excluding some metadata like size by default.
Our strategy for finding a plugin is to first look on the path, and then to iterate through all plugins in the plugin cache (a directory). We do this for each plugin that is loaded when NewProvider is called. Unfortunately, the codepath that gets all plugins is shared by pulumi plugin ls that needs to do things like display the total size of all plugins, the size of each plugin, and when the plugin was last installed/last used.
This means that any time a plugin is loaded, we are computing the size of all plugins by recursively enumerating all folder (including all of the node_modules directories of any installed node multi-lang plugins!). For my 5 gb of node plugins this translated to 10s of overhead each time a plugin was loaded.
This change is a very simple fix. pulumi plugin ls is the only code path that uses size, so we create a dedicated code path GetPluginsWithMetadata that populates that info, excluding from the result of GetPlugins by default.
* Send plugin install output to stderr
We currently send plugin install output to stdout. This interferes
with --json (#5747), automation API scenarios, and in general is bad
CLI hygiene. This change sends plugin output to stdout instead.
* Add a changelog entry
Temporarily disable the new config secret warning to avoid unactionable warnings from provider `config` modules. We'll re-enable the warning when we've addressed that issue.
The Pulumi Go SDK does not currently await all outstanding asynchronous
work associated with a Pulumi program. Because all relevant asynchronous
work is created via the Pulumi SDK, we can track this asynchronous work
and ensure that it has all completed prior to returning from
`Context.Run`.
This is complicated by the fact that many of the existing APIs that are
able to create `Output`s--`NewOutput`, `ToOutput`, `Any`, `ToSecret`,
and `All`--do not have a `*Context` parameter, and so have no
straightforward way to associate themselves with a `*Context`. To address
this, these changes add new versions of each of these APIs as methods on
`*Context`.
Despite these new methods, most Pulumi programs should work without
changes: the bulk of `Output`s are created by the SDK itself as part of
resource registration, and for `Any` and `All`, we can pick up the
context from any `Output`s present in the arguments. The only programs
that should require changes are those that create outputs from whole
cloth using `NewOutput`, `ToOutput`, or `ToSecret` and create unawaited
async work rooted at those outputs.
On an implementation level, these changes track asynchronous work using
a `sync.WaitGroup` associated with each `*Context`. This `WaitGroup` is
passed to each output associated with the context. The SDK increments
this `WaitGroup`'s count prior to starting any asynchronous work and
decrements it once the work (including any callbacks triggered by the
work) is complete.
This fixes the Go portion of #3991.
* Propagate workspace.Project metadata to plugin init
* Get to a working fix
* Propagate Root via plugin context
* Propagate root instead of yaml path
* Revert out unnecessary parameter propagation
* Root is now always absolute at this point; simplify code and docs
* Drop python conditional and propagate unused -root to all lang hosts
* Add tests that fail before and pass after
* Lint
* Add changelog entry
Config values that are objects are represented in memory as JSON strings. When a config map is being saved to a file, object values are first unmarshaled from JSON to `interface{}` and then the entire config map is marshaled to YAML (or JSON) and saved to disk. When an object value is unmarshaled from JSON, any numbers in the JSON string were being implicitly unmarshaled as `float64`, which resulted in some numbers in the nested objects being emitted in YAML using exponential notation (e.g. a number `12321123131` in an object value was being saved in the YAML as `1.2321123131e+10`). To address this, when unmarshaling the JSON for an object value, first try to unmarshal any numbers as `int64`, falling back to `float64`.
This commit makes it possible to opt out of loading plugins from PATH by
setting PULUMI_IGNORE_AMBIENT_PLUGINS to any non-empty value. This is
useful when automatic IDE tooling may build remote component plugins
into GOBIN unbeknownst to the user, and a resulting stale version of the
plugin is loaded in place of newer versions - even those , explicitly
installed.
When calling l.GetEnvVars() to evaluate whether version checking should
be skipped because of variables passed into the workspace, they have not
yet been set. This commit modifies the logic to look at the environment
variables in the options instead of in the workspace.
Since the original functionality has not yet been released, I do not
believe it warrants a CHANGELOG entry.
#6636 inadvertently changed the minimum Go version
requirement to 1.16 since ReadFile was moved in that
version. Switch back to ioutil.ReadFile to avoid forcing
an upgrade at this time.
Some YAML parsers don't correctly handle Byte-order marks,
so automatically strip it off during load.
Related to #423
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
These methods were getting promoted onto every struct that implemented
the Output interface, and are not necessary.
On a real world program, this saves 4% in binary size overall, and
15% of remaining binary size if `Apply<TypeName>` functions are
removed (https://github.com/pulumi/pulumi/issues/6592).
* [sdk/go] Cache loaded configuration files
Previously, the CLI did not cache configuration files, which
required a read from disk + unmarshalling + validation each
time a consumer needed to read one of these configurations.
This change introduces global caches for each type of Pulumi
configuration file (Project, ProjectStack, PolicyPackProject, and
PluginProject). The configuration is cached after the first request
and the cached value will be used for any subsequent operations.
Important note: The global configurations are not concurrency safe,
but this same problem exists using the previous method of
reading/writing config files on disk. Synchronization
will be added in a follow up change to allow for concurrency safe config
operations.
Fixes:#6565
As part of #6460, the logic for determing the version of the build was
moved to be a dependency on pulumictl.
Unfortunately, the homebrew installs use the "make dist" command to
build + install Pulumi to the user maching and as that would have a
dependency on pulumictl and it not existing on the user machine, it
would pass an empty version to the ldflag
This then manifested to the user as:
```
▶ pulumi version
warning: A new version of Pulumi is available. To upgrade from version '0.0.0' to '2.22.0', run
$ brew upgrade pulumi
or visit https://pulumi.com/docs/reference/install/ for manual instructions and release notes.
```
We are able to mitigate this behaviour by bringing back the get-version
script and using that script as part of the make brew installation
We can see that the versions are the same between the 2 different
installation techniques
```
make dist <------- uses pulumict
DIST:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=2.24.0-alpha.1616029310+787eb70a" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
DIST:
BUILD:
```
```
make brew <----- uses the legacy script
▶ make brew
BREW:
go install -ldflags "-X github.com/pulumi/pulumi/sdk/v2/go/common/version.Version=v2.24.0-alpha.1616029310+g787eb70a2" github.com/pulumi/pulumi/sdk/v2/dotnet/cmd/pulumi-language-dotnet
BREW:
```
A full post mortem will be carried out to ensure we mitigate these
types of errors going forward and that we are able to better test
these types of situations
When using the filestate backend (local files and cloud buckets) there is no protection to prevent two processes from managing the same stack simultaneously.
This PR creates a locks directory in the management directory that stores lock files for a stack. Each backend implementation gets its own UUID that is joined with the stack name. The feature is currently available behind the `PULUMI_SELF_MANAGED_STATE_LOCKING=1` environment variable flag.
Adds a `--limit` flag to `pulumi stack history. This allows limiting to the last few entries rather than fetching the entirety of a stack's update history (which can be quite slow for stacks with lots of updates). Example: `pulumi stack history --limit 1` fetches the last history entry only.
`stack.up` and related operations in the Automation API have been updated to consume this change, drastically reducing overhead.
* Fix resource-ref-as-ID marshaling. (#6125)
This reapplies 2f0dba23ab.
* Fix malformed resource value bug
PR #6125 introduced a bug by marshaling resource
ids as PropertyValues, but not handling that case on
the unmarshaling side. The previous code assumed
that the id was a simple string value. This bug prevents
any stack update operations (preview, update, destroy,
refresh). Since this change was already
released, we must now handle both cases in the
unmarshaling code.
* Add resource ref unit tests for the Go SDK. (#6142)
This reapplies 3d505912b8.
Co-authored-by: Pat Gavlin <pat@pulumi.com>
- Add tests that deserialize known custom and component resources
- Add tests that deserialize missing custom and component resources
These changes also add support for deserializing resources with missing
modules/packages. Such resources are deserialized as generic component,
custom, or provider resources as appropriate.
Contributes to #5943.
- Add tests that deserialize known custom and component resources
- Add tests that deserialize missing custom and component resources
These changes also add support for deserializing resources with missing
modules/packages. Such resources are deserialized as generic component,
custom, or provider resources as appropriate.
Contributes to #5943.
When marshaling a resource reference as its ID (i.e. when
opts.KeepResources is false, as it will be in the case of downlevel SDKs
and resource providers), we must take care to marshal/unmarshal an empty
ID as the unknown property value.
This includes the following changes to the resource ref APIs:
- Bifurcate resource reference creation into two methods: one for
creating references to custom resources and one for creating
references to component resources.
- Store the ID in a resource reference as a PropertyValue s.t. it can be
computed.
- Add a helper method for retrieving the ID as a string + an indicator of
whether or not the reference has an ID.
Fixes#5939.
In order to support resource construction in the Go SDK, the
engine context needs to be available in the RPC unmarshaling
code. This change adds a context parameter to the Construct and
ConstructProvider functions, and plumbs the engine context through
to the relevant calls to these functions.
