Commit graph

2886 commits

Author SHA1 Message Date
James Nugent 257c176dc0
Use variable over placeholder for Python version (#7479)
This commit modifies the generation of `setup.py` to use Python
variables as the source for the package version and plugin version
instead of placeholder strings. This has the effect of making the
packages installable via the `-e` flag directly from their source
directory rather than requiring a build step, which is useful while
developing a plugin and examples in tandem.
2021-07-14 16:44:22 -07:00
James Nugent 59a320f050
Stop producing pointers for map and slice elements (#7472)
This commit modifies Go program generation to prevent producing array
and slice object elements as pointers in args structures, which fails at
runtime and does not make sense in any case. For example, in the case of
a type defined like this in schema:

```json
"statements": {
  "type": "array",
  "items": {
    "$ref": "/aws/v4.8.0/schema.json#/types/aws:iam/getPolicyDocumentStatement:getPolicyDocumentStatement"
  }
},

```

The following (which fails at runtime) was produced before this change:

```go
Statements []*iam.GetPolicyDocumentStatement `pulumi:"statements"`
```

And the following is produced after after this change:

```go
Statements []iam.GetPolicyDocumentStatement `pulumi:"statements"`
```

Test expectations are updated accordingly.
2021-07-14 15:12:25 -07:00
James Nugent c94f923774
Fix Node.js intermediate module import path generation (#7473)
This commit fixes code generation for intermediate module paths to
produce valid TypeScript identifiers.

Before this change, the following (non-compilable) import was produced
in `./jetstack/certmanager/acme`:

```
import * as jetstack/certmanager/acme/v1alpha2 from "./jetstack/certmanager/acme/v1alpha2";
```

After this change, the following import is produced:

```
import * as v1alpha2 from "./v1alpha2";
```

This example is repeated at each level of the module tree. Test
expectations are adjusted to reflect this change.
2021-07-14 15:12:15 -07:00
James Nugent 1b78e3dd63
Respect package import alias for top-level module (#7471)
This commit modifies the Go code generator to use configured aliases for
the root package name of a Go module. This is useful when a version 2
package is present, as it prevents generation of identifiers such as
"v2" which were produced by the old logic.
2021-07-14 15:12:07 -07:00
Komal 083fc64ff5
[codegen/python] - Implement dynamic config-getters (#7447) 2021-07-14 14:26:50 -07:00
Pat Gavlin 067b630366 [codegen] Update baselines.
Two PRs that touched codegen raced and left the baselines in a bad
state.
2021-07-13 19:30:28 -07:00
Pat Gavlin 4d1ca1596b
[codegen/schema] YAML {un,}marshaling support. (#7509)
These changes add support for unmarshaling and marshaling package
schemas using YAML instead of JSON. Language-specific data is
canonically JSON. Users of the `*Spec` types will need to update the
types of the the their `Language` values to use the new
`schema.RawMessage` type instead of `json.RawMessage`: the former has
support for YAML while the latter does not.
2021-07-13 16:41:40 -07:00
Komal 0e3665f9bd
Emit To[ElementType]PtrOutput methods for go enum output types (#7499) 2021-07-13 12:54:19 -07:00
Pat Gavlin 82c631bb2b
[codegen/nodejs] Fix required input annotations. (#7497)
Ensure that required input properties that are _not_ mentioned in the
NodeJS-specific required inputs metadata are made optional.
2021-07-13 09:39:58 -07:00
Luke Hoban c3f5177f22
Fix rendering of diffs for resource without DetailedDiffs (#7500)
The diff rendering logic tests whether the DetailedDiff is nil to determine whether to use it for rendering or defer to older older supported approaches to computing diffs.

The new logic added in https://github.com/pulumi/pulumi/pull/7226 could lead to replacing a nil DetailedDiff with an empty DetailedDiff, whcih would make the rendering logic believe that a DetailedDiff was present but empty, instead of missing entirely.

This change ensures that we maintain nil-ness of the DetailedDiff when we transform it for handling of replaceOnChanges.

Also adds tests for this and other cases on applyReplaceOnChanges.

Fixes #7486.
2021-07-13 08:39:48 -06:00
Justin Van Patten 343bc4c778
Add methods lifecycle tests (#7488) 2021-07-12 15:51:11 -07:00
Emiliza Gutierrez 5acf002356
Fix typo: genrated->generated (#7478) 2021-07-12 13:44:09 -07:00
Anton Tayanovskyy 83a249cf2c
Relax errmsg check (#7482) 2021-07-12 09:05:06 -04:00
Justin Van Patten e169844652
Fix lint error (#7481) 2021-07-09 16:05:03 -07:00
James Nugent 1afb620706
Add newline to generated package.json files (#7480)
This commit adds a newline to the end of the package.json files
generated by Pulumi codegen, such that they can be installed in place
without modification.
2021-07-09 15:46:27 -07:00
Pat Gavlin 8cf588d408
[codegen] Add more type tests. (#7475)
- Add another regression test for #7454
- Add tests for singly-nested plain collections
- Fix a bug when binding {"type": "object"}.
2021-07-09 13:33:15 -07:00
Pat Gavlin 8fc47f01a5 Fix a test build break. 2021-07-09 10:41:35 -07:00
Pat Gavlin d07b325138
[codegen] Add type name generation tests. (#7470)
The inputs and expected outputs for the tests are encoded using a
schema. Each property present in the schema forms a testcase; the
expected outputs for each language are stored in each property's
`Language` field with the language name "test". Expected outputs can be
regenerated using `PULUMI_ACCEPT`.
2021-07-09 10:23:10 -07:00
Levi Blackstone 8f000c0ca1
[codegen] Refactor generateModuleContextMap (#7416)
Move ImportLanguages logic into generateModuleContextMap
rather than duplicating this work.
2021-07-09 09:52:30 -06:00
Pat Gavlin 843ce9943b
[query] Do not access config with a nil Target. (#7460)
Query mode does not fill out the Target field of EvalRunInfo before
booting its resource monitor. These changes fix a recent update to query
mode that attempts to dereference the nil Target.

Note that the missing target is by design, and other portions of the
query infrastructure have identical code to compensate.
2021-07-08 19:57:47 -07:00
Christian Nunciato bc7a88e7d3
Change resource-docs filenames to _index.md (#7450)
* Change resource-docs filenames to _index.md
* Rename test files
2021-07-08 18:07:05 -07:00
Pat Gavlin 8ec05580fd
[codegen] Fix SimplifyInputUnion. (#7459)
SimplifyInputUnion was dropping element types that were not inputs.
These changes fix that bug.

Fixes #7454.
2021-07-08 16:01:35 -07:00
Pat Gavlin 6887f76ff7
[codegen/test] Optionally skip testcases. (#7463)
Add support for skipping testcases when run against specific languages.
2021-07-08 13:54:03 -07:00
Justin Van Patten ccae7c80f8
[codegen/nodejs] Emit resource methods (#7378) 2021-07-07 20:16:42 -07:00
Justin Van Patten fd49c338f4
[codegen/go] Emit resource methods (#7438) 2021-07-07 20:11:40 -07:00
Pat Gavlin c7422228a5
[codegen/dotnet] Deeply unwrap collection element types. (#7451)
`Input{List,Map}` handle some of this themselves. This fixes a breaking
change in the .NET SDKs.

Fixes #7448.
2021-07-07 16:36:27 -07:00
Komal 8db30bdc14
[codegen/go] - Inputty Go enums (#7383) 2021-07-07 16:25:26 -07:00
Justin Van Patten 1c7cd99ecd
[codegen/python] Emit resource methods (#7364) 2021-07-07 06:57:18 -07:00
Pat Gavlin 46400d502b
[codegen] Unify SDK codegen testing (#7433)
Rather than duplicating the list of tests and codegen driver across each
SDK, move its definition into `pkg/codegen/internal/test`. This has a
few notable benefits:

- All SDK code generators will be tested against each test. Though some
  tests may exercise a particular code generator more than others, the
  extra coverage will be generally beneficial.
- Adding a new test is simpler, as only a single file needs to be
  changed.
- All SDKs now honor the `PULUMI_ACCEPT` environment variable for
  updating baselines.
- Codegen tests now validate all generated files instead of only a
  particular subset.
2021-07-06 15:40:53 -07:00
Luke Hoban eb32039013
Add replaceOnChanges resource option (#7226)
Adds a new resource option to force replacement when certain properties report changes, even if the resource provider itself does not require a replacement.

Fixes #6753.

Co-authored-by: Levi Blackstone <levi@pulumi.com>
2021-07-01 13:32:08 -06:00
Justin Van Patten 84b574f0df
Initial support for resource methods (authoring from Node.js, calling from Python) (#7363)
Adds initial support for resource methods (via a new `Call` gRPC method similar to `Invoke`), with support for authoring methods from Node.js, and calling methods from Python.
2021-06-30 07:48:56 -07:00
Komal e0902d2489
[codegen/python] - Include python enum docstrings (#7374) 2021-06-28 15:06:50 -05:00
Evan Boyle 634e97cd55
Include config secret info in Construct calls (#7358) 2021-06-24 15:38:01 -07:00
James Nugent d525a8b331
Add complete third-party Python module and fix setup.py indentation (#7362)
Co-authored-by: Komal <komalsali@gmail.com>
2021-06-24 16:01:49 -05:00
Pat Gavlin 7b1d6ec1ac
Reify Input and Optional types in the schema type system. (#7059)
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.

With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.
2021-06-24 09:17:55 -07:00
Paul Stack 414367963f
[codegen/python] Update Python codegen to use a non-conflicting property name (#7239) 2021-06-23 14:55:23 +03:00
Komal 0f97a9bdfa
Fix python module nesting (#7353) 2021-06-22 15:43:21 -07:00
Pat Gavlin 2cc89defbc
Read passphrase from the terminal when rotating. (#7347)
Rotating a passphrase requires that the old passphrase is available via
one of the `PULUMI_CONFIG_PASSPHRASE` or `PULUMI_CONFIG_PASSPHRASE_FILE`
environment variables. This confuses `readPassphrase` when reading a new
passphrase, since that function checks the aforementioned environment
variables prior to reading from the console. The overall effect is that
it is impossible to rotate the passphrase for a stack using the
passphrase provider. These changes fix this by always reading from the
console when rotating a passphrase.
2021-06-22 11:13:57 -07:00
Komal a41ed75942
Fix watch command in macos arm64 builds (#7345) 2021-06-21 14:33:30 -07:00
Adrian Smijulj 98211718ad
Watch Command - Added --path Argument (#7247) 2021-06-21 17:34:21 +10:00
Pat Gavlin 4da051a799
Add a convert-trace command. (#7319)
This command converts an appdash trace into a pprof file for use with
`go tool pprof`. Spans are converted into stacks by sampling each root
span at a given rate and recording the stack of subspans at each sample.

These changes also replace the conditional addition of experimental and
debug commands with conditional visibility. Experimental and debug
commands will always be available, but will be hidden unless the
appropraite environment variables are set.

Co-authored-by: Levi Blackstone <levi@pulumi.com>
2021-06-17 14:46:05 -07:00
Justin Van Patten 258fb00bc2
[codegen/schema] Fix intermittent failing test (#7304)
The `bad-methods-2.json` test ensures there is an error when trying to use a function as a method twice.

The schema looks like:

```
"methods": {
    "bar": "xyz:index:Foo/bar",
    "baz": "xyz:index:Foo/bar"
}
```

And the expected error is:

> function xyz:index:Foo/bar for method baz is already a method

However, when the schema is unmarshalled into a map, the keys in the map are unordered (just as JSON keys in objects are unordered), so occaisonally we'd see an error mentioning method `bar` rather than `baz`:

> function xyz:index:Foo/bar for method bar is already a method

To address this, we'll fix the portion of the code that is generating the error to walk the map in a deterministic order, and we'll also return the list of methods in the same deterministic order as well.
2021-06-16 07:14:53 -07:00
Anton Tayanovskyy 30e999ff1a
Tracing enhancements for CLI perf metrics capture (#7279)
* Allow ProgramTest Tracing flag to expand {command}

* Fix comment typos

* Memory stats collection
2021-06-15 13:25:03 -04:00
Pat Gavlin af87128fb5
Add an option to skip resource typechecking. (#7305)
More testing has found further issues with incorrect attribute shapes
generated by tf2pulumi. This knob will allow us to continue to generate
examples until those bugs are fixed.
2021-06-15 10:22:57 -07:00
Pat Gavlin 5867fa84ce
[codegen/pcl] Allow missing object properties. (#7297)
Add an option to allow missing object properties. This will prevent us
from losing examples once resource typechecking is fixed (it is
currently unintentionally disabled because the resource inputs object
type has an unexpected shape).
2021-06-15 09:09:16 -07:00
Justin Van Patten 4156df31f7
[codegen/python] Cache the package version (#7293)
Every time a resource is created (without an explicit provider version specified), `_utilities.get_version()` is called to determine the package version. This call is expensive and can measurably impact performance, especially for programs with many resources. This change caches the value so it is only determined once.
2021-06-14 10:22:57 -07:00
Anton Tayanovskyy 7ff1491397
Add trace proxying to fix sub-process trace collection into files (#7248)
* Add trace proxying to fix sub-process trace collection when tracing to files

* Better func naming in test

* Avoid dealing with Windows path nightmare

* On Windows it is go.exe of course

* Rename operation to component to better align with existing trace output
2021-06-10 22:57:18 -04:00
Komal 470be8c2b8
[cli/import] - Handle panic during import codegen (#7265) 2021-06-10 19:24:15 -07:00
Justin Van Patten 4e9e017cd2
[codegen/python] Rename conflicting ResourceArgs classes (#7171)
Python resource constructor overloads were recently added that accept a
`<Resource>Args` class for input properties, as an alternative to the
other constructor overload that accepts keyword arguments. The name of
the new args class is the name of the resource concatenated with an
`Args` suffix.

Some providers (e.g. Kubernetes, Azure Native, and Google Native) have
input types with the same name as resources in the same module, which
results in two different `<Resource>Args` classes in the same module.

When you try to use the new args class with the constructor, e.g.:

```python
pulumi_kubernetes.storage.v1.StorageClass(
            resource_name='string',
            args=pulumi_kubernetes.storage.v1.StorageClassArgs(...),
            opts=pulumi.ResourceOptions(...),
)
```

You run into an error, because
`pulumi_kubernetes.storage.v1.StorageClassArgs` is actually referring to
the existing input type rather than the intended `StorageClassArgs`
class for the constructor arguments.

Having the duplicate classes hasn't broken existing usage of the input
type because we "export" all the input types for a module _after_ all
the resources and resource args classes are exported, so the input type
just ends up "overwriting" the duplicate resource args class.

Other languages don't have this problem because the input type is either
in it's own module/namespace (e.g. Node.js and .NET) or a different name
is used for the input type (Go). But with Python, the input types and
resources are all available in the same module.

To address this for Python, when there is an input type in the same
module with the same name as the resource, the args class for the
resource will be emitted as `<Resource>InitArgs` instead of
`<Resource>Args`.
2021-06-10 10:41:49 -07:00
Justin Van Patten de871cec8e
Add schema support for methods (#7141) 2021-06-10 09:47:25 -07:00