Commit graph

2402 commits

Author SHA1 Message Date
Evan Boyle 4ea1f011c6
Merge pull request #4858 from pulumi/evan/goProgramAllApplyRewrite
Go program gen: All.Apply func rewriter
2020-06-23 11:00:20 -07:00
Evan Boyle b302dbcbca
go program gen: lift expressions where prompt optionals are required (#4875) 2020-06-23 10:59:26 -07:00
Komal 1837c0a779
[codegen/python] Allow setting constant values for props (#4864)
* Allow setting constant values for props

* PR feedback
2020-06-22 10:58:13 -07:00
Komal 9a62c92efc
[codegen/python] Add missing props to casing table (#4863) 2020-06-22 09:49:34 -07:00
Levi Blackstone 096a406691
[codegen] Generate correct opts type for ComponentResources (#4853)
The docs generator previously assumed that the opts parameter
for every resource was of the CustomResource type. This is
incorrect for the YAML and Helm overlays, which are
ComponentResources. This should be handled more generally
once our schema supports ComponentResources, but this fixes
the docs for now.
2020-06-22 10:27:17 -06:00
Levi Blackstone 282c95ee40
[codegen] Use nullish coalescing for provider defaults on k8s (#4841)
For the NodeJS k8s SDK, rather than falling back to default values
using the || operator, use the nullish coalescing operator (??).
This avoid situations where the primary value is set to false,
and then is overridden by the default value.
2020-06-22 09:38:20 -06:00
Komal 14ef36553d
[codegen/python] Add whitespace to make the linter happy (#4857) 2020-06-18 17:32:52 -07:00
evanboyle 2ba1fd608a go program gen All.Apply func rewriter 2020-06-18 17:17:51 -07:00
evanboyle 7f2a83ae5a Revert "go program gen All.Apply rewriter"
This reverts commit bdd240f522.
2020-06-18 16:57:55 -07:00
evanboyle bdd240f522 go program gen All.Apply rewriter 2020-06-18 16:56:26 -07:00
Komal 63e7388879
[codegen/python] Remaining k8s updates (#4786) 2020-06-18 15:46:17 -07:00
Pat Gavlin 3e6e28af36
Fix lint errors. (#4856)
These errors appear when using golangci-lint 1.27.0.
2020-06-18 14:17:11 -07:00
Evan Boyle 8b8170252b
[Go Program Gen] multiline strings, get/lookup disambiguation, webserver example (#4850) 2020-06-18 13:34:22 -07:00
Pat Gavlin 4a903f95b9
[codegen/docs] Reimplement example extraction. (#4851)
Use the schema package's Markdown parser and walk its AST to extract
examples.

These changes also rename StripNonRelevantExamples to FilterExamples.

This is preparatory work for #4159 and #4632.
2020-06-18 12:32:15 -07:00
Paul Stack 49eb06b2f0
Merge pull request #4846 from pulumi/python-deprecation-escaping
Escape deprecation messages coming from upstream TF providers
2020-06-18 08:42:26 +01:00
Evan Boyle 9cdddd2566
[Go Program Gen] Support for EKS example (#4831) 2020-06-17 17:19:39 -07:00
Pat Gavlin 3d4c6523d2
[schema/docs] Use a Markdown parser. (#4838)
In particular, use the parser to filter and extract examples. This also
sets up support for entity references in documentation that can be used
in order to render language-specific names for resources, functions,
types, and properties.

Related to #4632 and #4159.
2020-06-17 14:02:45 -07:00
Levi Blackstone 19a113de7a
Handle invalid UTF-8 characters before RPC calls (#4816)
Several users reported cases where error messages would
cause a panic if they contained accented characters. I wasn't
able to reproduce this failure locally, but tracked down the
panic to logging gRPC calls. The Message field is typed as
a string, which requires all of the characters to be valid UTF-8.

This change runs each log string through the strings.ToValidUTF8
function, which will replace any invalid characters with the
"unknown" character. This should prevent the the logger from
panicking.
2020-06-17 13:30:59 -06:00
stack72 0a93270ffd Escape deprecation messages coming from upstream TF providers
Fixes: #4827

Diff:

```
             if encrypted is not None:
-                warnings.warn("Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"", DeprecationWarning)
+                warnings.warn("Rename PostgreSQL role resource attribute \"encrypted\" to \"encrypted_password\"", DeprecationWarning)
                 pulumi.log.warn("encrypted is deprecated: Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"")

```
2020-06-17 20:25:06 +03:00
Komal 92e44bee6d
[codegen/python] Handle extra files (#4842) 2020-06-16 19:27:06 -07:00
Evan Boyle 2d61852e4f
Go program gen: resource range, readDir, template strings, etc (#4818) 2020-06-15 23:00:02 -07:00
Komal f9074f6bcb
[codegen/python] - Exclude nested docs for k8s (#4828) 2020-06-15 16:17:05 -07:00
Komal 481750ec5b
[codegen/python] Fix edge case in relative import (#4809) 2020-06-11 15:31:39 -07:00
Justin Van Patten 78270ae3fc
Fix panic on pulumi up prompt when filtering and hitting arrow key (#4808)
When running `pulumi up`, after the preview, we prompt asking whether to proceed with the update. If you type to filter the options and then hit an arrow key a couple times, the CLI panics. This is a bug in the `survey` library we depend on. The issue has been fixed in the library upstream; this change updates our dependency.
2020-06-11 12:34:15 -07:00
Komal ea3d6aa005
[codegen/python] Don't use config for k8s (#4807) 2020-06-11 11:52:26 -07:00
Komal 5ec251ae66
[codegen/python] Fix relative import path (#4796) 2020-06-11 09:10:53 -07:00
Komal 074eb67e82
[codegen/python] Add newlines to be pep8 compliant (#4797) 2020-06-10 15:11:59 -07:00
Komal d699627ff6
[codegen/python] Fix k8s __init__.py generation (#4795)
* Fix k8s __init__.py generation

* Add comment for regex
2020-06-10 13:55:10 -07:00
Evan Boyle 45a901d3e7
Turn program gen NYIs into diagnostic errors (#4794) 2020-06-10 10:21:53 -07:00
Justin Van Patten b77ec919d4
Install and use dependencies automatically for new Python projects (#4775)
Automatically create a virtual environment and install dependencies in it with `pulumi new` and `pulumi policy new` for Python templates.

This will save a new `virtualenv` runtime option in `Pulumi.yaml` (`PulumiPolicy.yaml` for policy packs):

```yaml
runtime:
  name: python
  options:
    virtualenv: venv
```

`virtualenv` is the path to a virtual environment that Pulumi will use when running `python` commands.

Existing projects are unaffected and can opt-in to using this by setting `virtualenv`, otherwise, they'll continue to work as-is.
2020-06-09 16:42:53 -07:00
Komal 238adf2f2f
[codegen/python] Allow import path overrides (#4787)
* Allow import path overrides

* Fix typo

* Rename ModuleToPackage to ModuleNameOverrides
2020-06-09 14:33:02 -07:00
Komal b93259c6bb
[codegen/python] Fix readme generation (#4759)
* Fix readme generation

* Allow readme text to be passed in with the package.
2020-06-08 13:07:56 -07:00
Komal 6c29b67c03
[codegen/python] Fix doc generation in python codegen (#4757) 2020-06-08 12:44:57 -07:00
Pat Gavlin 9754465b04
[codegen/nodejs/sdk] Export extra files. (#4766)
If extra files are present, make sure that appropriate modules are
created as necessary and that extra sources are exported from each
module's `index.ts`.
2020-06-07 09:21:09 -07:00
Evan Boyle 94cd67b564
[Go Program Gen] Improved type handling and invokes (#4749) 2020-06-05 18:52:00 -07:00
Levi Blackstone 7254779b28
[codegen/nodejs] Fix k8s index.ts generation (#4756) 2020-06-05 16:41:07 -06:00
Levi Blackstone 875c4dab89
[codegen/nodejs] Fix remaining issues with k8s codegen (#4325) 2020-06-05 11:11:18 -06:00
Pat Gavlin e6fb3ef0f8
[codegen/nodejs/sdk] Fix token type config getter. (#4753)
If the type of a configuration variable is a token type (i.e. a type
provided by an overlay) with the underlying type string, call
`config.get` rather than `config.getObject`. This fixes a regression in
the `aws.config.region` variable.
2020-06-04 14:51:21 -07:00
Paul Stack 58002bae62
Ensuring a property level deprecation warning is set for python codegen (#4744) 2020-06-04 11:29:23 +01:00
Levi Blackstone 8a7e10d37a
[codegen/nodejs] Refactor GeneratePackage method (#4737)
- Split out common functionality into generateModuleContextMap method
- Add LanguageResource and LanguageProperty structs for use by downstream
codegen tools
- Add LanguageResources function for use by downstream codegen tools
2020-06-03 19:20:32 -06:00
Pat Gavlin bd1c0e6d7b
[codegen/schema] Support default union types. (#4746)
Add support for union types that indicate a default type for targets
that do not support unions, or do not support unions in certain
positions (e.g. output properties). The NodeJS backend makes use of this
in combination with a new flag, `disableUnionOutputTypes`, to avoid
generating unions in output types.

These changes also refactor the various module ->
package/module/namespace mapping methods so that these entities can be
fetched by their language name rather than their token.
2020-06-02 18:15:21 -07:00
Pat Gavlin d77cda98fc
[codegen/hcl2] Fixes for RewriteConversions. (#4743)
- Typecheck in all cases where a type may have changed
- Do not perform literal conversions if the type is already correct
- Perform literal conversions before checking to see if a call to
  `__convert` is required. This catches cases such as string literals
  passed where ints are required. Without this change, that form in
  particular generates a bare number literal rather than a number
  literal wrapped in a `__convert`.
2020-06-02 12:00:35 -07:00
Lee Briggs bc05e2c955
improve dev version detection (#4732)
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2020-06-02 11:51:04 +01:00
Pat Gavlin 962fe01734
[codegen/nodejs/sdk] Do not write reserved files. (#4742)
If the source file for a resource or function definition would collide
with a reserved filename (e.g. `index.ts`), append an underscore to its
filename (e.g. `index_.ts`).
2020-06-01 16:24:21 -07:00
Pat Gavlin 726d43a5bd
[codegen/nodejs/sdk] Split required {in,out}puts. (#4736)
These changes are necessary in order to move tfgen over to the schema
code generator.

The schema generator for tfgen has an annoying behavior for nested types
in which it does not separate input and output types. Worse, the shape
of the type that results from a collision of input and output types is
order-dependent: whichever of the two was observed last wins (note that
the shape is still determenistic, as the order in which types are
recorded is always the same). As a result, the NodeJS code generator
needs to know the set of required properties for the other aspect of the
type in order to generate proper code.

Contributes to
https://github.com/pulumi/pulumi-terraform-bridge/issues/179.
2020-06-01 11:50:10 -07:00
Levi Blackstone 9b980c14f2
Address NodeJS codegen feedback (#4734)
Follow up on feedback for https://github.com/pulumi/pulumi/pull/4730
2020-05-29 16:35:42 -06:00
Levi Blackstone 3fd51da719
Add k8s compatibility mode for NodeJS codegen (#4730) 2020-05-29 13:39:29 -06:00
Levi Blackstone d0388e5525
Only generate top-level README in NodeJS packages (#4729) 2020-05-29 13:36:16 -06:00
Lee Zen 4ff083639a
Clarify backend login help text (#4726) 2020-05-29 10:10:12 -07:00
Mikhail Shilkov cf8b6c1237
Convert primitive types in C# program gen (#4686) 2020-05-29 12:38:53 +02:00
Pat Gavlin 75c3ccf53d
[codegen/nodejs/sdk] Minor fixes. (#4725)
- Only wrap field types in `pulumi.Input<>` in types that are part of
  the input SDK.
- Ensure that READMEs have a trailing newline.

Contributes to https://github.com/pulumi/pulumi-terraform-bridge/issues/179.
2020-05-28 16:24:48 -07:00
Levi Blackstone a8486520a3
Update NodeJS codegen to use codegen.StringSet (#4724) 2020-05-28 17:22:50 -06:00
Levi Blackstone 754a4ae51d
Fix NodeJS codegen to handle recursive types (#4723)
Track the input/output types that have been encountered
and don't recurse into previously seen types.
2020-05-28 16:37:39 -06:00
Evan Boyle 9b2c6d726d
[Go Program Gen] binary, unary, and conditional expression support (#4716) 2020-05-28 13:47:49 -07:00
Paul Stack 45e1917a30
Upgrade to Go 1.14.x (#4697) 2020-05-28 12:01:33 +01:00
Komal e2ba2a067c
Fix typo in nodejs importer (#4711) 2020-05-27 08:58:27 -07:00
Evan Boyle c8142677cc
expression tests for basic go program literals (#4709) 2020-05-27 07:32:13 -07:00
Mikhail Shilkov b6f61b45ad
Don't mark provider token string types with json attribute in C# (#4658)
Don't mark provider token string types with json attribute in C#
2020-05-27 10:14:38 +02:00
Mikhail Shilkov c66192130a
Small bug fixes in C# codegen (#4702) 2020-05-26 19:12:56 +02:00
Pat Gavlin e973ec0491
[codegen/hcl2] Fix assignability from dynamic. (#4704)
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.

This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:

```
if t.AssignableFrom(model.DynamicType) {
}
```

Fixes #4703.
2020-05-26 09:05:42 -07:00
Pat Gavlin cb8066d74a
[codegen/hcl2] Fix ForExpression.Evaluate. (#4701)
This expression was not evaluated correctly when producing a dictionary.
2020-05-26 09:05:35 -07:00
Pat Gavlin 05032d6850 [codegen/hcl2] Fix assignability from dynamic.
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.

This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:

```
if t.AssignableFrom(model.DynamicType) {
}
```

Fixes #4703.
2020-05-26 08:21:09 -07:00
Evan Boyle ea114b04ee
output generation for s3 logging example (#4681) 2020-05-26 07:07:54 -07:00
Chris Smith f613b6917b
Download plugins from get.pulumi.com (#4692)
* Remove long-since deprecated --cloud-url flag

* Download plugins from get.pulumi.com

* Update CHANGELOG.md

* Use more straight forward TrimSuffix
2020-05-25 09:16:57 -07:00
Justin Van Patten 6b0a845cc1
Support publishing Python policy packs (#4644)
Adds support for publishing Python policy packs to the service, and downloading/using such policy packs when applied to stacks in an organization.
2020-05-22 15:01:15 -07:00
Sean Holung c538db64c8
Anchoring properties (#4488) 2020-05-22 14:53:34 -07:00
Pat Gavlin 5330c97684
[codegen/hcl2/model] Remove type caches. (#4683) 2020-05-22 10:47:34 +01:00
Mikhail Shilkov 7e3b60b3d1
Fix C# (and a bit of Node.js) program gen based on errors from AWS samples (#4673) 2020-05-22 08:46:25 +02:00
Lee Zen 8bc3d0e906
Fix single-quote examples in command help (#4680) 2020-05-21 14:27:30 -07:00
Evan Boyle 8dd329f57f
Go basic program gen (#4667) 2020-05-21 10:23:33 -07:00
Lee Briggs afb9456aa8
Remove replace for gocloud.dev 2020-05-19 08:09:58 -07:00
Lee Briggs 362fc8d7f2
Merge branch 'master' into feature/go-cloud-azurekeyvault-regions 2020-05-19 08:07:09 -07:00
Mikhail Shilkov adfa8116fb
Introduce k8s compatibility mode to C# codegen (#4544)
Introduce k8s compatibility mode to C# codegen
2020-05-19 11:41:06 +02:00
Dan Hernandez a46731f36c
Include case when attempting to escape the . in a name. (#4513) 2020-05-19 10:11:58 +01:00
Mikhail Shilkov f66100ce28
Initial support for C# program gen (#4611)
Initial support for C# program gen
2020-05-19 10:18:38 +02:00
Pat Gavlin 088751a4a8
[codegen/hcl2] Refactor package schema loading. (#4661)
- Move the implementation of loadPackageSchema into a method on
  PackageCache
- Protect the cache with synchronization primitives to enable
  concurrency in downstream consumers
- Use jsoniter to deserialize schemas
2020-05-18 17:35:11 -07:00
Lee Briggs 72766fd4e6
Use latest upstream of gocloud.dev 2020-05-18 07:43:41 -07:00
Lee Briggs e045c2ac71
# This is a combination of 2 commits.
# This is the 1st commit message:

Use patched azure gocloud library

Fixes #4642

# The commit message #2 will be skipped:

# sync with pulumi-master branch
2020-05-18 07:43:20 -07:00
Praneet Loke 30b12cff49
[codegen/docs] Various Go-related fixes (#4646)
* Remove code that was hiding Go as a language option for k8s overlay resources. Mark input args type for Functions as optional in Go.

* Show a special note in the Go function's snippet when the function name does not match other languages.

* Make sure the draft PR stays assigned to the original author. Add some more comments.

* Remove logic to use the Go ModuleToPackage map to then lookup the relevant C# namespace. Make getLanguageModuleName a method of modContext since passing in a package is not needed anymore.
2020-05-15 17:33:56 -07:00
Mikhail Shilkov 5b410ec440
Resolve referenced packages for every TF program (#4636) 2020-05-15 19:18:07 +02:00
Sean Holung f5ed6190ba
Link Python and Go constructors (#4593) 2020-05-15 09:51:24 -07:00
Pat Gavlin e9dd0e5e69
[codegen/schema] Path unescape type tokens in refs (#4641)
Some type tokens contain characters that are not valid in a single
URL path element (`/` in particular). These type tokens may be
path-escaped, and should be unescaped before they are interpreted.
2020-05-14 13:35:57 -07:00
Evan Boyle 0d257bb32d
go gen_program shell + test + sample (#4630) 2020-05-14 08:43:12 -07:00
Pat Gavlin 8f0a002a26
Generate deprecation comments in Go. (#4631)
This is consistent with the convention for marking members as
deprecated: https://github.com/golang/go/wiki/Deprecated.
2020-05-13 17:12:59 -07:00
Pat Gavlin 99852bc18e
[codegen/nodejs] SDK generator fixes. (#4618)
- Add support for constant values in unions (e.g. `"foo" | "bar"`)
- Generate `Input<T | U>` instead of `Input<T> | Input<U>`
- Emit deprecation messages for resources
- Handle nil state inputs
- Allow packages to provide a README
- Remove sync invoke support

These changes are part of
https://github.com/pulumi/pulumi-terraform-bridge/issues/179, and
restore functional parity with the current `tfgen` generator.
2020-05-13 10:55:37 -07:00
Praneet Loke 54f5a0ac36
[codegen/docs] Use the C# namespaces override map for links. (#4616) 2020-05-13 09:32:40 -07:00
Pat Gavlin 647b6627a2
[codegen/hcl2] Fix descent in RewriteConversions. (#4614)
Descend into anonymous function expressions, the operands to conditional
expressions, and the value in for expressions.
2020-05-13 08:25:26 -07:00
Chris Smith 71910a1c52
Add PULUMI_SKIP_CONFIRMATIONS env var (#4477) 2020-05-13 14:01:02 +01:00
Praneet Loke fcac4af8ae
[codegen/docs] Generate a table with links to the language-specific module pages (#4597)
* Generate language package details in the index pages.

* Add a new DocLanguageHelper interface method to get to a module per-language. 

* Add a new workflow file for automatically creating draft docs PRs for previewing resource docs for AWS and Kubernetes as a result of changes in the resource docs generator.
2020-05-12 15:37:37 -07:00
Pat Gavlin 33258326e0
[codegen/hcl2] Add a conversion insertion pass. (#4594)
Add a rewriter that reifies implicit conversions into a call to the
`__convert` intrinsic. Code generators can recognize this intrinsic and
use it to generate appropriate conversion code.

Part of this work involves redesigning the type annotations system.
Annotations are now only applicable to opaque and object types. Instead
of inspecting annotations directly, code generators should use
`hcl2.GetSchemaForType` to extract the `schema.Type` for a `model.Type`.
2020-05-11 11:17:36 -07:00
Paul Stack 48f906e026
Allow pulumi stack export to decrypt secrets (#4046) 2020-05-11 19:16:30 +01:00
Pat Gavlin 255620ea1e
[codegen/nodejs] Await invokes in async contexts. (#4598)
If we are generating code into an async context (e.g. an async main),
await calls to invoke rather than leaving them as promises. This results
in more idiomatic code withing such contexts.
2020-05-11 10:21:56 -07:00
Paul Stack d9e01da0aa
Merge pull request #4586 from pulumi/stack72/gh-4585
Ensure Go build is an optional step in integration testing framework
2020-05-09 01:29:28 +01:00
Praneet Loke aa313aecf8
[codegen/docs] Add a new template for examples section (#4591)
* Add a new template for examples section. Extract the examples section into a structured format for custom template processing.

* Update the description IFF we were able to extract examples sections from it.

* Update doc comments and add missing file header for the newly added file.

* Make the example description readable. Add a check for empty example sections.

* Add a chooser right below the Example Usage header. Remove javascript as a language.

* Allow an empty new-line between short-codes boundaries.
2020-05-08 16:25:28 -07:00
Justin Van Patten d43a0dd3f8
Update NPM test to install @pulumi/pulumi 2.0 (#4596)
Future proof this test by depending on `@pulumi/pulumi` 2.0. `@pulumi/pulumi` 1.x depends on the native `grpc` package. In the future, when we run on future versions of Node.js, prebuilt native binaries of `grpc` might not be available which could result in this test failing.

In `@pulumi/pulumi` 2.0, we have moved to using `@grpc/grpc-js`, a pure JavaScript implementation, which doesn’t have this problem.
2020-05-08 11:15:43 -07:00
Praneet Loke 5613867b3b
[codegen/docs] Add apiextensions as a k8s overlay resource (#4583)
* Add apiextensions as a k8s overlay. Add comment.
2020-05-08 11:13:21 -07:00
Pat Gavlin fc00c16e31
[codegen/hcl2] Fix iterator apply rewrites. (#4589)
There are two expressions in HCL2 that are used to iterate over
collections:

- Splat expressions, e.g. `foo.*.bar`, and
- For expressions, e.g. `[for v in foo: v.bar]`

In both of these cases, the parts of the expression that are not the
collection being iterated behave like callbacks, and must be treated as
such by the apply rewriter.
2020-05-07 12:15:40 -07:00
Pat Gavlin 8fded5f29f
[codegen/hcl2] Fix template control tokens. (#4584)
Token detection was broken for conditional and for expressions that
represent template control sequences. The code originally attempted to
determine whether or not a conditional or for expression was a control
sequence by inspecting the expression's parent. Unfortunately, that
approach is unable to distinguish between expressions that are control
sequences and those that are merely template parts. These changes
instead inspect the first token of the expression for a template control
token (i.e. `%{`): if such a token is found, the expression is detected
as a template control sequence.
2020-05-07 09:12:00 -07:00
stack72 e701e24151 Ensure Go build is an optional step in integration testing framework
Fixes: #4585

This was happening by default in the test framework and then again
in the go language runtime

Also we were replacing a module and then running go mod tidy - this
is unnecessary - go mod download is sufficient here to ensure we
have the correct dependency we need. The tidy may have been removing
the replacements added
2020-05-07 09:23:05 +01:00
Praneet Loke 3b9ec6590b
[codegen/docs] Fix kubernetes resource docs links for overlay resources (#4578)
* Adjust the args type names for k8s overlay resources.

* Revert to using input type link for kubernetes non-overlay resources.

* Use the correct module name to lookup the C# namespace for resources that belong to a module.

* Helm's arg types don't use the version number.
2020-05-06 12:11:31 -07:00