Commit graph

486 commits

Author SHA1 Message Date
Komal
56fa087bd6
[codegen/python] pep-8 compliant spacing (#4958)
* pep-8 compliant spacing

* PR fixes
2020-07-07 11:32:48 -07:00
Paul Stack
0e9b865f68
Adding the missing providers to the docs generation titleLookup Map (#4970) 2020-07-06 15:24:51 +01:00
Levi Blackstone
d611740ab0
[codegen] Set additionalSecretOutputs for secret props (#4956) 2020-07-02 13:30:10 -06:00
Lee Briggs
af2580c10a
Fix incorrect packagespec field name 2020-07-02 09:40:31 -07:00
Lee Briggs
c25cbf0fc8
switch terminology to plugin download URL
After a discussion about terminology, the better phrasing is to use
pluginDownloadURL
2020-07-02 09:29:19 -07:00
Lee Briggs
f310c21b85
Add providerURL to package definition 2020-07-01 19:10:25 -07:00
Levi Blackstone
137f39151b
[codegen] Add Secret field to Property schema (#4943) 2020-07-01 14:17:49 -06:00
Levi Blackstone
956d362d8b
[codegen] Don't log deprecation warnings in k8s SDKs (#4942)
The k8s SDK logs deprecations warnings as part of the
provider logic, and includes a flag to suppress these
warnings. Including them in the SDKs breaks this logic.
2020-07-01 13:50:26 -06:00
Mikhail Shilkov
de2a6500cd
Do not import non-existing inputs or outputs in Node.js codegen (#4937)
Do not import non-existing inputs or outputs in Node.js codegen
2020-07-01 21:47:46 +02:00
Levi Blackstone
92cb042aee
[codegen/dotnet] Add LanguageResource for downstream codegen (#4932) 2020-06-30 16:19:21 -06:00
Pat Gavlin
a8a20ecb4b
[codegen/*] Add support for explicit secrets. (#4927)
- Add a new builtin function, secret, that marks its input value as
  secret
- Add support for this function to the various code generators

Fixes #4924.
2020-06-30 11:35:24 -07:00
Mikhail Shilkov
bb358c4d21
Escape \x in Python comments (#4922) 2020-06-30 11:19:45 +02:00
Evan Boyle
395a68e0f6
[Go program gen]: namespaceless resources (#4915) 2020-06-29 18:10:34 -07:00
Pat Gavlin
69ba47cff2
[codegen/*] Add support for resource options. (#4925)
The PCL binder has supported resource options for some time, but these
options haven't been used or processed by the various code generators.
These options--particularly the parent and provider options0--are
critical for import codegen. These changes implement the basic set of
options, and add a note about fleshing out the rest as necessary.

One component of these changes is a new rewriter that rewrites property
references into property paths that are understood by the Pulumi engine.
This rewriter is used to preprocess the contents of the `ignoreChanges`
resource option.

These changes also hack around a weakness in the HCL2 type system:
In Go, references to resources should be typed as `hcl2.ResourceType`.
Unfortunately, this breaks the existing collection semantics associated
with resources. Because of this, the Go code generator does not have
enough information to know that it should generate a `[]pulumi.Resource`
for lists of resources. These changes hack around that limitation using
a Go-specific opaque type and some hardcoded comparisons in
`argumentTypeName`.

Fixes #4923.
2020-06-29 16:33:52 -07:00
Evan Boyle
91828b4310
[Go Program Gen] Improved handling for pulumi.Map types (#4914) 2020-06-29 15:29:41 -07:00
Mikhail Shilkov
14a3871346
Avoid stack overflow in Python codegen in case of recursive types (#4908) 2020-06-26 20:47:13 +02:00
Komal
3eacf8bf90
[codegen/python] Only add item to map if it doesn't already exist. (#4907) 2020-06-26 09:05:36 -07:00
Komal
b35a94cac6
[codegen/python] - Exclude single word properties from casing tables. (#4895)
Co-authored-by: Levi Blackstone <levi@pulumi.com>
2020-06-25 11:51:57 -07:00
Komal
93fa20a44b
[codegen/python] - Only import json when needed (#4892) 2020-06-25 09:59:00 -07:00
Evan Boyle
fa3000801e
skip transient test (#4886) 2020-06-24 13:53:18 -07:00
Komal
ed752bc384
[codegen/python] Don't use __all__ (#4873) 2020-06-24 11:41:03 -07:00
Evan Boyle
31770c3300
go program gen: prompt array conversion, unused range vars, id handling (#4884) 2020-06-24 11:07:26 -07:00
Komal
669448ab40
[codegen/python] Use CamelCase filenames for k8s. (#4878) 2020-06-23 13:02:22 -07:00
Evan Boyle
4ea1f011c6
Merge pull request #4858 from pulumi/evan/goProgramAllApplyRewrite
Go program gen: All.Apply func rewriter
2020-06-23 11:00:20 -07:00
Evan Boyle
b302dbcbca
go program gen: lift expressions where prompt optionals are required (#4875) 2020-06-23 10:59:26 -07:00
Komal
1837c0a779
[codegen/python] Allow setting constant values for props (#4864)
* Allow setting constant values for props

* PR feedback
2020-06-22 10:58:13 -07:00
Komal
9a62c92efc
[codegen/python] Add missing props to casing table (#4863) 2020-06-22 09:49:34 -07:00
Levi Blackstone
096a406691
[codegen] Generate correct opts type for ComponentResources (#4853)
The docs generator previously assumed that the opts parameter
for every resource was of the CustomResource type. This is
incorrect for the YAML and Helm overlays, which are
ComponentResources. This should be handled more generally
once our schema supports ComponentResources, but this fixes
the docs for now.
2020-06-22 10:27:17 -06:00
Levi Blackstone
282c95ee40
[codegen] Use nullish coalescing for provider defaults on k8s (#4841)
For the NodeJS k8s SDK, rather than falling back to default values
using the || operator, use the nullish coalescing operator (??).
This avoid situations where the primary value is set to false,
and then is overridden by the default value.
2020-06-22 09:38:20 -06:00
Komal
14ef36553d
[codegen/python] Add whitespace to make the linter happy (#4857) 2020-06-18 17:32:52 -07:00
evanboyle
2ba1fd608a go program gen All.Apply func rewriter 2020-06-18 17:17:51 -07:00
evanboyle
7f2a83ae5a Revert "go program gen All.Apply rewriter"
This reverts commit bdd240f522.
2020-06-18 16:57:55 -07:00
evanboyle
bdd240f522 go program gen All.Apply rewriter 2020-06-18 16:56:26 -07:00
Komal
63e7388879
[codegen/python] Remaining k8s updates (#4786) 2020-06-18 15:46:17 -07:00
Evan Boyle
8b8170252b
[Go Program Gen] multiline strings, get/lookup disambiguation, webserver example (#4850) 2020-06-18 13:34:22 -07:00
Pat Gavlin
4a903f95b9
[codegen/docs] Reimplement example extraction. (#4851)
Use the schema package's Markdown parser and walk its AST to extract
examples.

These changes also rename StripNonRelevantExamples to FilterExamples.

This is preparatory work for #4159 and #4632.
2020-06-18 12:32:15 -07:00
Paul Stack
49eb06b2f0
Merge pull request #4846 from pulumi/python-deprecation-escaping
Escape deprecation messages coming from upstream TF providers
2020-06-18 08:42:26 +01:00
Evan Boyle
9cdddd2566
[Go Program Gen] Support for EKS example (#4831) 2020-06-17 17:19:39 -07:00
Pat Gavlin
3d4c6523d2
[schema/docs] Use a Markdown parser. (#4838)
In particular, use the parser to filter and extract examples. This also
sets up support for entity references in documentation that can be used
in order to render language-specific names for resources, functions,
types, and properties.

Related to #4632 and #4159.
2020-06-17 14:02:45 -07:00
stack72
0a93270ffd Escape deprecation messages coming from upstream TF providers
Fixes: #4827

Diff:

```
             if encrypted is not None:
-                warnings.warn("Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"", DeprecationWarning)
+                warnings.warn("Rename PostgreSQL role resource attribute \"encrypted\" to \"encrypted_password\"", DeprecationWarning)
                 pulumi.log.warn("encrypted is deprecated: Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"")

```
2020-06-17 20:25:06 +03:00
Komal
92e44bee6d
[codegen/python] Handle extra files (#4842) 2020-06-16 19:27:06 -07:00
Evan Boyle
2d61852e4f
Go program gen: resource range, readDir, template strings, etc (#4818) 2020-06-15 23:00:02 -07:00
Komal
f9074f6bcb
[codegen/python] - Exclude nested docs for k8s (#4828) 2020-06-15 16:17:05 -07:00
Komal
481750ec5b
[codegen/python] Fix edge case in relative import (#4809) 2020-06-11 15:31:39 -07:00
Komal
ea3d6aa005
[codegen/python] Don't use config for k8s (#4807) 2020-06-11 11:52:26 -07:00
Komal
5ec251ae66
[codegen/python] Fix relative import path (#4796) 2020-06-11 09:10:53 -07:00
Komal
074eb67e82
[codegen/python] Add newlines to be pep8 compliant (#4797) 2020-06-10 15:11:59 -07:00
Komal
d699627ff6
[codegen/python] Fix k8s __init__.py generation (#4795)
* Fix k8s __init__.py generation

* Add comment for regex
2020-06-10 13:55:10 -07:00
Evan Boyle
45a901d3e7
Turn program gen NYIs into diagnostic errors (#4794) 2020-06-10 10:21:53 -07:00
Komal
238adf2f2f
[codegen/python] Allow import path overrides (#4787)
* Allow import path overrides

* Fix typo

* Rename ModuleToPackage to ModuleNameOverrides
2020-06-09 14:33:02 -07:00
Komal
b93259c6bb
[codegen/python] Fix readme generation (#4759)
* Fix readme generation

* Allow readme text to be passed in with the package.
2020-06-08 13:07:56 -07:00
Komal
6c29b67c03
[codegen/python] Fix doc generation in python codegen (#4757) 2020-06-08 12:44:57 -07:00
Pat Gavlin
9754465b04
[codegen/nodejs/sdk] Export extra files. (#4766)
If extra files are present, make sure that appropriate modules are
created as necessary and that extra sources are exported from each
module's `index.ts`.
2020-06-07 09:21:09 -07:00
Evan Boyle
94cd67b564
[Go Program Gen] Improved type handling and invokes (#4749) 2020-06-05 18:52:00 -07:00
Levi Blackstone
7254779b28
[codegen/nodejs] Fix k8s index.ts generation (#4756) 2020-06-05 16:41:07 -06:00
Levi Blackstone
875c4dab89
[codegen/nodejs] Fix remaining issues with k8s codegen (#4325) 2020-06-05 11:11:18 -06:00
Pat Gavlin
e6fb3ef0f8
[codegen/nodejs/sdk] Fix token type config getter. (#4753)
If the type of a configuration variable is a token type (i.e. a type
provided by an overlay) with the underlying type string, call
`config.get` rather than `config.getObject`. This fixes a regression in
the `aws.config.region` variable.
2020-06-04 14:51:21 -07:00
Paul Stack
58002bae62
Ensuring a property level deprecation warning is set for python codegen (#4744) 2020-06-04 11:29:23 +01:00
Levi Blackstone
8a7e10d37a
[codegen/nodejs] Refactor GeneratePackage method (#4737)
- Split out common functionality into generateModuleContextMap method
- Add LanguageResource and LanguageProperty structs for use by downstream
codegen tools
- Add LanguageResources function for use by downstream codegen tools
2020-06-03 19:20:32 -06:00
Pat Gavlin
bd1c0e6d7b
[codegen/schema] Support default union types. (#4746)
Add support for union types that indicate a default type for targets
that do not support unions, or do not support unions in certain
positions (e.g. output properties). The NodeJS backend makes use of this
in combination with a new flag, `disableUnionOutputTypes`, to avoid
generating unions in output types.

These changes also refactor the various module ->
package/module/namespace mapping methods so that these entities can be
fetched by their language name rather than their token.
2020-06-02 18:15:21 -07:00
Pat Gavlin
d77cda98fc
[codegen/hcl2] Fixes for RewriteConversions. (#4743)
- Typecheck in all cases where a type may have changed
- Do not perform literal conversions if the type is already correct
- Perform literal conversions before checking to see if a call to
  `__convert` is required. This catches cases such as string literals
  passed where ints are required. Without this change, that form in
  particular generates a bare number literal rather than a number
  literal wrapped in a `__convert`.
2020-06-02 12:00:35 -07:00
Pat Gavlin
962fe01734
[codegen/nodejs/sdk] Do not write reserved files. (#4742)
If the source file for a resource or function definition would collide
with a reserved filename (e.g. `index.ts`), append an underscore to its
filename (e.g. `index_.ts`).
2020-06-01 16:24:21 -07:00
Pat Gavlin
726d43a5bd
[codegen/nodejs/sdk] Split required {in,out}puts. (#4736)
These changes are necessary in order to move tfgen over to the schema
code generator.

The schema generator for tfgen has an annoying behavior for nested types
in which it does not separate input and output types. Worse, the shape
of the type that results from a collision of input and output types is
order-dependent: whichever of the two was observed last wins (note that
the shape is still determenistic, as the order in which types are
recorded is always the same). As a result, the NodeJS code generator
needs to know the set of required properties for the other aspect of the
type in order to generate proper code.

Contributes to
https://github.com/pulumi/pulumi-terraform-bridge/issues/179.
2020-06-01 11:50:10 -07:00
Levi Blackstone
9b980c14f2
Address NodeJS codegen feedback (#4734)
Follow up on feedback for https://github.com/pulumi/pulumi/pull/4730
2020-05-29 16:35:42 -06:00
Levi Blackstone
3fd51da719
Add k8s compatibility mode for NodeJS codegen (#4730) 2020-05-29 13:39:29 -06:00
Levi Blackstone
d0388e5525
Only generate top-level README in NodeJS packages (#4729) 2020-05-29 13:36:16 -06:00
Mikhail Shilkov
cf8b6c1237
Convert primitive types in C# program gen (#4686) 2020-05-29 12:38:53 +02:00
Pat Gavlin
75c3ccf53d
[codegen/nodejs/sdk] Minor fixes. (#4725)
- Only wrap field types in `pulumi.Input<>` in types that are part of
  the input SDK.
- Ensure that READMEs have a trailing newline.

Contributes to https://github.com/pulumi/pulumi-terraform-bridge/issues/179.
2020-05-28 16:24:48 -07:00
Levi Blackstone
a8486520a3
Update NodeJS codegen to use codegen.StringSet (#4724) 2020-05-28 17:22:50 -06:00
Levi Blackstone
754a4ae51d
Fix NodeJS codegen to handle recursive types (#4723)
Track the input/output types that have been encountered
and don't recurse into previously seen types.
2020-05-28 16:37:39 -06:00
Evan Boyle
9b2c6d726d
[Go Program Gen] binary, unary, and conditional expression support (#4716) 2020-05-28 13:47:49 -07:00
Komal
e2ba2a067c
Fix typo in nodejs importer (#4711) 2020-05-27 08:58:27 -07:00
Evan Boyle
c8142677cc
expression tests for basic go program literals (#4709) 2020-05-27 07:32:13 -07:00
Mikhail Shilkov
b6f61b45ad
Don't mark provider token string types with json attribute in C# (#4658)
Don't mark provider token string types with json attribute in C#
2020-05-27 10:14:38 +02:00
Mikhail Shilkov
c66192130a
Small bug fixes in C# codegen (#4702) 2020-05-26 19:12:56 +02:00
Pat Gavlin
e973ec0491
[codegen/hcl2] Fix assignability from dynamic. (#4704)
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.

This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:

```
if t.AssignableFrom(model.DynamicType) {
}
```

Fixes #4703.
2020-05-26 09:05:42 -07:00
Pat Gavlin
cb8066d74a
[codegen/hcl2] Fix ForExpression.Evaluate. (#4701)
This expression was not evaluated correctly when producing a dictionary.
2020-05-26 09:05:35 -07:00
Pat Gavlin
05032d6850 [codegen/hcl2] Fix assignability from dynamic.
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.

This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:

```
if t.AssignableFrom(model.DynamicType) {
}
```

Fixes #4703.
2020-05-26 08:21:09 -07:00
Evan Boyle
ea114b04ee
output generation for s3 logging example (#4681) 2020-05-26 07:07:54 -07:00
Sean Holung
c538db64c8
Anchoring properties (#4488) 2020-05-22 14:53:34 -07:00
Pat Gavlin
5330c97684
[codegen/hcl2/model] Remove type caches. (#4683) 2020-05-22 10:47:34 +01:00
Mikhail Shilkov
7e3b60b3d1
Fix C# (and a bit of Node.js) program gen based on errors from AWS samples (#4673) 2020-05-22 08:46:25 +02:00
Evan Boyle
8dd329f57f
Go basic program gen (#4667) 2020-05-21 10:23:33 -07:00
Mikhail Shilkov
adfa8116fb
Introduce k8s compatibility mode to C# codegen (#4544)
Introduce k8s compatibility mode to C# codegen
2020-05-19 11:41:06 +02:00
Mikhail Shilkov
f66100ce28
Initial support for C# program gen (#4611)
Initial support for C# program gen
2020-05-19 10:18:38 +02:00
Pat Gavlin
088751a4a8
[codegen/hcl2] Refactor package schema loading. (#4661)
- Move the implementation of loadPackageSchema into a method on
  PackageCache
- Protect the cache with synchronization primitives to enable
  concurrency in downstream consumers
- Use jsoniter to deserialize schemas
2020-05-18 17:35:11 -07:00
Praneet Loke
30b12cff49
[codegen/docs] Various Go-related fixes (#4646)
* Remove code that was hiding Go as a language option for k8s overlay resources. Mark input args type for Functions as optional in Go.

* Show a special note in the Go function's snippet when the function name does not match other languages.

* Make sure the draft PR stays assigned to the original author. Add some more comments.

* Remove logic to use the Go ModuleToPackage map to then lookup the relevant C# namespace. Make getLanguageModuleName a method of modContext since passing in a package is not needed anymore.
2020-05-15 17:33:56 -07:00
Mikhail Shilkov
5b410ec440
Resolve referenced packages for every TF program (#4636) 2020-05-15 19:18:07 +02:00
Sean Holung
f5ed6190ba
Link Python and Go constructors (#4593) 2020-05-15 09:51:24 -07:00
Pat Gavlin
e9dd0e5e69
[codegen/schema] Path unescape type tokens in refs (#4641)
Some type tokens contain characters that are not valid in a single
URL path element (`/` in particular). These type tokens may be
path-escaped, and should be unescaped before they are interpreted.
2020-05-14 13:35:57 -07:00
Evan Boyle
0d257bb32d
go gen_program shell + test + sample (#4630) 2020-05-14 08:43:12 -07:00
Pat Gavlin
8f0a002a26
Generate deprecation comments in Go. (#4631)
This is consistent with the convention for marking members as
deprecated: https://github.com/golang/go/wiki/Deprecated.
2020-05-13 17:12:59 -07:00
Pat Gavlin
99852bc18e
[codegen/nodejs] SDK generator fixes. (#4618)
- Add support for constant values in unions (e.g. `"foo" | "bar"`)
- Generate `Input<T | U>` instead of `Input<T> | Input<U>`
- Emit deprecation messages for resources
- Handle nil state inputs
- Allow packages to provide a README
- Remove sync invoke support

These changes are part of
https://github.com/pulumi/pulumi-terraform-bridge/issues/179, and
restore functional parity with the current `tfgen` generator.
2020-05-13 10:55:37 -07:00
Praneet Loke
54f5a0ac36
[codegen/docs] Use the C# namespaces override map for links. (#4616) 2020-05-13 09:32:40 -07:00
Pat Gavlin
647b6627a2
[codegen/hcl2] Fix descent in RewriteConversions. (#4614)
Descend into anonymous function expressions, the operands to conditional
expressions, and the value in for expressions.
2020-05-13 08:25:26 -07:00
Praneet Loke
fcac4af8ae
[codegen/docs] Generate a table with links to the language-specific module pages (#4597)
* Generate language package details in the index pages.

* Add a new DocLanguageHelper interface method to get to a module per-language. 

* Add a new workflow file for automatically creating draft docs PRs for previewing resource docs for AWS and Kubernetes as a result of changes in the resource docs generator.
2020-05-12 15:37:37 -07:00
Pat Gavlin
33258326e0
[codegen/hcl2] Add a conversion insertion pass. (#4594)
Add a rewriter that reifies implicit conversions into a call to the
`__convert` intrinsic. Code generators can recognize this intrinsic and
use it to generate appropriate conversion code.

Part of this work involves redesigning the type annotations system.
Annotations are now only applicable to opaque and object types. Instead
of inspecting annotations directly, code generators should use
`hcl2.GetSchemaForType` to extract the `schema.Type` for a `model.Type`.
2020-05-11 11:17:36 -07:00
Pat Gavlin
255620ea1e
[codegen/nodejs] Await invokes in async contexts. (#4598)
If we are generating code into an async context (e.g. an async main),
await calls to invoke rather than leaving them as promises. This results
in more idiomatic code withing such contexts.
2020-05-11 10:21:56 -07:00
Praneet Loke
aa313aecf8
[codegen/docs] Add a new template for examples section (#4591)
* Add a new template for examples section. Extract the examples section into a structured format for custom template processing.

* Update the description IFF we were able to extract examples sections from it.

* Update doc comments and add missing file header for the newly added file.

* Make the example description readable. Add a check for empty example sections.

* Add a chooser right below the Example Usage header. Remove javascript as a language.

* Allow an empty new-line between short-codes boundaries.
2020-05-08 16:25:28 -07:00
Praneet Loke
5613867b3b
[codegen/docs] Add apiextensions as a k8s overlay resource (#4583)
* Add apiextensions as a k8s overlay. Add comment.
2020-05-08 11:13:21 -07:00
Pat Gavlin
fc00c16e31
[codegen/hcl2] Fix iterator apply rewrites. (#4589)
There are two expressions in HCL2 that are used to iterate over
collections:

- Splat expressions, e.g. `foo.*.bar`, and
- For expressions, e.g. `[for v in foo: v.bar]`

In both of these cases, the parts of the expression that are not the
collection being iterated behave like callbacks, and must be treated as
such by the apply rewriter.
2020-05-07 12:15:40 -07:00
Pat Gavlin
8fded5f29f
[codegen/hcl2] Fix template control tokens. (#4584)
Token detection was broken for conditional and for expressions that
represent template control sequences. The code originally attempted to
determine whether or not a conditional or for expression was a control
sequence by inspecting the expression's parent. Unfortunately, that
approach is unable to distinguish between expressions that are control
sequences and those that are merely template parts. These changes
instead inspect the first token of the expression for a template control
token (i.e. `%{`): if such a token is found, the expression is detected
as a template control sequence.
2020-05-07 09:12:00 -07:00
Praneet Loke
3b9ec6590b
[codegen/docs] Fix kubernetes resource docs links for overlay resources (#4578)
* Adjust the args type names for k8s overlay resources.

* Revert to using input type link for kubernetes non-overlay resources.

* Use the correct module name to lookup the C# namespace for resources that belong to a module.

* Helm's arg types don't use the version number.
2020-05-06 12:11:31 -07:00
Pat Gavlin
f23b98a7df
[codegen/nodejs] Fix #4563. (#4579)
Do not wrap non-output-typed arguments to `pulumi.all` in
`pulumi.output`. `pulumi.all` does this implicitly.

Fixes #4563.
2020-05-06 10:09:20 -07:00
Pat Gavlin
0e512aa0ef
[codegen/{nodejs,python}] Parse more proxy applies (#4576)
Specifically, handle index and relative traversal expressions, and clean
up the code a little bit.

This should also help us pick up more `pulumi.interpolate` calls in TS.
2020-05-05 21:51:53 -07:00
Pat Gavlin
7b446d6c55
Escape """ in Python doc comments. (#4570)
With the addition of Python examples, doc comments may now contain
triple quotes. These must be escaped in order to avoid malformed source
files.

Fixes #4568.
2020-05-05 12:39:35 -07:00
Pat Gavlin
96300f12d9
[codegen/*] Improve range type binding + codegen. (#4552)
- Determine variable types for ranged resources by typechecking an
  equivalent expression
- Detect top-level await in NodeJS and generate an async main
- Fix `pulumi.all` generation for NodeJS
- Fix a bug in the lowering of relative traversals in Python
2020-05-04 15:04:35 -07:00
Praneet Loke
e66f4bd839
[codegen/docs] Remove exclusions used for Kubernetes' provider resource (#4551) 2020-05-03 16:27:33 -07:00
Paul Stack
3026395785
Add support for NPM Resolutions to nodejs codegen (#4554) 2020-05-02 22:17:31 +01:00
Praneet Loke
fe3bd9ed99
[codegen/docs] Restore the API type links for C#. (#4543)
* Restore the API type links for C#.

* Also restore them in function.tmpl.

* Add package details to the Functions template as well. Add a global template function to detect if the APIDocLinks has links for a language. Don't generate C# API doc links for k8s.
2020-05-01 17:13:56 -07:00
Pat Gavlin
d96f5263fb
Merge pull request #4548 from pulumi/pgavlin/codegenFixes
[hcl2] Fix package loading and trivia detection
2020-05-01 14:10:22 -07:00
Pat Gavlin
8c4c46052d Fix the entries() signature 2020-05-01 13:10:56 -07:00
Pat Gavlin
50b1049337 Fix a misspelling 2020-05-01 11:04:44 -07:00
Pat Gavlin
21634768aa [hcl2] Fix package loading and trivia detection
- Load packages referenced by all node types, not just resource nodes
- Fix trivia detection
2020-05-01 11:00:58 -07:00
Levi Blackstone
cca94a5a71
Add helpers for downsteam Go codegen (#4546) 2020-05-01 11:52:07 -06:00
Pat Gavlin
4278595f1b [codegen/python] Fix case mapping in traversals.
This code was accidentally removed in a prior PR.

These changes also remove an assert; the situation the assertion guards
against is valid and occurs when dealing with traversals inside of
splat expressions.
2020-05-01 10:11:50 -07:00
Pat Gavlin
93a7dbe412
[hcl2/model] String literal + traversal fixes. (#4542)
- Properly escape string literals inside template expressions and
  heredocs
- Fix trailing trivia detection for scope and relative traversals
2020-05-01 06:29:39 -07:00
Mikhail Shilkov
d91f658576
Codegen changes to prepare for C# Kubernetes schema (#4531)
Codegen changes to prepare for C# Kubernetes schema
2020-05-01 12:45:47 +02:00
Pat Gavlin
ca5cb05482 [hcl2/model] String literal + traversal fixes.
- Properly escape string literals inside template expressions and
  heredocs
- Fix trailing trivia detection for scope and relative traversals
2020-04-30 16:38:36 -07:00
Pat Gavlin
1d6cce98fe
[codegen/python] Fix nested quotes. (#4539)
Unlike most languages with interpolated strings, Python's formatted
string literals do not allow the nesting of quotes. For example,
this expression is not legal Python:

    f"Foo {"bar"} baz"

If an interpolation requires quotes, those quotes nust differ from the
quotes used by the enclosing literal. We can fix the previous example
by rewriting it with single quotes:

    f"Foo {'bar'} baz"

However, this presents a problem if there are more than two levels of
nesting, as Python only has two kinds of quotes (four if the outermost
string uses """ or '''): in this case, the expression becomes
unspellable, and must be assigned to a local that is then used in place
of the original expression. So this:

    f"Foo {bar[f'index {baz["qux"]}']} zed"

becomes this:

    index = "qux"
    f"Foo {bar[f'index {baz[index]}']}"

To put it bluntly, Python code generation reqiures register allocation,
but for quotes. These changes implement exactly that.

These changes also include a fix for traversals that access values that
are dictionaries rather than objects, and must use indexers rather than
attributes.
2020-04-30 16:34:25 -07:00
Pat Gavlin
76a0f75f5e Fix a build break. 2020-04-30 14:52:38 -07:00
Pat Gavlin
5271eda247
[codegen/python] Fix case mapping. (#4538)
Mapping happens at package scope, not type scope. This is what causes
some of the surprising mixes of camel and snake casing in our packages.

In order to accommodate this, build the case mapping tables up front,
and refer to them when performing case mapping.
2020-04-30 14:16:56 -07:00
Pat Gavlin
1eff2b9acf
[hcl2/model] Propagate annotations in InputType. (#4535)
When creating the inputty version of an `ObjectType`, ensure that the
result carries any annotations present on the original type.
2020-04-30 14:16:43 -07:00
Pat Gavlin
6bd81de8aa
[hcl2] Fix tokens and printing for TraverseIndex. (#4536)
The tokens that make up the "key" portion of an index traversal
(e.g. `"foo"` in `a["foo"]`) are structured like those that make up a
block label: an open quote token, a string literal token, and a close
quote token. The token mapper did not account for that fact, and instead
recorded the key token as the open quote. These changes correct that
error, and adjust the code in `literalText` to allow for
properly-escaped and quoted strings where necessary.
2020-04-30 13:44:32 -07:00
Pat Gavlin
6ae15f6a44
[codegen/hcl] Fix binop printing. (#4537)
The code that handles automatic parenthesis insertion was broken for
operators with the same precedence.
2020-04-30 13:44:05 -07:00
Pat Gavlin
b4daf94c2f
[codegen] Add support for caching package schemas. (#4534)
If a single process is going to bind and generate multiple programs, it
is useful to be able to cache package schemas in order to avoid the
(large) overhead of deserializing schemas multiple times.
2020-04-30 13:22:24 -07:00
Pat Gavlin
72a6add300
[codegen/python] Decrease alloc volume in PyName. (#4533)
Use strings.Builder instead of raw slices. We could go further here and
collapse to a single, ~right-sized bulder at the cost of complexity in
the `stateAcronym -> stateLowerOrNumber` transition.

Also, eliminate the pyName function, as it is redundant.
2020-04-30 13:22:13 -07:00
Praneet Loke
7b17463031
[codegen/docs] Fix the generation of Function names, args, and result type names (#4490)
* Add a new DocLangHelper interface method to generate function names per language. Use the functionNames override map in Go to correctly generate a function name.

* Add kong to the title lookup map.

* Check if item was found in map.

* Use title case for the nodejs Function args type name. Use title case for the function name in the index page.

* Also fix the result name for a Function in nodejs to use title case.

* Use the module name to lookup the package.

* Use the title-case name for the Function name in the index page. Fix the language value passed to the lang chooser in function.tmpl.

* Use title case for nested type titles.

* Add title lookup for GitHub.
2020-04-27 17:47:01 -07:00
Pat Gavlin
2e9499a000
[codegen/hcl2] Fix the apply rewriter. (#4486)
Some of the apply rewriter's assumptions were broken by the richer
expressions available in HCL2. These changes fix those broken
assumptions, in particular the assumption that only scope traversal
expressions are sources of eventual values.
2020-04-24 22:04:24 -07:00
Pat Gavlin
dc2d91e294
[codegen/hcl2] Separate binding and typechecking. (#4483)
This allows consumers to change the inputs to an expression and re-run
typechecking to compute the expression's new type.
2020-04-24 08:52:52 -07:00
Pat Gavlin
af344d9b72
Fix Python module names. (#4473)
In particular, ensure that they are keyword-safe. This affects
`aws:lambda:*` in particular: prior to these changes, we were generating
code into `pulumi_aws.lamdba`, which is not referencable due to its use
of the `lamdba` keyword. With these changes, we generate code into
`pulumi_aws.lambda_`.

There is also a small fix for multi-line non-formatted strings: these
strings do not need escaped braces.
2020-04-24 08:52:36 -07:00
Pat Gavlin
739634c702
Merge pull request #4475 from pulumi/pgavlin/applies
Apply rewriter: separate global and expr data
2020-04-24 08:52:14 -07:00
Paul Stack
090f390d7b
Ensure that DotNet codegen marks Deprecated classes as obsolete (#4482)
Related: https://github.com/pulumi/pulumi-terraform-bridge/issues/133
2020-04-24 01:18:29 +01:00
Praneet Loke
ffeeed965f
[codegen/go] Handle recursive types when marking optional properties to use Ptr types in Go (#4471) 2020-04-22 16:49:57 -07:00
Pat Gavlin
18ecd1c8a9 Apply rewriter: separate global and expr data
This will make some incoming fixes that address nested applies easier to
write and understand.
2020-04-22 15:48:56 -07:00
Mikhail Shilkov
b67c774fcc
C# name resolution (#4468) 2020-04-23 00:42:31 +02:00
Pat Gavlin
3a45d46046
Fix output property names in Python codegen. (#4460)
* Fix output property names in Python codegen.

Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.

Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.

The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.

* Fix Python input property names.

- Fix input property names and forms for invokes. Previously we
  generated a dict; now we generate properly-named args.
- Fix nested property names for resources.

* PR feedback
2020-04-22 10:25:19 -07:00
Praneet Loke
6f43784cf8
[codegen/docs] Add an exclusion to manually render the k8s overlay resource constructor params for Python (#4469) 2020-04-22 10:13:35 -07:00
Pat Gavlin
4f0aec56f6 PR feedback 2020-04-21 15:38:42 -07:00
Pat Gavlin
9204d70c54 Fix Python input property names.
- Fix input property names and forms for invokes. Previously we
  generated a dict; now we generate properly-named args.
- Fix nested property names for resources.
2020-04-21 13:55:33 -07:00
Luke Hoban
6afefe050f
[codegen/go] Fix accessors on struct ptr outputs (#4456)
* [codegen/go] Fix accessors on struct ptr outputs

The accesor methods on nestred struct Ptr outputs were previously not accepting pointer typed inputs as they should, and would thus always panic if used.

The "simple" fix would be to just accept the pointer type and blindly dereference it.  But this doesn't seem like the right experience - it would make these accessors very unsafe to use in practice.

Instead, this PR implements the accessors on pointer-typed outputs as nil-coaslescing, always lifting the output type into a pointer type and flowing a nil value into the result type.  This ensures the accessor will not nil-deref, and that user code can handle the `nil` value itself (or use `.Apply` directly to implement more specialized behaviour).

Before:

```go
// Name of your S3 bucket.
func (o BuildStorageLocationPtrOutput) Bucket() pulumi.StringOutput {
	return o.ApplyT(func(v BuildStorageLocation) string { return v.Bucket }).(pulumi.StringOutput)
}
```

After:

```go
// Name of your S3 bucket.
func (o BuildStorageLocationPtrOutput) Bucket() pulumi.StringPtrOutput {
	return o.ApplyT(func(v *BuildStorageLocation) *string {
		if v == nil {
			return nil
		}
		return &v.Bucket
	}).(pulumi.StringPtrOutput)
}
```

However, due to the decision to have this more usable behaviour, this is a breaking change, as some/many accessors now return a pointer type when they previously did not.

Fixes pulumi/pulumi-azure#530.

* Mark nested property types as requiring ptr types

* Add CHANGELOG

* More fixes
2020-04-21 13:33:38 -07:00
Pat Gavlin
1d672563da Fix output property names in Python codegen.
Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.

Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.

The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.
2020-04-21 10:25:27 -07:00
Pat Gavlin
807cf44cd7
Codegen: collect imports from function calls. (#4445)
Codegen: collect imports from function calls; fix Python resource names.

Just what it says on the tin.
2020-04-21 10:24:42 -07:00
Sean Holung
5d9e3626b3
[codegen/docs] Improve titles tags and meta descriptions (#4432)
* Optimize titles and descriptions

* Move regexp to file level variable

* wip

* Add title tags and description to module and pkg index pages

* Remove block_external_search_index from index.tmpl

* Improve resource descriptions

* Add more providers to look up map

* Improve module level descriptions

Clean up

cleanup

* Update the function.tmpl with the auto-generated disclaimer. Pass title tag and meta description for the function template as well.

* Fix whitespace in the package_details template..

* Account for empty module names for package-level resources. Move logic for generating title tag and description into separate methods for Resources and Functions.

* Add a test for the generated resource and function title tag.

Co-authored-by: Praneet Loke <1466314+praneetloke@users.noreply.github.com>
2020-04-21 12:28:44 +01:00
Pat Gavlin
1f19cbbe08
Clean up the language-specific schema APIs. (#4454)
Rather than forcing consumers to deal with language-specific data
ad-hoc, add an API that allows all language-specific data to be decoded
up-front.
2020-04-20 16:36:05 -07:00
Justin Van Patten
5328b6801f
Inject id resource output property in docs (#4443)
All resources have an implicit `id` output property that must be injected in docs.
2020-04-18 14:17:08 -07:00
Justin Van Patten
a9fac0c51c
Add a period to the end of the Context object comment (#4440)
To match the other comments that have periods.
2020-04-18 13:06:43 -07:00
Justin Van Patten
c04d1beaab
Fix header anchor ids in resource docs (#4436)
* Add ids to headers so they can be linked to

* Make header ids more general

Rather than including the identifier in the name of the id, use a simpler more generic id.

* H3=>H2 for Modules/Resources/Functions/Package Details headings

This way, on individual resource/function pages, it doesn't make it look like "Package Details" is a sub heading under "Supporting Types" in the right-hand "ON THIS PAGE".
2020-04-18 02:45:47 -07:00
Sean Holung
06c4c6b0a8
Polish input and output props (#4408)
* Polish input and output props

* Add resource name to heading.

* Update pkg/codegen/docs/templates/resource.tmpl

Co-Authored-By: Christian Nunciato <c@nunciato.org>

* Update pkg/codegen/docs/templates/resource.tmpl

Co-Authored-By: Christian Nunciato <c@nunciato.org>

* Update pkg/codegen/docs/templates/resource.tmpl

Co-Authored-By: Justin Van Patten <jvp@justinvp.com>

* Update pkg/codegen/docs/templates/resource.tmpl

Co-Authored-By: Justin Van Patten <jvp@justinvp.com>

* Fix relref links

Co-authored-by: Christian Nunciato <c@nunciato.org>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2020-04-17 16:48:47 -07:00
Chris Smith
2c03c158e2
Annotate .ctor args in docs (#4385)
* Annotate .ctor args in docs

* Update pkg/codegen/docs/gen.go

Co-Authored-By: Praneet Loke <1466314+praneetloke@users.noreply.github.com>

* Mark ctor params as required/optional

* Address PR feedback

* Address PR feedback

Co-authored-by: Praneet Loke <1466314+praneetloke@users.noreply.github.com>
2020-04-17 11:46:02 -07:00
Praneet Loke
78e9621379
[codegen/docs] Allow specifying the languages to show in the lang chooser (#4426)
* Add auto-generated disclaimer for the resource.tmpl as well.

* Allow specifying the languages to show in the lang chooser. Hide the go language for k8s overlay resources.

* Add another exclusion for generating the C# constructor param for k8s overlay resources.

* Replace hard-coded exclusions for skipping input properties with checking for ConstValue attribute of input properties.
2020-04-17 11:45:06 -07:00
Sean Holung
854a1d5758
Add version to package details (#4433) 2020-04-17 11:39:19 -07:00
Pat Gavlin
f081d98a39
HCL2: support for tolerating missing variables. (#4430)
When this tolerance is enabled, diagnostics will not be emitted for
missing variables.
2020-04-17 08:24:44 -07:00
Pat Gavlin
398878de31
Various codegen updates. (#4424)
- Define `null` in Pulumi HCL2
- Bind Pulumi HCL2 in topological order s.t. variable types can be
  properly computed
- Fix resources that range over bools and numbers
- Add element, length, lookup, readFile, and split functions
- Do not rewrite function signatures with input types during binding
- Fix splat expression binding for non-lists
- Add support for evaluating expressions
- Add support for operator precedence to code generators
- Add support for constants to the HCL2 IR
- Add support for generating ranged resources in Python
- Add support for generating conditional resource in Node and Python
- Fix various naming issues in Python
2020-04-16 16:44:34 -07:00
Sean Holung
7834f2fcb2
Generate links for primitive types. (#4386) 2020-04-15 12:56:28 -07:00
Praneet Loke
c830017383
[codegen/docs] Cleanup generating links for Pulumi types (#4395)
* Add a method to the DocLanguageHelper interface to get the API doc link for Pulumi types.

* Add a test for Go API doc link methods.

* Use GetDocLinkForPulumiType in gen_function as well.
2020-04-15 11:58:08 -07:00
Mikhail Shilkov
91dce426b2
Use Config.Get for string-based token types in .NET code gen (#4399) 2020-04-15 19:59:02 +02:00
Luke Hoban
39eeaef92d
[codegen] Generate correct links to v>=2 of Go packages (#4346) 2020-04-14 13:59:23 -07:00
Sean Holung
dd413723ad
Remove display of optional type identifiers. (#4374) 2020-04-14 13:34:53 -07:00
Mikhail Shilkov
9077292d14
Fix the namespace capitalization in C# codegen (#4392) 2020-04-14 19:53:43 +01:00
CyrusNajmabadi
66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls (#3750)

* Switch away from native grpc impl. (#3728)

* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)

* Only retry as long as we get unavailable back.  Anything else continues. (#3769)

* Handle all errors for now. (#3781)


* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes (#4288)

Adjust C# generation

* Replace IDeployment with a sealed class (#4318)

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty (#4320)

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen (#4379)
2020-04-14 09:30:25 +01:00
Evan Boyle
90ced9574d
fix go config codegen (#4388)
* fix go config codegen

* changelog
2020-04-14 09:24:32 +01:00
Pat Gavlin
51588ede49
Fix various printing issues. (#4378)
- Parentheses were not handled properly
- Literals inside of template control sequences were not handled
  properly

These changes also improve test coverage for the printers.
2020-04-13 19:11:56 -07:00
Praneet Loke
931da8f109
[codegen/docs] Fix Kubernetes resource links for NodeJS and C# constructor params (#4384)
* Fix k8s input type links for constructor args.

* Skip using property case maps for k8s Python.

* Fix the namespace of C# constructor args type and resource type.
2020-04-13 17:27:38 -07:00
Erin Krengel
562d1298f2
fix link to be lowercase (#4382) 2020-04-13 15:48:36 -07:00
stack72
69c804a4e7 Adding system namespace for Dotnet code gen
This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```
2020-04-12 21:37:52 +01:00
Praneet Loke
563233d16c
[codegen/docs] Use the correct module names for languages. (#4367)
* Use the correct module names for languages.

* Add Kubernetes to the title lookup map.

* Add output properties to the map only if filtered output properties is non-zero length.
2020-04-10 18:28:14 -07:00
Sean Holung
f28bf6acf5
Remove input props from output properties list (#4360)
* Remove input properties from displaying in output props list
2020-04-10 16:12:05 -07:00
Praneet Loke
729ae08c9d
Handle k8s cases in the resource doc generator (#4337)
* Add a check for the Python code gen to prevent infinte recursion. Remove the custom logic for k8s for generating Python property case maps.

* Fix bug with C# links for k8s constructor resources.

* Apply a style on the deprecation message.

* Use the display name when rendering a property type that does not have a link.

* Fix the python type string generator for docs to inspect union types.

* Update cleanTypeString to account for nodejs package names.

* Add missing lookup names for packages.
2020-04-10 14:07:52 -07:00
Pat Gavlin
71aa0f020f
Fix treatment of trivia in template expressions. (#4352)
And improve test coverage for the same.
2020-04-09 16:37:52 -07:00
Sean Holung
9cd7d7c5c0
Group required resource properties at top of list (#4351) 2020-04-09 15:57:17 -07:00
Sean Holung
7a2f020d2e
Map display names for API Reference TOC menu (#4339) 2020-04-09 09:43:38 -07:00
Levi Blackstone
ca6e47277f
Handle const values in Go codegen (#4311) 2020-04-08 10:33:01 -06:00
Praneet Loke
efaa026196
Make sure there is a new-line between the package description and the first header on the index page. (#4312) 2020-04-07 12:27:35 -07:00
Pat Gavlin
00f1433706
HCL2/NodeJS: fix proxy apply lowering for promises (#4317)
Applies cannot be proxied when the applied value is a promise.

Fixes #4315.
2020-04-07 09:25:50 -07:00
Pat Gavlin
2f22c1c59c
HCL2 updates (#4309)
Pulumi HCL2 IR:
- Add support for invokes
- Add support for resource options, incl. ranged resources
- Allow the apply rewriter to ignore promise-typed values
- Add tests for the binder
- Add support functions for TF: entries and range

NodeJS codegen:
- Simplify for expression codegen
- Add support for invoke codegen
- Add support for entries and range functions
- Add tests

Python codegen:
- Implement codegen for most expression types
- Add support for invoke codegen
- Add tests
2020-04-06 19:43:16 -07:00
Praneet Loke
50e1086124 Make sure there is a new-line between the package description and the first header on the index page. 2020-04-06 17:43:21 -07:00
Praneet Loke
22aeaa43b1
Move resource docs index file generation to Go templates (#4308)
* Add new templates for generating index files. Remove the now irrelevant code.

* Add comment for generating index files by deduping module names. Use the path format of a module name as the link.

* Add a trailing slash for module links in the index page.

* Move the categories rendered in the index file into their own template file. Handle kubernetes provider specific module name conversion while generating parent modules.

* Fix issue with the k8s provider resource being generated as a module rather than a resource on the package-level index page.

* Show the deprecation message of a resource at the top of the document.

* Move k8s specific things to its own file.

* Add title attribute to the list items. Show the last part of a module name, in case it contains path separators

* Lookup the package name from the Go language info object for k8s then use that to lookup the C# namespace.

* Move the logic for cleaning a property type string for display names to a separate function.

* Export the title function from go and dotnet code generators for use in the resource doc generator. Use the title function from the respective lang code gens to match the correct title case used there. Fix issue with the Kubernetes namespace not being stripped for C# property type strings.

* Skip including apiVersion and kind as input properties for Kubernetes until pulumi-kubernetes#1062 is merged.
2020-04-06 17:01:33 -07:00
Evan Boyle
ab659aa0c1
Reimplement getRequiredPlugins for go sdk (#4297) 2020-04-06 12:30:40 -07:00
Praneet Loke
ea383991f2
Update resource docs generator to handle some k8s cases (#4270)
* Fix an issue with stripping the module name from type doc links in nodejs.

* Remove unused code that visited object types. Update the nested types code to account for types that may reference themselves. Added glog logging statements. Save the package-level language info objects, so we can use them later.

* Check if a type token has already been added as an input or an output type when collecting nested types.

* Rename appearsIn to typeUsage.

* Scan provider resource also using the special-case function if package is k8s.

* Simplify the display names for Pulumi types by removing the Pulumi prefix.
2020-04-03 11:26:52 -07:00
Pat Gavlin
b02a84b8d9
Add a code generator for Python programs. (#4285)
This code generator processes a bound Pulumi program represented using
an HCL2-based IR and converts it to an equivalent Python program.
2020-04-02 23:29:05 -07:00
Pat Gavlin
4ae2867a15
Add a code generator for NodeJS programs. (#4284)
This code generator processes a bound Pulumi program represented using
an HCL2-based IR and converts it to an equivalent NodeJS program.
2020-04-02 23:27:05 -07:00
Pat Gavlin
742c047211 Remove an unused import 2020-04-02 22:09:50 -07:00
Pat Gavlin
1860b4f656 Appease the linter 2020-04-02 21:43:03 -07:00
Pat Gavlin
e831ecbc0e Update package references 2020-04-02 21:25:30 -07:00
Pat Gavlin
2f5238f130 Updates to the Pulumi HCL2 projection
- config variables are now specified as `config <name> <type>`
- output variables are now specified as `output <name> <type>`
- apply arguments are assigned better names by the apply rewriters
2020-04-02 21:25:30 -07:00
Pat Gavlin
89c9401fad Appease the linter 2020-04-02 21:25:30 -07:00
Pat Gavlin
aa655c6076 Add a package for binding HCL2 as Pulumi config.
These changes add preliminary (read: incomplete) support for
representing Pulumi programs using HCL2. Language-specific code
generators can use this representation as a basis for understanding the
semantics of a Pulumi program.
2020-04-02 21:25:30 -07:00
Pat Gavlin
c5782115d2
Various updates to the HCL2 semantic model. (#4281)
- Improve support for printing and triva
- Add support for HCL2 type functions
- Add visitors for body items
2020-04-02 21:23:12 -07:00
Levi Blackstone
e92b1b4e8d
Add schema support for constant values (#4283)
Some properties have constant values, such as the
apiVersion and kind for the Kubernetes provider. Add
a Const field to the Property schema to support this.
2020-04-02 21:47:07 -06:00
Pat Gavlin
a597934b3c
Updates to the HCL2 syntax helpers. (#4282)
- Fix token mapping for template control structures
- Split tokens out into their own file
- Add factory methods for token types
2020-04-02 20:01:14 -07:00
Evan Boyle
3efbc3705d
Update go codegen to include usage hints on Input types (#4279) 2020-04-02 19:59:08 -07:00
Luke Hoban
d018212adc
[codegen/docs] Skip adding Notes to package details if empty (#4275) 2020-04-02 11:26:11 -07:00
Luke Hoban
a0f615ad09
Add package details and atttribution to schema (#4256) 2020-04-01 17:22:23 -07:00
Christian Nunciato
7e46e7580f
Resource template fixes (#4245) 2020-03-31 04:07:04 -07:00
Christian Nunciato
21ee0919cd Update the resource-docs templates 2020-03-30 14:38:29 -07:00
Praneet Loke
bdc12b2c98
Fix csharp and nodejs doc links generated by resource docs generator (#4237)
* Decode the C# language info from the schema package and set it in the dotnet lang helper for docs. Pass the qualifier for C# property type strings based on whether it is an input or an output property.

* Don't pass a module name for Pulumi core types used in TS constructor and Function params.

* Add tests for generating doc links for nodejs types.

* Add test for confirming input doc link for C# type.

* Fix the C# type name for InvokeOptions.
2020-03-30 14:37:30 -07:00
Tasia Halim
655def51b3
Add word breaks to nested type names (#4191) 2020-03-26 12:42:15 -07:00
Praneet Loke
651d951416
Remove non-relevant examples and empty examples section from docs comments (#4184)
* Add tests for the examples processing.

* Update the schema-based code generators to strip out non-relevant code examples wrapped in short-codes.

* Extract language relevant examples for Functions too.

* Simpler way of extracting the surrounding text from the description.

* Add some more new-line characters to clean-up the examples sections.
2020-03-26 10:30:04 -07:00
Tasia Halim
bdefc557af make comment htmlSafe 2020-03-25 12:37:03 -07:00
Levi Blackstone
8ce10e1dfe
Add aliases to the Go SDK codegen pkg (#4157) 2020-03-24 11:18:21 -06:00
Praneet Loke
ab080c0c5d
Fix the signature for C# invokes in the resource docs generator (#4134)
* Update the C# invoke signature in the Functions template used by resource docs generator.

* Add a new function to the DocLanguageHelper interface for resource docs generator to generate language-specific property names.

* Add a new DisplayName property to the propertyType struct for simpler display names without module names.

* Update templates to use the DisplayName property of propertyType. Strip the namespace/module name from type names for use as display names.
2020-03-23 14:17:14 -07:00
Evan Boyle
1c4496a180
Merge pull request #4109 from pulumi/evan/gomod
Go Modules
2020-03-20 12:01:25 -07:00
Praneet Loke
861d568eb2
Add test for resource docs to confirm Python casing (#4126)
* Generate Go package maps at the beginning, so that the resource docs can use the appropriate package context for generating a property type string name.
2020-03-20 08:17:58 -07:00
evanboyle
ce1928988b Merge branch 'master' of https://github.com/pulumi/pulumi into evan/gomod 2020-03-19 13:01:38 -07:00
Praneet Loke
5ecf24ee81
Update the resource docs generator to use the correct casing for nested property names in Python (#4111)
* Temporarily block the resource docs from search indexing.

* Use PascalCase for nested type names in Python.

* Fix bug with Python property name casing for nested types by checking the property case maps.
2020-03-19 12:18:18 -07:00
Mikhail Shilkov
0f4e5a6a0b
Fix Go generation (#4113) 2020-03-19 16:41:43 +01:00
Luke Hoban
65899569ee
Fix Type suffix Go codegen for cross-package references (#4110)
* Avoid adding "Type" suffix unnecessarily

* Fix `Type` suffix for cross-package references

* Fix mixxing imports and format code
2020-03-19 08:32:40 -07:00
evanboyle
ec686bbaf6 Merge branch 'master' of https://github.com/pulumi/pulumi into evan/gomod 2020-03-18 17:57:58 -07:00
Pat Gavlin
7501f758b9
Add a utility package for HCL2-based codegen. (#4104)
This package contains a driver for generating source code from HCL2
expressions. The driver is based on the fmt package's support for custom
formatters.
2020-03-18 15:25:10 -07:00
evanboyle
fccf301d14 move pkg/util/contract -> sdk/go/common/util/contract 2020-03-18 14:40:07 -07:00
Evan Boyle
8b46e71887
Go plugin acquisition (#4060)
These changes implement `GetRequiredPlugins` for Go using a registry
mechanism and an alternate entry point for `pulumi.Run`. Packages that
require plugins are expected to register themselves with the Pulumi SDK.
When `pulumi.Run` is used and the `PULUMI_PLUGINS` envvar is truthy, the
program will dump a JSON-encoded description of its required plugins to
stdout. The language host then uses this description to respond to
2020-03-18 12:41:45 -07:00
Pat Gavlin
900379bd38
Add an HCL2 static typechecker and semantic model. (#4087)
These changes add a package for type checking and modeling HCL2
configurations. It is made up of three primary components:

1. A static type system
2. A semantic representation of HCL2 expressions and a binder from HCL2
   native syntax to this representation
3. A semantic representation of HCL2 structural elements and binders
   from HCL2 native syntax to this representation.

The type system is described in the "Extended Types" section of the
specification. The semantic representations of expressions and
structural elements are documented in their implementations.
2020-03-18 09:28:57 -07:00
Praneet Loke
cfcfad2f3c
Fixes for resource docs generator (#4085)
* Generate the constructor params for Python along with other languages.

* Remove redundant py_function_param nested template. Declare a new type for defining property characteristics rather than using inlining formal params. Generate the Lookup functions for all languages similar to the constructor params with linking enabled.

* Fix bug with generating the input arg type name for Functions in Go.

* Add prefix for args param of a Go-based Resource Function.

* Input args for Go-based Functions use Lookup*Args and not Get*Args.

* Turns out that args for Go-based Functions use a different prefix based on whether the function is a package-level or module-level Function.

* Update the Python list and dictionary type names for the resource doc generator.

* Add a separate function for Python doc helper return a type string representing dictionaries that are simple maps and don't have a known nested element type.
2020-03-17 13:37:40 -07:00
Levi Blackstone
655839009a
Update Go SDK generator to support k8s (#4096)
- Add GoInfo struct to improve control of package imports and names
- Fix import recursion bug
- Add support for aliased imports
2020-03-17 14:09:43 -06:00
Pat Gavlin
8111d33438
Shorten Python package descriptions. (#4068)
These descriptions should only be one line, and should not contain markdown.
2020-03-12 21:43:54 -07:00
Praneet Loke
9abc65d3c6
Remove hard-coded Bucket examples from the schema-based docs generator. (#4067)
* Also remove the Examples property from the resourceDocArgs type since the resource Comment will contain the multi-lang examples on their surrounded by a shortcode.
2020-03-12 13:16:08 -07:00
Praneet Loke
40f17faa73
Updates to resource doc generator for generating docs for Functions (#4055)
* Added a new template for Functions. Implement the genFunction method for generating the docs for Functions.

* Rename type resourceArgs to resourceDocArgs. Minor updates the resource template.

* Generate nested types for Functions.

* Unexport types that don't need to be exported. Create the doc language helper objects in an init function and reuse them rather than recreating them every time. Update genNestedTypes to work with schema functions or resources.

* Fixed bug in nested type generation for Functions. Fixed bug in generating input and output doc links for nested types.
2020-03-11 17:58:12 -07:00
Pat Gavlin
622cac4dd4
Stabilize Python codegen. (#4056)
Make sure all property casing conversions are computed prior to codegen.
2020-03-11 15:42:15 -07:00
Paul Stack
4ec9667a9f
Merge pull request #4043 from pulumi/fix-pygen-overlays
Ensure python overlays work as expected
2020-03-10 00:44:11 +02:00
stack72
b100f8494e Ensure python overlays work as expected
Fixes: https://github.com/pulumi/pulumi-terraform-bridge/issues/119

This allows us to specify an overlays block e.g.

```
Overlay: &tfbridge.OverlayInfo{
	DestFiles: []string{
		"pulumi_docker/docker.py",
		"pulumi_docker/image.py",
	},
},
```

The overlays files are treated differently to normal module files
as they are not generated. This structure means that we will emit
the correct entries in the __init__.py file

Without this structure (ie. pulumi_pkgname), the generator actually
copies the file (i.e. docker.py) to the root of the Python SDK. This
is because the structure of the Python SDK has a sub-folder than that
of the NodeJS SDK

I tested this using PR https://github.com/pulumi/pulumi-docker/pull/141
and this now works as expected and we can take advantage of the new
Python overlays for Docker
2020-03-10 00:03:41 +02:00
Praneet Loke
edbb05dddd
Update schema-based docs generator (#4035)
* Update properties.tmpl to render property comment as-is. WIP splitting out properties to lang-specific tables.

* Generate the constructor dynamically from the resource per language.

* Add doc functions in each language generator package for getting doc links for types..and later other functions too.

* Render the constructor params in the Go code and inject into the template.

* Generate nodejs types using the nodejs lang generator.

* Add a templates bundler. Added a new Make target for autogenerating a static bundle for the resource docs generator.

* Generate type links for all languages based on their schema type. Render the property type with a link if the underlying elements have a supporting type. Fix word-breaks for Python type names.

* Various changes including the introduction of an interface type under the codegen package to help with generating some language-specific information for the resource docs generator.

* Add a function to explicitly generate links for input types of nested types. Fix the resource doc link generator for Go. Don't replace the module name from the nodejs language type.

* Fix bug with C# property type html encoding.

* Fix some template formatting. Pass the state inputs for Python to generate the lookup function for it.
* Do not generate the examples section if there are none.

* Generating the property types per language.

* Formatting. Rename function for readability.

* Add comments. Update README.

* Use relative URLs for doc links within the main site
2020-03-09 10:35:20 -07:00
Evan Boyle
81b6afa3c7
Revert "GetRequiredPlugins for Go (#3830)" (#4034)
This reverts commit 3a2890c0cd.
2020-03-06 17:50:18 -08:00
Pat Gavlin
3a2890c0cd
GetRequiredPlugins for Go (#3830)
These changes implement `GetRequiredPlugins` for Go using a registry
mechanism and an alternate entry point for `pulumi.Run`. Packages that
require plugins are expected to register themselves with the Pulumi SDK.
When `pulumi.Run` is used and the `PULUMI_PLUGINS` envvar is truthy, the
program will dump a JSON-encoded description of its required plugins to
stdout. The language host then uses this description to respond to
`GetRequiredPlugins`.
2020-03-06 14:30:43 -08:00
Pat Gavlin
24b93e4c10 Address PR feedback from #3980. 2020-02-27 16:24:13 -08:00
Pat Gavlin
f42d7e756a
Add an HCL2 syntax helper package. (#3980)
These changes add a helper package for parsing HCL2 syntax files. The
helpers are intended to provide direct access to syntax elements that
are abstracted away by the standard HCL2 parser. The bulk of the code
deals with mapping syntax nodes to their relevant tokens in order to
avoid losing information about the comments associated with syntax
nodes.
2020-02-27 12:49:15 -08:00
Justin Van Patten
db559214e8
Initial schema-based docs gen (#3988)
Many TODOs and follow-ups remain, but this is the start of our
schema-based docs generation.
2020-02-27 10:04:34 -08:00
Pat Gavlin
2b59d1405d
Fix a few codegen issues. (#3915)
- Make go comment generation lint-clean for blank comment lines
- Fix the casing of `Provider.py` in Python to `provider.py`
- Fix a spacing issue in the NodeJS code generator
2020-02-11 14:34:22 -08:00
Pat Gavlin
add181e57c
Small fixes in schema codegen. (#3889)
- Do not JSON-encode provider properties that are token types with an
  underlying string type
- Emit PEP 561 type metadata in setup.py
2020-02-07 09:43:20 -08:00
Pat Gavlin
0dbbd919b8
Add schema support for deprecated properties. (#3865)
This is the schema-based version of
https://github.com/pulumi/pulumi-terraform-bridge/pull/96.
2020-02-06 09:29:06 -08:00
Pat Gavlin
ad96a45a05
Align defaults with OpenAPI (#3859)
Replace the various `defaults` maps in the schema with per-property
`default` and `defaultInfo` fields. The former holds the static default
value; the latter holds the envvars and language-specific info.

Also, fix a minor bug in the Python codegen that caused diffs in
property docstrings.
2020-02-05 11:43:39 -08:00
Pat Gavlin
77c0f5dad1 Fix a bug in the Go code generator.
Resource input properties must also be scanned for optionality.
2020-01-24 10:18:34 -08:00
Pat Gavlin
4d8027f057 Fix another typo. 2020-01-23 10:42:38 -08:00
Pat Gavlin
60a5d06d37 Fix a typo. 2020-01-23 10:00:38 -08:00
Pat Gavlin
d400f26832
Add a schema package and code generators. (#3749)
The schema format is described in pkg/codegen/schema/schema.go. The code
generators are derived from the code generators contained in
https://github.com/pulumi/pulumi-terraform-bridge/pkg/tfgen, with the
exception of the Go code generator, which is net new.
2020-01-21 14:45:48 -08:00
Paul Stack
67194cddfd
Creation of generator package (#2970)
Fixes: #2151

This will allow us to be able to share the code that generates our
language providers. Currently there is a copy of the python code
generation in pulumi-kubernetes and also in pulumi-terraform

We want to be able to share these
2019-07-22 17:09:35 -07:00