Commit graph

78 commits

Author SHA1 Message Date
Mikhail Shilkov bb358c4d21
Escape \x in Python comments (#4922) 2020-06-30 11:19:45 +02:00
Pat Gavlin 69ba47cff2
[codegen/*] Add support for resource options. (#4925)
The PCL binder has supported resource options for some time, but these
options haven't been used or processed by the various code generators.
These options--particularly the parent and provider options0--are
critical for import codegen. These changes implement the basic set of
options, and add a note about fleshing out the rest as necessary.

One component of these changes is a new rewriter that rewrites property
references into property paths that are understood by the Pulumi engine.
This rewriter is used to preprocess the contents of the `ignoreChanges`
resource option.

These changes also hack around a weakness in the HCL2 type system:
In Go, references to resources should be typed as `hcl2.ResourceType`.
Unfortunately, this breaks the existing collection semantics associated
with resources. Because of this, the Go code generator does not have
enough information to know that it should generate a `[]pulumi.Resource`
for lists of resources. These changes hack around that limitation using
a Go-specific opaque type and some hardcoded comparisons in
`argumentTypeName`.

Fixes #4923.
2020-06-29 16:33:52 -07:00
Mikhail Shilkov 14a3871346
Avoid stack overflow in Python codegen in case of recursive types (#4908) 2020-06-26 20:47:13 +02:00
Komal 3eacf8bf90
[codegen/python] Only add item to map if it doesn't already exist. (#4907) 2020-06-26 09:05:36 -07:00
Komal b35a94cac6
[codegen/python] - Exclude single word properties from casing tables. (#4895)
Co-authored-by: Levi Blackstone <levi@pulumi.com>
2020-06-25 11:51:57 -07:00
Komal 93fa20a44b
[codegen/python] - Only import json when needed (#4892) 2020-06-25 09:59:00 -07:00
Komal ed752bc384
[codegen/python] Don't use __all__ (#4873) 2020-06-24 11:41:03 -07:00
Komal 669448ab40
[codegen/python] Use CamelCase filenames for k8s. (#4878) 2020-06-23 13:02:22 -07:00
Komal 1837c0a779
[codegen/python] Allow setting constant values for props (#4864)
* Allow setting constant values for props

* PR feedback
2020-06-22 10:58:13 -07:00
Komal 9a62c92efc
[codegen/python] Add missing props to casing table (#4863) 2020-06-22 09:49:34 -07:00
Levi Blackstone 282c95ee40
[codegen] Use nullish coalescing for provider defaults on k8s (#4841)
For the NodeJS k8s SDK, rather than falling back to default values
using the || operator, use the nullish coalescing operator (??).
This avoid situations where the primary value is set to false,
and then is overridden by the default value.
2020-06-22 09:38:20 -06:00
Komal 14ef36553d
[codegen/python] Add whitespace to make the linter happy (#4857) 2020-06-18 17:32:52 -07:00
Komal 63e7388879
[codegen/python] Remaining k8s updates (#4786) 2020-06-18 15:46:17 -07:00
Pat Gavlin 4a903f95b9
[codegen/docs] Reimplement example extraction. (#4851)
Use the schema package's Markdown parser and walk its AST to extract
examples.

These changes also rename StripNonRelevantExamples to FilterExamples.

This is preparatory work for #4159 and #4632.
2020-06-18 12:32:15 -07:00
stack72 0a93270ffd Escape deprecation messages coming from upstream TF providers
Fixes: #4827

Diff:

```
             if encrypted is not None:
-                warnings.warn("Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"", DeprecationWarning)
+                warnings.warn("Rename PostgreSQL role resource attribute \"encrypted\" to \"encrypted_password\"", DeprecationWarning)
                 pulumi.log.warn("encrypted is deprecated: Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"")

```
2020-06-17 20:25:06 +03:00
Komal 92e44bee6d
[codegen/python] Handle extra files (#4842) 2020-06-16 19:27:06 -07:00
Komal f9074f6bcb
[codegen/python] - Exclude nested docs for k8s (#4828) 2020-06-15 16:17:05 -07:00
Komal 481750ec5b
[codegen/python] Fix edge case in relative import (#4809) 2020-06-11 15:31:39 -07:00
Komal ea3d6aa005
[codegen/python] Don't use config for k8s (#4807) 2020-06-11 11:52:26 -07:00
Komal 5ec251ae66
[codegen/python] Fix relative import path (#4796) 2020-06-11 09:10:53 -07:00
Komal 074eb67e82
[codegen/python] Add newlines to be pep8 compliant (#4797) 2020-06-10 15:11:59 -07:00
Komal d699627ff6
[codegen/python] Fix k8s __init__.py generation (#4795)
* Fix k8s __init__.py generation

* Add comment for regex
2020-06-10 13:55:10 -07:00
Evan Boyle 45a901d3e7
Turn program gen NYIs into diagnostic errors (#4794) 2020-06-10 10:21:53 -07:00
Komal 238adf2f2f
[codegen/python] Allow import path overrides (#4787)
* Allow import path overrides

* Fix typo

* Rename ModuleToPackage to ModuleNameOverrides
2020-06-09 14:33:02 -07:00
Komal b93259c6bb
[codegen/python] Fix readme generation (#4759)
* Fix readme generation

* Allow readme text to be passed in with the package.
2020-06-08 13:07:56 -07:00
Komal 6c29b67c03
[codegen/python] Fix doc generation in python codegen (#4757) 2020-06-08 12:44:57 -07:00
Paul Stack 58002bae62
Ensuring a property level deprecation warning is set for python codegen (#4744) 2020-06-04 11:29:23 +01:00
Pat Gavlin bd1c0e6d7b
[codegen/schema] Support default union types. (#4746)
Add support for union types that indicate a default type for targets
that do not support unions, or do not support unions in certain
positions (e.g. output properties). The NodeJS backend makes use of this
in combination with a new flag, `disableUnionOutputTypes`, to avoid
generating unions in output types.

These changes also refactor the various module ->
package/module/namespace mapping methods so that these entities can be
fetched by their language name rather than their token.
2020-06-02 18:15:21 -07:00
Mikhail Shilkov f66100ce28
Initial support for C# program gen (#4611)
Initial support for C# program gen
2020-05-19 10:18:38 +02:00
Sean Holung f5ed6190ba
Link Python and Go constructors (#4593) 2020-05-15 09:51:24 -07:00
Praneet Loke fcac4af8ae
[codegen/docs] Generate a table with links to the language-specific module pages (#4597)
* Generate language package details in the index pages.

* Add a new DocLanguageHelper interface method to get to a module per-language. 

* Add a new workflow file for automatically creating draft docs PRs for previewing resource docs for AWS and Kubernetes as a result of changes in the resource docs generator.
2020-05-12 15:37:37 -07:00
Pat Gavlin 33258326e0
[codegen/hcl2] Add a conversion insertion pass. (#4594)
Add a rewriter that reifies implicit conversions into a call to the
`__convert` intrinsic. Code generators can recognize this intrinsic and
use it to generate appropriate conversion code.

Part of this work involves redesigning the type annotations system.
Annotations are now only applicable to opaque and object types. Instead
of inspecting annotations directly, code generators should use
`hcl2.GetSchemaForType` to extract the `schema.Type` for a `model.Type`.
2020-05-11 11:17:36 -07:00
Pat Gavlin 255620ea1e
[codegen/nodejs] Await invokes in async contexts. (#4598)
If we are generating code into an async context (e.g. an async main),
await calls to invoke rather than leaving them as promises. This results
in more idiomatic code withing such contexts.
2020-05-11 10:21:56 -07:00
Pat Gavlin 0e512aa0ef
[codegen/{nodejs,python}] Parse more proxy applies (#4576)
Specifically, handle index and relative traversal expressions, and clean
up the code a little bit.

This should also help us pick up more `pulumi.interpolate` calls in TS.
2020-05-05 21:51:53 -07:00
Pat Gavlin 7b446d6c55
Escape """ in Python doc comments. (#4570)
With the addition of Python examples, doc comments may now contain
triple quotes. These must be escaped in order to avoid malformed source
files.

Fixes #4568.
2020-05-05 12:39:35 -07:00
Pat Gavlin 96300f12d9
[codegen/*] Improve range type binding + codegen. (#4552)
- Determine variable types for ranged resources by typechecking an
  equivalent expression
- Detect top-level await in NodeJS and generate an async main
- Fix `pulumi.all` generation for NodeJS
- Fix a bug in the lowering of relative traversals in Python
2020-05-04 15:04:35 -07:00
Pat Gavlin 4278595f1b [codegen/python] Fix case mapping in traversals.
This code was accidentally removed in a prior PR.

These changes also remove an assert; the situation the assertion guards
against is valid and occurs when dealing with traversals inside of
splat expressions.
2020-05-01 10:11:50 -07:00
Pat Gavlin 93a7dbe412
[hcl2/model] String literal + traversal fixes. (#4542)
- Properly escape string literals inside template expressions and
  heredocs
- Fix trailing trivia detection for scope and relative traversals
2020-05-01 06:29:39 -07:00
Mikhail Shilkov d91f658576
Codegen changes to prepare for C# Kubernetes schema (#4531)
Codegen changes to prepare for C# Kubernetes schema
2020-05-01 12:45:47 +02:00
Pat Gavlin ca5cb05482 [hcl2/model] String literal + traversal fixes.
- Properly escape string literals inside template expressions and
  heredocs
- Fix trailing trivia detection for scope and relative traversals
2020-04-30 16:38:36 -07:00
Pat Gavlin 1d6cce98fe
[codegen/python] Fix nested quotes. (#4539)
Unlike most languages with interpolated strings, Python's formatted
string literals do not allow the nesting of quotes. For example,
this expression is not legal Python:

    f"Foo {"bar"} baz"

If an interpolation requires quotes, those quotes nust differ from the
quotes used by the enclosing literal. We can fix the previous example
by rewriting it with single quotes:

    f"Foo {'bar'} baz"

However, this presents a problem if there are more than two levels of
nesting, as Python only has two kinds of quotes (four if the outermost
string uses """ or '''): in this case, the expression becomes
unspellable, and must be assigned to a local that is then used in place
of the original expression. So this:

    f"Foo {bar[f'index {baz["qux"]}']} zed"

becomes this:

    index = "qux"
    f"Foo {bar[f'index {baz[index]}']}"

To put it bluntly, Python code generation reqiures register allocation,
but for quotes. These changes implement exactly that.

These changes also include a fix for traversals that access values that
are dictionaries rather than objects, and must use indexers rather than
attributes.
2020-04-30 16:34:25 -07:00
Pat Gavlin 76a0f75f5e Fix a build break. 2020-04-30 14:52:38 -07:00
Pat Gavlin 5271eda247
[codegen/python] Fix case mapping. (#4538)
Mapping happens at package scope, not type scope. This is what causes
some of the surprising mixes of camel and snake casing in our packages.

In order to accommodate this, build the case mapping tables up front,
and refer to them when performing case mapping.
2020-04-30 14:16:56 -07:00
Pat Gavlin b4daf94c2f
[codegen] Add support for caching package schemas. (#4534)
If a single process is going to bind and generate multiple programs, it
is useful to be able to cache package schemas in order to avoid the
(large) overhead of deserializing schemas multiple times.
2020-04-30 13:22:24 -07:00
Pat Gavlin 72a6add300
[codegen/python] Decrease alloc volume in PyName. (#4533)
Use strings.Builder instead of raw slices. We could go further here and
collapse to a single, ~right-sized bulder at the cost of complexity in
the `stateAcronym -> stateLowerOrNumber` transition.

Also, eliminate the pyName function, as it is redundant.
2020-04-30 13:22:13 -07:00
Praneet Loke 7b17463031
[codegen/docs] Fix the generation of Function names, args, and result type names (#4490)
* Add a new DocLangHelper interface method to generate function names per language. Use the functionNames override map in Go to correctly generate a function name.

* Add kong to the title lookup map.

* Check if item was found in map.

* Use title case for the nodejs Function args type name. Use title case for the function name in the index page.

* Also fix the result name for a Function in nodejs to use title case.

* Use the module name to lookup the package.

* Use the title-case name for the Function name in the index page. Fix the language value passed to the lang chooser in function.tmpl.

* Use title case for nested type titles.

* Add title lookup for GitHub.
2020-04-27 17:47:01 -07:00
Pat Gavlin 2e9499a000
[codegen/hcl2] Fix the apply rewriter. (#4486)
Some of the apply rewriter's assumptions were broken by the richer
expressions available in HCL2. These changes fix those broken
assumptions, in particular the assumption that only scope traversal
expressions are sources of eventual values.
2020-04-24 22:04:24 -07:00
Pat Gavlin af344d9b72
Fix Python module names. (#4473)
In particular, ensure that they are keyword-safe. This affects
`aws:lambda:*` in particular: prior to these changes, we were generating
code into `pulumi_aws.lamdba`, which is not referencable due to its use
of the `lamdba` keyword. With these changes, we generate code into
`pulumi_aws.lambda_`.

There is also a small fix for multi-line non-formatted strings: these
strings do not need escaped braces.
2020-04-24 08:52:36 -07:00
Pat Gavlin 9204d70c54 Fix Python input property names.
- Fix input property names and forms for invokes. Previously we
  generated a dict; now we generate properly-named args.
- Fix nested property names for resources.
2020-04-21 13:55:33 -07:00
Pat Gavlin 1d672563da Fix output property names in Python codegen.
Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.

Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.

The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.
2020-04-21 10:25:27 -07:00