Commit graph

139 commits

Author SHA1 Message Date
Pat Gavlin 255620ea1e
[codegen/nodejs] Await invokes in async contexts. (#4598)
If we are generating code into an async context (e.g. an async main),
await calls to invoke rather than leaving them as promises. This results
in more idiomatic code withing such contexts.
2020-05-11 10:21:56 -07:00
Praneet Loke aa313aecf8
[codegen/docs] Add a new template for examples section (#4591)
* Add a new template for examples section. Extract the examples section into a structured format for custom template processing.

* Update the description IFF we were able to extract examples sections from it.

* Update doc comments and add missing file header for the newly added file.

* Make the example description readable. Add a check for empty example sections.

* Add a chooser right below the Example Usage header. Remove javascript as a language.

* Allow an empty new-line between short-codes boundaries.
2020-05-08 16:25:28 -07:00
Praneet Loke 5613867b3b
[codegen/docs] Add apiextensions as a k8s overlay resource (#4583)
* Add apiextensions as a k8s overlay. Add comment.
2020-05-08 11:13:21 -07:00
Pat Gavlin fc00c16e31
[codegen/hcl2] Fix iterator apply rewrites. (#4589)
There are two expressions in HCL2 that are used to iterate over
collections:

- Splat expressions, e.g. `foo.*.bar`, and
- For expressions, e.g. `[for v in foo: v.bar]`

In both of these cases, the parts of the expression that are not the
collection being iterated behave like callbacks, and must be treated as
such by the apply rewriter.
2020-05-07 12:15:40 -07:00
Pat Gavlin 8fded5f29f
[codegen/hcl2] Fix template control tokens. (#4584)
Token detection was broken for conditional and for expressions that
represent template control sequences. The code originally attempted to
determine whether or not a conditional or for expression was a control
sequence by inspecting the expression's parent. Unfortunately, that
approach is unable to distinguish between expressions that are control
sequences and those that are merely template parts. These changes
instead inspect the first token of the expression for a template control
token (i.e. `%{`): if such a token is found, the expression is detected
as a template control sequence.
2020-05-07 09:12:00 -07:00
Praneet Loke 3b9ec6590b
[codegen/docs] Fix kubernetes resource docs links for overlay resources (#4578)
* Adjust the args type names for k8s overlay resources.

* Revert to using input type link for kubernetes non-overlay resources.

* Use the correct module name to lookup the C# namespace for resources that belong to a module.

* Helm's arg types don't use the version number.
2020-05-06 12:11:31 -07:00
Pat Gavlin f23b98a7df
[codegen/nodejs] Fix #4563. (#4579)
Do not wrap non-output-typed arguments to `pulumi.all` in
`pulumi.output`. `pulumi.all` does this implicitly.

Fixes #4563.
2020-05-06 10:09:20 -07:00
Pat Gavlin 0e512aa0ef
[codegen/{nodejs,python}] Parse more proxy applies (#4576)
Specifically, handle index and relative traversal expressions, and clean
up the code a little bit.

This should also help us pick up more `pulumi.interpolate` calls in TS.
2020-05-05 21:51:53 -07:00
Pat Gavlin 7b446d6c55
Escape """ in Python doc comments. (#4570)
With the addition of Python examples, doc comments may now contain
triple quotes. These must be escaped in order to avoid malformed source
files.

Fixes #4568.
2020-05-05 12:39:35 -07:00
Pat Gavlin 96300f12d9
[codegen/*] Improve range type binding + codegen. (#4552)
- Determine variable types for ranged resources by typechecking an
  equivalent expression
- Detect top-level await in NodeJS and generate an async main
- Fix `pulumi.all` generation for NodeJS
- Fix a bug in the lowering of relative traversals in Python
2020-05-04 15:04:35 -07:00
Praneet Loke e66f4bd839
[codegen/docs] Remove exclusions used for Kubernetes' provider resource (#4551) 2020-05-03 16:27:33 -07:00
Paul Stack 3026395785
Add support for NPM Resolutions to nodejs codegen (#4554) 2020-05-02 22:17:31 +01:00
Praneet Loke fe3bd9ed99
[codegen/docs] Restore the API type links for C#. (#4543)
* Restore the API type links for C#.

* Also restore them in function.tmpl.

* Add package details to the Functions template as well. Add a global template function to detect if the APIDocLinks has links for a language. Don't generate C# API doc links for k8s.
2020-05-01 17:13:56 -07:00
Pat Gavlin d96f5263fb
Merge pull request #4548 from pulumi/pgavlin/codegenFixes
[hcl2] Fix package loading and trivia detection
2020-05-01 14:10:22 -07:00
Pat Gavlin 8c4c46052d Fix the entries() signature 2020-05-01 13:10:56 -07:00
Pat Gavlin 50b1049337 Fix a misspelling 2020-05-01 11:04:44 -07:00
Pat Gavlin 21634768aa [hcl2] Fix package loading and trivia detection
- Load packages referenced by all node types, not just resource nodes
- Fix trivia detection
2020-05-01 11:00:58 -07:00
Levi Blackstone cca94a5a71
Add helpers for downsteam Go codegen (#4546) 2020-05-01 11:52:07 -06:00
Pat Gavlin 4278595f1b [codegen/python] Fix case mapping in traversals.
This code was accidentally removed in a prior PR.

These changes also remove an assert; the situation the assertion guards
against is valid and occurs when dealing with traversals inside of
splat expressions.
2020-05-01 10:11:50 -07:00
Pat Gavlin 93a7dbe412
[hcl2/model] String literal + traversal fixes. (#4542)
- Properly escape string literals inside template expressions and
  heredocs
- Fix trailing trivia detection for scope and relative traversals
2020-05-01 06:29:39 -07:00
Mikhail Shilkov d91f658576
Codegen changes to prepare for C# Kubernetes schema (#4531)
Codegen changes to prepare for C# Kubernetes schema
2020-05-01 12:45:47 +02:00
Pat Gavlin ca5cb05482 [hcl2/model] String literal + traversal fixes.
- Properly escape string literals inside template expressions and
  heredocs
- Fix trailing trivia detection for scope and relative traversals
2020-04-30 16:38:36 -07:00
Pat Gavlin 1d6cce98fe
[codegen/python] Fix nested quotes. (#4539)
Unlike most languages with interpolated strings, Python's formatted
string literals do not allow the nesting of quotes. For example,
this expression is not legal Python:

    f"Foo {"bar"} baz"

If an interpolation requires quotes, those quotes nust differ from the
quotes used by the enclosing literal. We can fix the previous example
by rewriting it with single quotes:

    f"Foo {'bar'} baz"

However, this presents a problem if there are more than two levels of
nesting, as Python only has two kinds of quotes (four if the outermost
string uses """ or '''): in this case, the expression becomes
unspellable, and must be assigned to a local that is then used in place
of the original expression. So this:

    f"Foo {bar[f'index {baz["qux"]}']} zed"

becomes this:

    index = "qux"
    f"Foo {bar[f'index {baz[index]}']}"

To put it bluntly, Python code generation reqiures register allocation,
but for quotes. These changes implement exactly that.

These changes also include a fix for traversals that access values that
are dictionaries rather than objects, and must use indexers rather than
attributes.
2020-04-30 16:34:25 -07:00
Pat Gavlin 76a0f75f5e Fix a build break. 2020-04-30 14:52:38 -07:00
Pat Gavlin 5271eda247
[codegen/python] Fix case mapping. (#4538)
Mapping happens at package scope, not type scope. This is what causes
some of the surprising mixes of camel and snake casing in our packages.

In order to accommodate this, build the case mapping tables up front,
and refer to them when performing case mapping.
2020-04-30 14:16:56 -07:00
Pat Gavlin 1eff2b9acf
[hcl2/model] Propagate annotations in InputType. (#4535)
When creating the inputty version of an `ObjectType`, ensure that the
result carries any annotations present on the original type.
2020-04-30 14:16:43 -07:00
Pat Gavlin 6bd81de8aa
[hcl2] Fix tokens and printing for TraverseIndex. (#4536)
The tokens that make up the "key" portion of an index traversal
(e.g. `"foo"` in `a["foo"]`) are structured like those that make up a
block label: an open quote token, a string literal token, and a close
quote token. The token mapper did not account for that fact, and instead
recorded the key token as the open quote. These changes correct that
error, and adjust the code in `literalText` to allow for
properly-escaped and quoted strings where necessary.
2020-04-30 13:44:32 -07:00
Pat Gavlin 6ae15f6a44
[codegen/hcl] Fix binop printing. (#4537)
The code that handles automatic parenthesis insertion was broken for
operators with the same precedence.
2020-04-30 13:44:05 -07:00
Pat Gavlin b4daf94c2f
[codegen] Add support for caching package schemas. (#4534)
If a single process is going to bind and generate multiple programs, it
is useful to be able to cache package schemas in order to avoid the
(large) overhead of deserializing schemas multiple times.
2020-04-30 13:22:24 -07:00
Pat Gavlin 72a6add300
[codegen/python] Decrease alloc volume in PyName. (#4533)
Use strings.Builder instead of raw slices. We could go further here and
collapse to a single, ~right-sized bulder at the cost of complexity in
the `stateAcronym -> stateLowerOrNumber` transition.

Also, eliminate the pyName function, as it is redundant.
2020-04-30 13:22:13 -07:00
Praneet Loke 7b17463031
[codegen/docs] Fix the generation of Function names, args, and result type names (#4490)
* Add a new DocLangHelper interface method to generate function names per language. Use the functionNames override map in Go to correctly generate a function name.

* Add kong to the title lookup map.

* Check if item was found in map.

* Use title case for the nodejs Function args type name. Use title case for the function name in the index page.

* Also fix the result name for a Function in nodejs to use title case.

* Use the module name to lookup the package.

* Use the title-case name for the Function name in the index page. Fix the language value passed to the lang chooser in function.tmpl.

* Use title case for nested type titles.

* Add title lookup for GitHub.
2020-04-27 17:47:01 -07:00
Pat Gavlin 2e9499a000
[codegen/hcl2] Fix the apply rewriter. (#4486)
Some of the apply rewriter's assumptions were broken by the richer
expressions available in HCL2. These changes fix those broken
assumptions, in particular the assumption that only scope traversal
expressions are sources of eventual values.
2020-04-24 22:04:24 -07:00
Pat Gavlin dc2d91e294
[codegen/hcl2] Separate binding and typechecking. (#4483)
This allows consumers to change the inputs to an expression and re-run
typechecking to compute the expression's new type.
2020-04-24 08:52:52 -07:00
Pat Gavlin af344d9b72
Fix Python module names. (#4473)
In particular, ensure that they are keyword-safe. This affects
`aws:lambda:*` in particular: prior to these changes, we were generating
code into `pulumi_aws.lamdba`, which is not referencable due to its use
of the `lamdba` keyword. With these changes, we generate code into
`pulumi_aws.lambda_`.

There is also a small fix for multi-line non-formatted strings: these
strings do not need escaped braces.
2020-04-24 08:52:36 -07:00
Pat Gavlin 739634c702
Merge pull request #4475 from pulumi/pgavlin/applies
Apply rewriter: separate global and expr data
2020-04-24 08:52:14 -07:00
Paul Stack 090f390d7b
Ensure that DotNet codegen marks Deprecated classes as obsolete (#4482)
Related: https://github.com/pulumi/pulumi-terraform-bridge/issues/133
2020-04-24 01:18:29 +01:00
Praneet Loke ffeeed965f
[codegen/go] Handle recursive types when marking optional properties to use Ptr types in Go (#4471) 2020-04-22 16:49:57 -07:00
Pat Gavlin 18ecd1c8a9 Apply rewriter: separate global and expr data
This will make some incoming fixes that address nested applies easier to
write and understand.
2020-04-22 15:48:56 -07:00
Mikhail Shilkov b67c774fcc
C# name resolution (#4468) 2020-04-23 00:42:31 +02:00
Pat Gavlin 3a45d46046
Fix output property names in Python codegen. (#4460)
* Fix output property names in Python codegen.

Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.

Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.

The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.

* Fix Python input property names.

- Fix input property names and forms for invokes. Previously we
  generated a dict; now we generate properly-named args.
- Fix nested property names for resources.

* PR feedback
2020-04-22 10:25:19 -07:00
Praneet Loke 6f43784cf8
[codegen/docs] Add an exclusion to manually render the k8s overlay resource constructor params for Python (#4469) 2020-04-22 10:13:35 -07:00
Pat Gavlin 4f0aec56f6 PR feedback 2020-04-21 15:38:42 -07:00
Pat Gavlin 9204d70c54 Fix Python input property names.
- Fix input property names and forms for invokes. Previously we
  generated a dict; now we generate properly-named args.
- Fix nested property names for resources.
2020-04-21 13:55:33 -07:00
Luke Hoban 6afefe050f
[codegen/go] Fix accessors on struct ptr outputs (#4456)
* [codegen/go] Fix accessors on struct ptr outputs

The accesor methods on nestred struct Ptr outputs were previously not accepting pointer typed inputs as they should, and would thus always panic if used.

The "simple" fix would be to just accept the pointer type and blindly dereference it.  But this doesn't seem like the right experience - it would make these accessors very unsafe to use in practice.

Instead, this PR implements the accessors on pointer-typed outputs as nil-coaslescing, always lifting the output type into a pointer type and flowing a nil value into the result type.  This ensures the accessor will not nil-deref, and that user code can handle the `nil` value itself (or use `.Apply` directly to implement more specialized behaviour).

Before:

```go
// Name of your S3 bucket.
func (o BuildStorageLocationPtrOutput) Bucket() pulumi.StringOutput {
	return o.ApplyT(func(v BuildStorageLocation) string { return v.Bucket }).(pulumi.StringOutput)
}
```

After:

```go
// Name of your S3 bucket.
func (o BuildStorageLocationPtrOutput) Bucket() pulumi.StringPtrOutput {
	return o.ApplyT(func(v *BuildStorageLocation) *string {
		if v == nil {
			return nil
		}
		return &v.Bucket
	}).(pulumi.StringPtrOutput)
}
```

However, due to the decision to have this more usable behaviour, this is a breaking change, as some/many accessors now return a pointer type when they previously did not.

Fixes pulumi/pulumi-azure#530.

* Mark nested property types as requiring ptr types

* Add CHANGELOG

* More fixes
2020-04-21 13:33:38 -07:00
Pat Gavlin 1d672563da Fix output property names in Python codegen.
Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.

Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.

The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.
2020-04-21 10:25:27 -07:00
Pat Gavlin 807cf44cd7
Codegen: collect imports from function calls. (#4445)
Codegen: collect imports from function calls; fix Python resource names.

Just what it says on the tin.
2020-04-21 10:24:42 -07:00
Sean Holung 5d9e3626b3
[codegen/docs] Improve titles tags and meta descriptions (#4432)
* Optimize titles and descriptions

* Move regexp to file level variable

* wip

* Add title tags and description to module and pkg index pages

* Remove block_external_search_index from index.tmpl

* Improve resource descriptions

* Add more providers to look up map

* Improve module level descriptions

Clean up

cleanup

* Update the function.tmpl with the auto-generated disclaimer. Pass title tag and meta description for the function template as well.

* Fix whitespace in the package_details template..

* Account for empty module names for package-level resources. Move logic for generating title tag and description into separate methods for Resources and Functions.

* Add a test for the generated resource and function title tag.

Co-authored-by: Praneet Loke <1466314+praneetloke@users.noreply.github.com>
2020-04-21 12:28:44 +01:00
Pat Gavlin 1f19cbbe08
Clean up the language-specific schema APIs. (#4454)
Rather than forcing consumers to deal with language-specific data
ad-hoc, add an API that allows all language-specific data to be decoded
up-front.
2020-04-20 16:36:05 -07:00
Justin Van Patten 5328b6801f
Inject id resource output property in docs (#4443)
All resources have an implicit `id` output property that must be injected in docs.
2020-04-18 14:17:08 -07:00
Justin Van Patten a9fac0c51c
Add a period to the end of the Context object comment (#4440)
To match the other comments that have periods.
2020-04-18 13:06:43 -07:00