Commit graph

228 commits

Author SHA1 Message Date
Anton Tayanovskyy 4d4ff9f1d6
Fixes 8403 name conflicts in Go codegen (#8492)
* Apply fn renaming on Result name conflict

* Add tests

* Add test declaration to test driver

* Accept baselines for other langs
2021-11-24 20:13:47 -05:00
Ian Wahbe 4b7985384c
[codegen/go] Call site defaults for Pulumi Object types (#8411)
* Add test case

* Fix tests

* Add test dependencies correctly

* Feed through error handling

* Include test output

* Get types to line up

* Add remaining test files

* Update changelog

* Correctly find type paths

* Handle transitive objects

* Handle required fields

* Add feature flag for go

* Add required+default test case

* Don't `<any>` cast known types.

* Add more flags.

I realize this should really wait for PR#8400 to merge.

* Add plain object to env-helper test

This test fails right now. My next problem is fixing it.

* Handle plain types

* Handle function inputs

* Fix the indentation

* Handle output types correctly

* Remove unnecessary `!`

* Add test case

* Fix tests

* Add test dependencies correctly

* Feed through error handling

* Include test output

* Get types to line up

* Add remaining test files

* Update changelog

* Correctly find type paths

* Handle transitive objects

* Handle required fields

* Add required+default test case

* Don't `<any>` cast known types.

* Add plain object to env-helper test

This test fails right now. My next problem is fixing it.

* Handle plain types

* Handle function inputs

* Fix the indentation

* Handle output types correctly

* Remove unnecessary `!`

* Start on `genPlainObjectDefaultFunc`

* Add missing change to fix test

* Run tests with merge

* Refactor out assign

* Merge in next _index.md diff

* Change method name to `Defaults`

* Handle enums correctly

* Another attempt at _index.md

* Make module generation deterministic

* Add checks for old values

* Insert defaults in resources

* Fix docs generation

Credit to @praneetloke

* Progress on adding defaults to Resource arguments

* Handle resource argument defaults

* Don't create defaults if disableObjectDefaults

* Rename test folder

* Add test for disable flag

* Fix disable test

* Update docs

* Abstract out nil comparisons

* Use reflection to test for empty values

* Simplify Ptr and pulumi.Any type handling

* Remove unused function

* Apply defaults to functions

* Update new test with master codegen

* Tests + nil check
2021-11-23 15:10:15 -08:00
Anton Tayanovskyy 6cb801cf17
Add a unit test to check Array/Index (#8486) 2021-11-23 16:52:02 -05:00
Pat Gavlin 52d01bb915
[codegen/go] Remove ResourcePtr input/output types (#8449)
These changes remove the `Ptr` variants of input/ouptut types for
resources. A `TPtr` input or output is normally generated for `T` if `T`
is present in an `optional(input(T))` or `optional(output(T))` and if
the Go representation for `T` is not nilable. The generation of `Ptr`
variants for resource types breaks the latter rule: the canonical
representation of a resource type named `Foo` is a pointer to a struct
type named `Foo` (i.e. `*Foo`). `Foo` itself is not a resource, as it
does not implement the Go `Resource` interface. Because this
representation already accommodates `nil` to indicate the lack of a
value, we need not generate `FooPtr{Input,Output}` types.

Besides being unnecessary, the implementation of `Ptr` types for
resources was incorrect. Rather than using `**Foo` as their element
type, these types use `*Foo`--identical to the element type used for
the normal input/output types. Furthermore, the generated code for
at least `FooOutput.ToFooPtrOutputWithContext` and `FooPtrOutput.Elem`
was incorrect, making these types virtually unusable in practice.

Finally, these `Ptr` types should never appear on input/output
properties in practice, as the logic we use to generate input and output
type references never generates them for `optional({input,output}(T)).
Instead, it generates references to the standard input/output types.

Though this is _technically_ a breaking change--it changes the set of
exported types for any package that defines resources--I believe that in
practice it will be invisible to users for the reasons stated above.
These types are not usable, and were never referenced.

This is preparatory work for #7943.
2021-11-23 10:24:56 -08:00
Anton Tayanovskyy f98d39b84b
Fix non-determinism in docsgen (#8456) 2021-11-19 11:23:06 -05:00
Anton Tayanovskyy e60d6bf248
Programgen support for F.Invoke forms in .NET (#7949) (#8432)
* Implement .NET codegen for F.Invoke forms

* Add tests for .NET Invoke

* Fixed conflict

* Accept changes reverting fargate example
2021-11-18 17:53:17 -05:00
Anton Tayanovskyy e846b3201f
Go implementation of fn.Output program gen (#7949) (#8431)
* Modify Go codegen for fn.Output overloads

* Fix go tests

* Fixed CHANGELOG conflict

* ACCEPT changes reverting Fargate example to master, file issue

* Link an issue
2021-11-18 17:50:51 -05:00
Anton Tayanovskyy 1cedb29193
Programgen support for fn_output forms in Python (#7949) (#8433)
* Update Python programgen to use fn_output forms

* Fix skips

* Fix CHANGELOG

* Accept changes, backing out Fargate change
2021-11-18 17:43:13 -05:00
Ian Wahbe 3e2f36548e
[codegen/typescript] Call site defaults for plain Pulumi Object types (#8400)
* Add test case

* Fix tests

* Add test dependencies correctly

* Feed through error handling

* Include test output

* Get types to line up

* Add remaining test files

* Update changelog

* Correctly find type paths

* Handle transitive objects

* Handle required fields

* Add required+default test case

* Don't `<any>` cast known types.

* Add plain object to env-helper test

This test fails right now. My next problem is fixing it.

* Handle plain types

* Handle function inputs

* Fix the indentation

* Handle output types correctly

* Remove unnecessary `!`

* Add missing change to fix test

* Run tests with merge

* Merge in next _index.md diff

* Another attempt at _index.md

* Make module generation deterministic

* Fix docs generation

Credit to @praneetloke
2021-11-18 12:23:30 -08:00
Anton Tayanovskyy 06a19b53ed
Programgen support for fnOutput forms in node (#7949) (#8434)
* Teach PCL about fnOutput forms

* Teach PCL about fnOutput forms

* Teach Node program gen to emit fnOutput forms

* TypeCheck fix

* AWS package bump

* Add tests

* CHANGELOG

* Temporarily skip non-Node affected tests

* Address PR feedback: restrict new form to Output args only
2021-11-17 15:27:50 -05:00
Pat Gavlin a1339277f0
[schema] Add enum overlay support. (#8425)
And update the metaschema to accommodate the `isOverlay` properties
added in #8338. Overlay enums, like other overlay members, are
implemented out-of-band by the declaring package. Code generators should
not generate declarations for overlay enums.
2021-11-16 15:53:28 -08:00
Anton Tayanovskyy 372ddc7e5c
Skip flaky tests for now (#8420) 2021-11-15 15:17:20 -05:00
Ian Wahbe 272c4643b2
Update error handling (#8406)
This is the result of a change applied via `go-rewrap-errors`.
2021-11-12 18:37:17 -08:00
Levi Blackstone 0d4fb3e340
[schema] Add IsOverlay option to disable codegen for particular types (#8338)
Add a new `IsOverlay` option to schema types and functions that allows providers to document overlays in the schema. This makes it easier to generate API docs consistently, even for code that is generated outside of the typical codegen process.
2021-11-11 17:00:03 -07:00
Anton Tayanovskyy afcbcd3a51
7947: docgen for $fnOutput forms (#8293)
* Toward doc gen of fn.Output version signatures

* Fixup Python docgen, and reorder forms so the direct form comes firs

* Respect go opt-out flag

* Fix tempalte bug with unbalanced HTML tags

* Edit CHANGELOG_PENDING.md

* Merge codeblocks in the function template

* Accept docs changes
2021-11-08 17:07:09 -05:00
Anton Tayanovskyy 99fdad0ed9
Fix python dependencies issues in GHA (#8361) 2021-11-06 00:55:17 -07:00
Justin Van Patten 414ef7e9d0
[programgen/go] Don't change imported resource names (#8353)
When importing a resource in Go, we change resource names with hyphens to snake_case. This happens because we introduced a change that converts the resource name to a valid Go identifier so it can be used as a local variable. But we were also using this converted name as the string resource name, which causes problems with import: the imported resource in the state file has the original resource name, but the generated program uses the converted name, causing Pulumi to delete and recreate the resource when adopting the generated import code.

This change fixes the issue by maintaining the original resource name, and only using the converted name for variables.
2021-11-04 09:28:48 -07:00
Anton Tayanovskyy 343b4a3501
Fix #8172 (#8320)
* Fix #8172

* Accept test baselines

* Update CHANGELOG_PENDING
2021-11-01 13:21:01 -04:00
Ian Wahbe 530641576d
[codegen/go] allow plain default types (#8254)
* Respect `plain` when generating default values.

* Add new test

* Simplify test case

* Add indirection to allow taking references

* Reflect indirection problem in tests

* Remove Plain fields. Base off of primitive type

* Reenable docs

* Update changelog

* Implement always default solution

* Fix test by pulling in master

* Add enum test and cleanup

* Fix const handling

* Clarify the changelog
2021-10-26 17:18:48 -07:00
Anton Tayanovskyy cb51313512
Cover GetAmiIds at unit test level (#8261) 2021-10-19 18:18:25 -04:00
Ian Wahbe f12b8fa00c
Cleanup program-gen test dependency gathering (#8248) 2021-10-19 12:17:19 -07:00
Anton Tayanovskyy e710125885
5758 for C#/.NET (#7899)
* Rebase 5758 .NET work and make output-funcs pass

* Propagate changes to the other examples

* CHANGELOG

* Address PR feedback

* Add a test reproducing aws-native compilation failure

* Fix dangling type ref issue in the .NET backend for codegen

* Accept changes and unskip simple-methods-schema compile check

* Accept changes in node, python, go codegen

* SDK changes to enable a better implementation approach

* Switch approach to support functions like GetAmiIds; avoid name conflicts under tfbridge20

* Make all dotnet tests pass, mechanical fixes + accept test output

* Accept python changes

* Accept node output

* Accept docs changes

* Deepen the unit test to cover the interesting helper type

* Accept go changes and fixup tests

* Implement dep propagation through Invoke

* Fixup cyclic-types

* Accept codegen

* NOTE we now require .NET SDK 3.15 or higher
2021-10-18 18:18:15 -04:00
Praneet Loke 0f5b7c765b
[codegen/docs] Update API docs templates to include layout front-matter param (#8242)
* Update generation templates

* Updated templates for codegen

* Rm api-doc-resource template

* Update templates to use layout:api and no alias

* Set no_edit_this_page to true in the header and index templates used by docs gen.

* Update docs generation baseline test files with template updates

* Remove the notion of menu from resource templates (#8200)

* Update the title lookup map entry for aws-api-gateway

* Updated CSS classes for icons in API docs template

* Updated tests:

* Additional test file updates

* Fixup package names

Co-authored-by: Devon Grove <devon@pulumi.com>
Co-authored-by: Christian Nunciato <c@nunciato.org>
Co-authored-by: stack72 <public@paulstack.co.uk>
2021-10-17 22:36:31 -07:00
Ian Wahbe dbc5979501
iwahbe/add aditional input registration types (#8204)
* Add registration for resources and enums

* Update tests

* Fix enum registration instances

* Update changelog

* Add the rest of the reference types

* Fix nit

* Remove reference from Map and Array types
2021-10-14 10:41:40 -07:00
Ian Wahbe 208c6ec44d
Add flag for input registration to the go schema (#8198)
* Add flag

* Update CHANGELOG_PENDING.md

* Add this feature to an orthogonal test
2021-10-12 10:15:24 -07:00
Ian Wahbe b978a9f2d2
Accommodate - in base segments for baseImportPaths (#8190) 2021-10-12 09:13:13 -07:00
Justin Van Patten 789e392194
[codegen/go] Emit input type registrations (#7959)
Register an appropriate input type with the runtime for each object
type and composite thereof (e.g. pointer, array, etc.). This does not
cover registrations for enum and resource input types, which will be
added with future changes.

Co-authored-by: Pat Gavlin <pat@pulumi.com>
2021-10-07 15:26:24 -07:00
Anton Tayanovskyy 6f8d411bc4
5758 for Node JS (#8047)
* Implement 5758 for Node JS

* Codegen tests

* Remove typo
2021-10-07 15:39:19 -04:00
Anton Tayanovskyy 5f9f38c9c9
Do not share maps so tests can run in parallel (#8136)
* Do not share maps so tests can run in parallel

* Fix comment

* Try not to break dependencies

* Address PR feedback

* Fix downstream compilation failure

* Fix lint

* Address PR feedback
2021-10-06 11:03:21 -04:00
Anton Tayanovskyy 1ee85d8d2f
Fix 8110 (#8116)
* Add a test demonstrating the problem

* Fix the bug

* Accept changes over the example set

* Accept go changes

* Accept python codegen; avoid nil lang info panic

* Accept .NET changes, compile

* Accept docs changes

* Add changelog notes

* Fix lint

* Add parens sparingly

* Flatten unions

* PR feedback
2021-10-04 17:26:49 -04:00
Justin Van Patten 230f4ee006
Support "lifting" single-valued method returns to their return type (#8111)
This change allows lifting single valued method returns rather than using an output structure.
2021-10-01 11:33:02 -07:00
Pat Gavlin f21eda521f
[codegen] Rename the PCL package. (#8103)
It's just confusing that PCL lives in a package named `hcl2`.
2021-09-29 20:11:56 -07:00
Ian Wahbe 9f6589ed8c
iwahbe/7802/compile program generator test output (#8036)
* Move program tests into folders

* update package schema

* Enabled tests pass

* Fix lints and begin to update test cases

* Re-enable tests

* Update aws version to v4

* Refactor language specific parts

* Hook up dotnet and nodejs

* Update tests from master

* SSOT for schema/version

* Name blocking errors. Leave tests in valid state

* Give each language its own folder

* Remove SkipCompile for azure-sa (bug was fixed)

* Fix nits + changes asserts to require

* Remove unused import

* One last assert => require
2021-09-29 11:33:57 -07:00
Pat Gavlin 0a45f8d3ab
[codegen/go] Rewrite cyclic types. (#8049)
When computing the type name for a field of an object type, we must
ensure that we do not generate invalid recursive struct types. A struct
type T contains invalid recursion if the closure of its fields and its
struct-typed fields' fields includes a field of type T. A few examples:

Directly invalid:

    type T struct { Invalid T }

Indirectly invalid:

    type T struct { Invalid S }

    type S struct { Invalid T }

In order to avoid generating invalid struct types, we replace all
references to types involved in a cyclical definition with *T. The
examples above therefore become:

(1) type T struct { Valid *T }

(2) type T struct { Valid *S }

    type S struct { Valid *T }

We do this using a rewriter that turns all fields involved in reference
cycles into optional fields.

These changes also include an enhancement to the SDK codegen test
driver in the interest of making iterating and debugging more convenient:  if the -sdk.no-checks flag is passed, the driver will not run post-generation checks.
2021-09-28 07:33:14 -07:00
Komal 74dfa83de5
[codegen/nodejs] - Fix provider enum with env var (#8051)
* Add failing test

* Fix

* PR feedback
2021-09-24 15:02:56 -07:00
Anton Tayanovskyy e14f74fbe7
Do report stdout and stderr in case of failure (#8055)
* Do report stdout and stderr in case of failure

* PR feedback
2021-09-24 15:25:30 -04:00
Komal b8b6f3dd70
[codegen/go] - Resolve type name collisions (#7985) 2021-09-23 15:42:48 -07:00
Ian Wahbe d43b8984fd
iwahbe/turn on dotnet compile checks (#8044)
* Add dotnet tests back in

* Merge in new test results from master

* re-enable output-funcs code
2021-09-23 14:31:17 -07:00
Anton Tayanovskyy 3f6f9bc398
Fix build by accepting nodejs codegen output (#8045) 2021-09-23 15:59:48 -04:00
Anton Tayanovskyy 49ccd9ac97
Simplify output-funcs codegen test (#8039)
* Consolidate output-funcs into a single normal schema.json

* Accept nodejs codegen output

* Accept dotnet output-funcs output; does not compile yet

* Accept docs output-funcs output

* Permit parallel test runs

* Accept nodejs codegen

* Fix and speed up Python codegen tests

* Dedup dash-named-schema

* Make dotnet tests pass

* Satisfy go lint
2021-09-23 13:42:20 -04:00
Komal 41b8882fe8
[codegen/nodejs] - Don't import types for functions if they don't exist (#8038)
* Add failing test

* Fix

* Update changelog

* Existing tests are sufficient
2021-09-23 10:12:27 -07:00
Ian Wahbe c9b719c6c0
Revert "Emit schema.Package.Version when possible (#7938)" (#8035)
* Revert "Emit schema.Package.Version when possible (#7938)"

This reverts commit 906dce73a1.

* Remove dotnet SDK compile checks

* Add to changelog
2021-09-22 19:39:07 -07:00
Anton Tayanovskyy 09aa2c677e
Fix master build (#8025)
* Fixing build failure

* Python manifest

* Dotnet manifest

* Node manifest

* Docs manifest

* Fix/restore test case running
2021-09-22 14:53:30 -04:00
Anton Tayanovskyy 49298fb433
Codegen testing upgrades (#7989)
* Multi-pass, in-place checks for SDK codegen tests; toward working Python checks

* Remove temp debug output

* Upgrade Node

* Update dotnet; need to follow up on version.txt quirks

* WIP

* Sounds like we can use non-github package names to ensure things are local

* Fix simple-enum-schema

* Fix dash-named-schema

* Fix nested-module

* Start building a test-running pass

* Infer skipping tests from skipping compiles

* Move tree schma tests to a proper place

* Address lint issues on Go code

* Build against local Go SDK

* Update pkg/codegen/internal/test/sdk_driver.go

Co-authored-by: Ian Wahbe <ian@wahbe.com>

* Make go tests work by copying them into the tree from go-extras

* Fix lint

* Fix bad merge

* Manifest-based file discovery

* Remove version-related TODO from dotnet codegen

* Add doc comment

* Do not overwrite go.mod if found from mixins

* Accept python codegen change

* Accept node codegen

* Ignore lint issue

* Accept docs changes

Co-authored-by: Ian Wahbe <ian@wahbe.com>
2021-09-22 13:55:20 -04:00
Evan Boyle adc0f2adeb
Use case insensitive checks when detecting duplicate type tokens (#8017) 2021-09-21 20:48:45 -07:00
Ian Wahbe 2e5fedff54
Switch from golint to revive (#8010) 2021-09-21 10:00:44 -07:00
Pat Gavlin 236ce54269
[schema] Add the Pulumi Package metaschema. (#7952)
The Pulumi Package metaschema is a JSON schema definition that describes
the format of a Pulumi Package schema. The metaschema can be used to
validate certain basic properties of a Pulumi Package schema, including
(but not limited to):

- data types (e.g. is this property a string?)
- data formats (e.g. is this string property a valid regex?)
- object shapes (e.g. is this object missing required properties?)

The schema binder has been updated to use the metaschema as its first
validation pass.

In addition to its use in the binder, the metaschema has its own page in
the developer documentation. This page is generated using a small tool,
jsonschema2md.go.
2021-09-20 12:00:42 -07:00
Ian Wahbe c338876b9f
iwahbe/7858/fix nodejs hypen imports (#7993)
* Add pascal name case

* Add test to prevent regressions

* Update CHANGELOG_PENDING.md

* Give node/tsc more memory

* Use camelcase instead of snake case

* Add clearer comments
2021-09-20 10:11:44 -07:00
Ian Wahbe 906dce73a1
Emit schema.Package.Version when possible (#7938)
* Emit schema.Package.Version when possible

* Update CHANGELOG_PENDING.md

* Correctly interpret python versions (I hope)

* Update PLUGIN_VERSION to the package version

* Modify tests to conform with master merge
2021-09-17 23:11:54 -07:00
Ian Wahbe 7f90e12886
Validate Name, Version and Enviroment (#7896)
* Validate Name, Version and Enviroment

For the full path:
Package.Name
Package.Version
Package.Property.Default

* Update tests

* Update CHANGELOG_PENDING.md

* Add more versions to tests

* Add another "Version" field

* Even more "version" tags

* One more "version" tag added

* Update test results from codegen

* Fix py codegen tests

* Fix doc test

* Remove `version` validation

* Unformat json files

* Fail only on errors
2021-09-17 12:12:22 -07:00