Commit graph

3091 commits

Author SHA1 Message Date
Anton Tayanovskyy 4d4ff9f1d6
Fixes 8403 name conflicts in Go codegen (#8492)
* Apply fn renaming on Result name conflict

* Add tests

* Add test declaration to test driver

* Accept baselines for other langs
2021-11-24 20:13:47 -05:00
Fraser Waters 2d26cdd9ed
Run validate function even if dryRun=true (#8494)
Hit this while trying to add some validation checks to runs of Preview while using plans. Seems one test actually was assuming this was the case already and just hasn't been running it's expected validate function.
2021-11-24 22:13:29 +00:00
Ian Wahbe 4b7985384c
[codegen/go] Call site defaults for Pulumi Object types (#8411)
* Add test case

* Fix tests

* Add test dependencies correctly

* Feed through error handling

* Include test output

* Get types to line up

* Add remaining test files

* Update changelog

* Correctly find type paths

* Handle transitive objects

* Handle required fields

* Add feature flag for go

* Add required+default test case

* Don't `<any>` cast known types.

* Add more flags.

I realize this should really wait for PR#8400 to merge.

* Add plain object to env-helper test

This test fails right now. My next problem is fixing it.

* Handle plain types

* Handle function inputs

* Fix the indentation

* Handle output types correctly

* Remove unnecessary `!`

* Add test case

* Fix tests

* Add test dependencies correctly

* Feed through error handling

* Include test output

* Get types to line up

* Add remaining test files

* Update changelog

* Correctly find type paths

* Handle transitive objects

* Handle required fields

* Add required+default test case

* Don't `<any>` cast known types.

* Add plain object to env-helper test

This test fails right now. My next problem is fixing it.

* Handle plain types

* Handle function inputs

* Fix the indentation

* Handle output types correctly

* Remove unnecessary `!`

* Start on `genPlainObjectDefaultFunc`

* Add missing change to fix test

* Run tests with merge

* Refactor out assign

* Merge in next _index.md diff

* Change method name to `Defaults`

* Handle enums correctly

* Another attempt at _index.md

* Make module generation deterministic

* Add checks for old values

* Insert defaults in resources

* Fix docs generation

Credit to @praneetloke

* Progress on adding defaults to Resource arguments

* Handle resource argument defaults

* Don't create defaults if disableObjectDefaults

* Rename test folder

* Add test for disable flag

* Fix disable test

* Update docs

* Abstract out nil comparisons

* Use reflection to test for empty values

* Simplify Ptr and pulumi.Any type handling

* Remove unused function

* Apply defaults to functions

* Update new test with master codegen

* Tests + nil check
2021-11-23 15:10:15 -08:00
Anton Tayanovskyy 6cb801cf17
Add a unit test to check Array/Index (#8486) 2021-11-23 16:52:02 -05:00
Pat Gavlin 52d01bb915
[codegen/go] Remove ResourcePtr input/output types (#8449)
These changes remove the `Ptr` variants of input/ouptut types for
resources. A `TPtr` input or output is normally generated for `T` if `T`
is present in an `optional(input(T))` or `optional(output(T))` and if
the Go representation for `T` is not nilable. The generation of `Ptr`
variants for resource types breaks the latter rule: the canonical
representation of a resource type named `Foo` is a pointer to a struct
type named `Foo` (i.e. `*Foo`). `Foo` itself is not a resource, as it
does not implement the Go `Resource` interface. Because this
representation already accommodates `nil` to indicate the lack of a
value, we need not generate `FooPtr{Input,Output}` types.

Besides being unnecessary, the implementation of `Ptr` types for
resources was incorrect. Rather than using `**Foo` as their element
type, these types use `*Foo`--identical to the element type used for
the normal input/output types. Furthermore, the generated code for
at least `FooOutput.ToFooPtrOutputWithContext` and `FooPtrOutput.Elem`
was incorrect, making these types virtually unusable in practice.

Finally, these `Ptr` types should never appear on input/output
properties in practice, as the logic we use to generate input and output
type references never generates them for `optional({input,output}(T)).
Instead, it generates references to the standard input/output types.

Though this is _technically_ a breaking change--it changes the set of
exported types for any package that defines resources--I believe that in
practice it will be invisible to users for the reasons stated above.
These types are not usable, and were never referenced.

This is preparatory work for #7943.
2021-11-23 10:24:56 -08:00
Ian Wahbe 72a4e1fc3f
Update pkg -> sdk dependency (#8481) 2021-11-22 14:39:54 -08:00
raphaelauv dbc8ab9ad6
[FIX] unprotect cmd example (#8430)
* [FIX] unprotect cmd example
2021-11-19 12:39:34 -08:00
Praneet Loke 2c25c3fbd0
[backend/filestate] Don't unwrap go-cloud errors (#8455)
* Don't unwrap go-cloud errors

* Remove unused func

* Add changelog entry

Co-authored-by: Ian Wahbe <ian@wahbe.com>
2021-11-19 12:21:37 -08:00
Anton Tayanovskyy f98d39b84b
Fix non-determinism in docsgen (#8456) 2021-11-19 11:23:06 -05:00
Anton Tayanovskyy e60d6bf248
Programgen support for F.Invoke forms in .NET (#7949) (#8432)
* Implement .NET codegen for F.Invoke forms

* Add tests for .NET Invoke

* Fixed conflict

* Accept changes reverting fargate example
2021-11-18 17:53:17 -05:00
Anton Tayanovskyy e846b3201f
Go implementation of fn.Output program gen (#7949) (#8431)
* Modify Go codegen for fn.Output overloads

* Fix go tests

* Fixed CHANGELOG conflict

* ACCEPT changes reverting Fargate example to master, file issue

* Link an issue
2021-11-18 17:50:51 -05:00
Anton Tayanovskyy 1cedb29193
Programgen support for fn_output forms in Python (#7949) (#8433)
* Update Python programgen to use fn_output forms

* Fix skips

* Fix CHANGELOG

* Accept changes, backing out Fargate change
2021-11-18 17:43:13 -05:00
Ian Wahbe 3e2f36548e
[codegen/typescript] Call site defaults for plain Pulumi Object types (#8400)
* Add test case

* Fix tests

* Add test dependencies correctly

* Feed through error handling

* Include test output

* Get types to line up

* Add remaining test files

* Update changelog

* Correctly find type paths

* Handle transitive objects

* Handle required fields

* Add required+default test case

* Don't `<any>` cast known types.

* Add plain object to env-helper test

This test fails right now. My next problem is fixing it.

* Handle plain types

* Handle function inputs

* Fix the indentation

* Handle output types correctly

* Remove unnecessary `!`

* Add missing change to fix test

* Run tests with merge

* Merge in next _index.md diff

* Another attempt at _index.md

* Make module generation deterministic

* Fix docs generation

Credit to @praneetloke
2021-11-18 12:23:30 -08:00
Ian Wahbe b0dfad44c5
Update pkg -> sdk dependency (#8442) 2021-11-17 15:24:09 -08:00
Anton Tayanovskyy 06a19b53ed
Programgen support for fnOutput forms in node (#7949) (#8434)
* Teach PCL about fnOutput forms

* Teach PCL about fnOutput forms

* Teach Node program gen to emit fnOutput forms

* TypeCheck fix

* AWS package bump

* Add tests

* CHANGELOG

* Temporarily skip non-Node affected tests

* Address PR feedback: restrict new form to Output args only
2021-11-17 15:27:50 -05:00
Ian Wahbe b9f57bc6b9
Move /opt/pulumi to $HOME/.pulumi (#8437)
* Move `/opt/pulumi` to `$HOME/.pulumi`

* Don't suggest adding $HOME/.pulumi to PATH
2021-11-17 10:37:38 -08:00
Ian Wahbe 3329d81c1a
Update command respects --target-dependents (#8395)
* Update command respects `--target-dependents`

* Update CHANGELOG_PENDING.md

* Depend on parent and provider

* Add tests for new feature

* Separate predicate and mutation in code

* Remove `targetDependentsForUpdate`

* Refactor `isTargetedForUpdate`

* Add very important nil check
2021-11-16 17:12:36 -08:00
Pat Gavlin a1339277f0
[schema] Add enum overlay support. (#8425)
And update the metaschema to accommodate the `isOverlay` properties
added in #8338. Overlay enums, like other overlay members, are
implemented out-of-band by the declaring package. Code generators should
not generate declarations for overlay enums.
2021-11-16 15:53:28 -08:00
Anton Tayanovskyy 372ddc7e5c
Skip flaky tests for now (#8420) 2021-11-15 15:17:20 -05:00
Ian Wahbe 554660b23a
Implement the --exclude-protected feature (#8359)
* Implement the --exclude-protected feature

This piggybacks on the same machinery used by the --target flag. By
examining the stack, we find a list of all resources managed by
Pulumi (in that stack). We then form them into a DAG, and mark all
resources as either protected or unprotected.

A resource is protected it has the `Protect` flag set or is has a child
with the `protect` flag set. It is unprotected otherwise.

We then pass the urns of unprotected resources to the update options
passed to the destroy operation in the same way that `--target` does.

* Update changelog

* Handle providers correctly

* Add integration test

* Protect dependencies of protected resources

* Handle --exclude-protected in separate function

* Simplify implementation via DependencyGraph

* Add TransitiveDependenciesOf

* Cleanup unused functions

* Gate printed message behind !jsonDisplay

* Ensure provider is not `""`

* Clean up documentation (and some code)
2021-11-15 11:45:14 -08:00
Ian Wahbe 272c4643b2
Update error handling (#8406)
This is the result of a change applied via `go-rewrap-errors`.
2021-11-12 18:37:17 -08:00
Anton Tayanovskyy 5dd7851293
PBT for dependency_graph (#8404) 2021-11-12 15:12:48 -05:00
Paul Stack 74ba28ad55
[CLI] Adding the ability to create a default org for backends that support orgs (#8352) 2021-11-12 20:44:51 +02:00
Joe Duffy 0a38bc295c
Fix issue with --target deletion dependent calculation (#8360)
* Fix issue with --target deletion dependant calculation

The code that computed --target deletion dependants was not correct.
It used parent/child component relationships, but did not respect actual
DAG dependencies. As a result, it could erroneously leave hanging
references to resources that no longer exist after performing a
`pulumi destroy --target X` operation. This manifested in bugs like
https://github.com/pulumi/pulumi/issues/6283, which is fixed by this
change. The solution is to compute the (transitive!) dependency graph
correctly, factoring in both parent/child, as well as explicit and
implicit, dependencies. The existing logic does the correct thing once
we do this. I've also added tests for this area, including regression
tests that cover transitive dependency relationships, as well as ones
that would cause an infinite loop given a naive implementation.

* Add a changelog entry

* Fix failing test to include all destroyed targets

Unless I'm missing something, the entire tree should be deleted
in this test case because A is the ancestor for the entire tree.

* Use DependencyGraph to compute dependents

Per code review feedback from @pgavlin.

Co-authored-by: Anton Tayanovskyy <anton@pulumi.com>
2021-11-12 10:02:51 -05:00
Levi Blackstone 0d4fb3e340
[schema] Add IsOverlay option to disable codegen for particular types (#8338)
Add a new `IsOverlay` option to schema types and functions that allows providers to document overlays in the schema. This makes it easier to generate API docs consistently, even for code that is generated outside of the typical codegen process.
2021-11-11 17:00:03 -07:00
Praneet Loke 0343a6de76
[codegen/docs] Use go:embed for templates and use DisplayName from schema for titles (#8389)
* Use the display name from the schema, if available

* Update comment on the titleLookup map

* Add examples of reserved labels that can be used with the Keywords array in the schema
2021-11-10 11:07:43 -08:00
Vivek Lakshmanan d7ec7ce06c
Merge pull request #8394 from pulumi/vl/Integration
Move AssertHTTPResultWithRetry from pulumi-eks utils
2021-11-09 16:56:25 -08:00
Vivek Lakshmanan aa9bed5508 Fix linting 2021-11-09 15:41:28 -08:00
Vivek Lakshmanan d401912902 Move AssertHTTPResultWithRetry from pulumi-eks since its usable across integration tests 2021-11-09 14:41:09 -08:00
Anton Tayanovskyy ac3f0a36c2
Update pkg -> sdk dependency for 3.17.1 (#8391) 2021-11-09 16:54:45 -05:00
Anton Tayanovskyy afcbcd3a51
7947: docgen for $fnOutput forms (#8293)
* Toward doc gen of fn.Output version signatures

* Fixup Python docgen, and reorder forms so the direct form comes firs

* Respect go opt-out flag

* Fix tempalte bug with unbalanced HTML tags

* Edit CHANGELOG_PENDING.md

* Merge codeblocks in the function template

* Accept docs changes
2021-11-08 17:07:09 -05:00
Anton Tayanovskyy 99fdad0ed9
Fix python dependencies issues in GHA (#8361) 2021-11-06 00:55:17 -07:00
Justin Van Patten 414ef7e9d0
[programgen/go] Don't change imported resource names (#8353)
When importing a resource in Go, we change resource names with hyphens to snake_case. This happens because we introduced a change that converts the resource name to a valid Go identifier so it can be used as a local variable. But we were also using this converted name as the string resource name, which causes problems with import: the imported resource in the state file has the original resource name, but the generated program uses the converted name, causing Pulumi to delete and recreate the resource when adopting the generated import code.

This change fixes the issue by maintaining the original resource name, and only using the converted name for variables.
2021-11-04 09:28:48 -07:00
Fraser Waters b58c39476f
Fix cmdutil.PrintTable to handle ansi escapes and non-byte glyphs (#8344)
Fixes two bugs in how padding was calculated in PrintTable.

Firstly we remove all ANSI escape codes from the string before measuring
how wide it is. Secondly we measure glyph count (using rivo/uniseg) not
byte or rune count of the string.

Together these fix the padding/alignment issues I saw when using
PrintTable with plan output. They also slightly change the layout of
"pulumi stack", for example the below is printed with current master and
has 6 characters of space for padding between SecurityGroup and
web-secgrp:

```
Current stack resources (4):
    TYPE                                        NAME
    pulumi:pulumi:Stack                         aws-cs-webserver-test
    ├─ aws:ec2/securityGroup:SecurityGroup      web-secgrp
    ├─ aws:ec2/instance:Instance                web-server-www
    └─ pulumi:providers:aws                     default_4_25_0
```

While printed with this commit you only get 2 characters of space for
padding (which is correct, the column gap is set to "  "):
```
Current stack resources (4):
    TYPE                                    NAME
    pulumi:pulumi:Stack                     aws-cs-webserver-test
    ├─ aws:ec2/securityGroup:SecurityGroup  web-secgrp
    ├─ aws:ec2/instance:Instance            web-server-www
    └─ pulumi:providers:aws                 default_4_25_0
```
2021-11-04 10:06:20 +00:00
Justin Van Patten f989e1cc9d
Update pkg -> sdk dependency (#8346) 2021-11-03 11:25:20 -07:00
Pat Gavlin af151b60c5
[codegen/go] Slightly simplify enum type gen. (#8341)
- Remove enumElementType, whose functionality duplicates work done in
  argsTypeImpl
- Clarify the args- vs. Go-ness of the two element types

This is prep work for #7943. These changes should not produce codegen diffs.
2021-11-02 17:41:06 -07:00
Anton Tayanovskyy 343b4a3501
Fix #8172 (#8320)
* Fix #8172

* Accept test baselines

* Update CHANGELOG_PENDING
2021-11-01 13:21:01 -04:00
Praneet Loke 9a78ca1ca4
[codegen/schema] Add DisplayName and Publisher to the schema (#8276) 2021-10-27 12:53:09 -07:00
Ian Wahbe 530641576d
[codegen/go] allow plain default types (#8254)
* Respect `plain` when generating default values.

* Add new test

* Simplify test case

* Add indirection to allow taking references

* Reflect indirection problem in tests

* Remove Plain fields. Base off of primitive type

* Reenable docs

* Update changelog

* Implement always default solution

* Fix test by pulling in master

* Add enum test and cleanup

* Fix const handling

* Clarify the changelog
2021-10-26 17:18:48 -07:00
Komal 2f433d64b7
[cli] Emit JSON events for updates via --json flag (#8275) 2021-10-26 16:21:27 -07:00
Ben Schiborr df61004e3f
Ability to require minimum Python version (#8296)
Co-authored-by: Komal <komalsali@gmail.com>
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-10-26 14:33:21 -07:00
svangordon-fruit 22d2ef247f
Allow hyphens in middle portion of token (#8282) 2021-10-25 17:15:17 -07:00
Anton Tayanovskyy b8514450b5
Update pkg -> sdk dependency for v3.16.0 (#8271) 2021-10-20 14:11:38 -04:00
Praneet Loke d73c380bef
[codegen/docs] Sort the package tree items (#8262)
* Sort the modules, resources and functions individually as the package tree is created

* Update package tree test
2021-10-19 15:21:39 -07:00
Anton Tayanovskyy cb51313512
Cover GetAmiIds at unit test level (#8261) 2021-10-19 18:18:25 -04:00
Ian Wahbe f12b8fa00c
Cleanup program-gen test dependency gathering (#8248) 2021-10-19 12:17:19 -07:00
Anton Tayanovskyy e710125885
5758 for C#/.NET (#7899)
* Rebase 5758 .NET work and make output-funcs pass

* Propagate changes to the other examples

* CHANGELOG

* Address PR feedback

* Add a test reproducing aws-native compilation failure

* Fix dangling type ref issue in the .NET backend for codegen

* Accept changes and unskip simple-methods-schema compile check

* Accept changes in node, python, go codegen

* SDK changes to enable a better implementation approach

* Switch approach to support functions like GetAmiIds; avoid name conflicts under tfbridge20

* Make all dotnet tests pass, mechanical fixes + accept test output

* Accept python changes

* Accept node output

* Accept docs changes

* Deepen the unit test to cover the interesting helper type

* Accept go changes and fixup tests

* Implement dep propagation through Invoke

* Fixup cyclic-types

* Accept codegen

* NOTE we now require .NET SDK 3.15 or higher
2021-10-18 18:18:15 -04:00
Paul Stack 7ef0b83c0c
Ensuring Helm based components for CodeDNS and Cert Manager are available in title Lookups" (#8244) 2021-10-18 15:24:46 +03:00
Praneet Loke 0f5b7c765b
[codegen/docs] Update API docs templates to include layout front-matter param (#8242)
* Update generation templates

* Updated templates for codegen

* Rm api-doc-resource template

* Update templates to use layout:api and no alias

* Set no_edit_this_page to true in the header and index templates used by docs gen.

* Update docs generation baseline test files with template updates

* Remove the notion of menu from resource templates (#8200)

* Update the title lookup map entry for aws-api-gateway

* Updated CSS classes for icons in API docs template

* Updated tests:

* Additional test file updates

* Fixup package names

Co-authored-by: Devon Grove <devon@pulumi.com>
Co-authored-by: Christian Nunciato <c@nunciato.org>
Co-authored-by: stack72 <public@paulstack.co.uk>
2021-10-17 22:36:31 -07:00
Anton Tayanovskyy a6e02d6dd8
Update pkg and tests pulumi/sdk refs to v3.15.0 (#8227) 2021-10-14 21:05:42 -04:00