Commit graph

577 commits

Author SHA1 Message Date
Pat Gavlin 236ce54269
[schema] Add the Pulumi Package metaschema. (#7952)
The Pulumi Package metaschema is a JSON schema definition that describes
the format of a Pulumi Package schema. The metaschema can be used to
validate certain basic properties of a Pulumi Package schema, including
(but not limited to):

- data types (e.g. is this property a string?)
- data formats (e.g. is this string property a valid regex?)
- object shapes (e.g. is this object missing required properties?)

The schema binder has been updated to use the metaschema as its first
validation pass.

In addition to its use in the binder, the metaschema has its own page in
the developer documentation. This page is generated using a small tool,
jsonschema2md.go.
2021-09-20 12:00:42 -07:00
Ian Wahbe c338876b9f
iwahbe/7858/fix nodejs hypen imports (#7993)
* Add pascal name case

* Add test to prevent regressions

* Update CHANGELOG_PENDING.md

* Give node/tsc more memory

* Use camelcase instead of snake case

* Add clearer comments
2021-09-20 10:11:44 -07:00
Ian Wahbe 906dce73a1
Emit schema.Package.Version when possible (#7938)
* Emit schema.Package.Version when possible

* Update CHANGELOG_PENDING.md

* Correctly interpret python versions (I hope)

* Update PLUGIN_VERSION to the package version

* Modify tests to conform with master merge
2021-09-17 23:11:54 -07:00
Ian Wahbe 7f90e12886
Validate Name, Version and Enviroment (#7896)
* Validate Name, Version and Enviroment

For the full path:
Package.Name
Package.Version
Package.Property.Default

* Update tests

* Update CHANGELOG_PENDING.md

* Add more versions to tests

* Add another "Version" field

* Even more "version" tags

* One more "version" tag added

* Update test results from codegen

* Fix py codegen tests

* Fix doc test

* Remove `version` validation

* Unformat json files

* Fail only on errors
2021-09-17 12:12:22 -07:00
Ian Wahbe 67303e1b99
Run type checker against all languages (#7931)
We run the best static check we can on generated code, ensuring that it is valid. 

* Run type checker against all languages (not docs)

* Fix package location, add some deps for schemas

* More tests passing

* These tests finally work

* Make linter happy

* Fix tests for merge from master

* Opt out of input-collision(nodejs) test

* Get more visibility into testing nodejs

* Fix type assumption

* Specify ts-node version

* Retrofit typescript dependencies for node14

* Give each go instance it's own module

* Attempt to diagnose remote go mod init failure

* Provide root for go mod init

* Make linter happy
2021-09-15 09:49:36 -07:00
T-Vova e696fb6c50
Implemented filebase64() support for Code Generator (#7863)
* Added filebase64 support for Golang

* Fixed function signature

* Added filebase64 support for Typescript

* Added filebase64 support for Python

* Added filebase64 support for Dotnet, fixed Sha1

* Fixed helper method list

Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 23:09:28 +01:00
Ian Wahbe 5e65204463
Fixup for 7874 (#7921)
* Fix some nits from 7874

This was a premature merge

* Fix #7940

We don't surface recursion warnings if there is no child where `replaceOnChanges` is set.
2021-09-10 14:56:56 -07:00
T-Vova 87e4c92c66
Implemented sha1() support for Code Generator (#7834)
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 21:40:38 +03:00
T-Vova 913173a603
Revert "Fixed "unknown property '0'" error (#7912)" (#7939)
This reverts commit 8f40edc9e0.
2021-09-09 13:36:49 -10:00
T-Vova 8f40edc9e0
Fixed "unknown property '0'" error (#7912)
Co-authored-by: Pat Gavlin <pat@pulumi.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-09 22:12:19 +03:00
T-Vova 90c0fa30af
Implemented file reading support for Go code generator (#7806)
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-09 22:10:25 +03:00
Anton Tayanovskyy 1647b2f021
Retry http downloads that intermittently fail builds (#7916)
* Retry http downloads that intermittently fail builds

* Address PR feedback
2021-09-08 17:37:57 -04:00
Anton Tayanovskyy c247f6e283
Fix 7913 Python codegen issue with double-quote inside docstrings (#7914)
* Introduce a test that showcases the invalid generated code

* Use shared printComment function

* Check for triple quote escaping

* Accept go

* Accept dotnet

* Accept nodejs

* Move codegen exampe into an existing schema

* Add CHANGELOG entry
2021-09-08 14:52:54 -04:00
Ian Wahbe 08b428ae96
Add replaceOnChange to schema (#7874)
* Add replaceOnChange to schema

* replaceOnChange at generate time for resources

* ReplaceOnChanges sees through optional types

* Correctly deal with map,array,object,resource type

This is responding to PR clarifications from @justinvp and @lblackstone.

* Update CHANGELOG_PENDING.md

* Detect recursively defined objects

* Display recursion warning

* Check which recursive structures fail

* Add internal logic tests for replaceOnChanges

* Add tests
2021-09-07 22:23:30 -07:00
Evan Boyle 95aa47d3c6
[codegen/go] detect and avoid collisions in resource and type code generation (#7857) 2021-09-03 19:42:45 -07:00
Pat Gavlin 76ee1b8ccf
[codegen/schema] Add a schema checker (#7865)
- Change the schema package to report semantic errors as diagnostics
  rather than Go errors
- Add a `pulumi schema check` command to the CLI for static checking of
  package schemas

The semantic checker can be extended in the future to add support for
target-specific checks.
2021-08-30 19:29:24 -07:00
Anton Tayanovskyy 4380a63ad9
Implement 5758 {fn}_output codgen for Python (#7825)
* Implement 5758 {fn}_output codgen for Python

* Fix lint issues

* Accept codegen changes in expected examples

* Test and fix positional arg handling

* Add parameter descriptions to illustrate docstring codegen

* Generate docstrings

* Accept doc changes in expected go codegen file

* Use platform-agnostic API to drive Python tests
2021-08-30 16:52:58 -04:00
T-Vova 2c63e6fb77
Merge pull request #7792 from pulumi/t-vova/invalid-go-source-fix
Fixed invalid go source code issue
2021-08-27 19:49:30 -10:00
Ian Wahbe bee803a970
Merge pull request #7824 from pulumi/iwahbe/7775/allow-hyphen-in-schema-path
Allow hyphen in schema path
2021-08-24 14:48:32 -07:00
Ian Wahbe b30c57924a Move tests to correct location 2021-08-23 14:23:01 -07:00
Anton Tayanovskyy eef208633c
5758 for Go (#7784)
* Go support for 5758 - resurrect stale PR

* Fix listStorageAccountKeys test

* Check err so linter is satisfied

* Use all the examples

* Accept codegen results

* Regenerate with PULUMI_IGNORE_AMBIENT_PLUGINS=1

* Compile and test generated code as part of the test suite

* Add a CHANGELOG entry

* Remove temp test marker

* Shorten output type name

* Simplify code

* Add issue link

* Accept more codegen changes

* Use the suggested format for linking an issue
2021-08-23 16:46:09 -04:00
Ian Wahbe 9c02fe610f Allow hyphen in schema path
Fixes: https://github.com/pulumi/pulumi/issues/7775
2021-08-23 13:29:25 -07:00
Vova Ivanov 82c94b6bbd Fixed invalid go source code issue 2021-08-18 21:22:27 -10:00
Evan Boyle 68b4cd756d
stabilize go codegen for nested collection types (#7787) 2021-08-18 12:07:30 -07:00
Evan Boyle a0e402118f
[codegen/go] Fix generation of nested collection types and remove invalid args types (#7779) 2021-08-18 10:04:07 -07:00
T-Vova 065ae27b91
Fixed nil pointer error with BindExpressionText and typeExpr (#7766)
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-08-17 14:22:06 +03:00
Justin Van Patten 1094d66659
Fix lint error (#7757) 2021-08-12 17:20:13 -07:00
Justin Van Patten 2b97340133
[codegen/docs] Emit docs for methods (#7731) 2021-08-12 14:31:15 -07:00
Mikhail Shilkov 9c501b5bed
Fix Go generation for top-level module resources in a provider with a dash in name (#7692)
* Fix Go generation for top-level module resources in a provider with a dash

* Add tests
2021-08-10 08:58:24 +03:00
Komal 24802142f2
[programgen] - Fix a panic in property binding (#7718) 2021-08-09 14:24:08 +03:00
Pat Gavlin 594fca76d2
[codegen/go] Unify some output type generation (#7662)
These changes take a step towards simplifying and unifying the
generation of output types in the Go SDKs, especially for pointer,
array, and map outputs. This code was previously duplicated amongst the
various specialized output type generators, which led to inconsistencies
between the various implementaitons.

This is prep work for fixing #7595.
2021-08-02 13:43:24 -07:00
Anton Tayanovskyy ab7d4263b4
Fix IDE completion issues for lazy imports (#7487)
* Improve codegen import code completion python experience

* Accept codegen changes
2021-07-30 16:35:02 -04:00
Justin Van Patten 88305ba896
Update test data for docs template change (#7678) 2021-07-29 10:15:03 -07:00
Pat Gavlin 1409d88438
[codegen/*] Add join and toBase64 functions. (#5012)
Co-authored-by: Komal Ali <komal@pulumi.com>
2021-07-28 20:41:23 -07:00
David Wrede 878af7aecf
Updating doc links to align with changes in docs #4707 (#6157)
Co-authored-by: David Wrede <dave@pulumi.com>
2021-07-28 14:14:48 -07:00
Komal fddeea8a88
[codegen] Unify program codegen testing (#7657)
* ProgramGen test driver

* Refactor to avoid exporting hcl2 entities
2021-07-27 18:00:39 -07:00
Justin Van Patten 1141dd4c98
[codegen/dotnet] Emit resource methods (#7606) 2021-07-27 16:42:17 -07:00
James Nugent 4a035c44aa
Allow setting plugin name and version for Node.js (#7631)
This commit adds two new fields to the Node package info struct to
permit setting the plugin name if it differs from the package name, and
the version if it differs from the package version. This was already
supported by the loader.
2021-07-27 11:34:22 -07:00
Justin Van Patten 42a1896bf9
[codegen/python] Delete dead code around casing tables (#7647)
Coincident with the release of Pulumi 3.0, we updated the provider SDK codegen for Python to no longer use casing tables for translating Python snake_case names to Pulumi camelCase names (and vice versa). Instead, the mapping is encoded in decorators applied on class properties.

Some of the code that was used to generate and use the casing tables has persisted. This commits removes this code, as it's no longer necessary, and will improve the quality of our generated examples.
2021-07-27 08:37:49 -07:00
Pat Gavlin 91683242d0
[codegen/go] Unify input method generation. (#7643)
The code for input method generation was duplicated. These changes
remove the duplicated code.

This is prep for fixing #7595.
2021-07-26 19:23:17 -07:00
Pat Gavlin a4ee050afc
[sdk/dotnet] Fix collections in input unions. (#7601)
Collection types nested inside of Input<Union<...>> types need to abide
by the usual rules for collection types nested inside of input types.
These changes replace the use of the generic SimplifyInputUnion with a
.NET-specific simplifyInputUnion that does not remove Input types
inside of a union if those Input types wrap collection types. Retaining
these Input types allows the usual logic for handling
Input<Collection<...> types in typeString to kick in.

Fixes #7569.
2021-07-22 13:11:33 -07:00
Komal a1b7f64bdd
[docgen/go] - Fix go docgen links for pulumi types (#7558) 2021-07-20 15:57:48 -07:00
Komal 5de09409f6
[codegen/dotnet] - Dynamic provider config getters (#7549) 2021-07-20 10:22:45 -07:00
Mikhail Shilkov 6fbe6ad651
Set UseSharedCompilation to false (#7577) 2021-07-20 15:50:37 +02:00
Pat Gavlin 28b1a25629
[codegen/python] Improve program gen perf. (#7547)
- Only build casing tables once per package
- Right-size buffers in name generation

These changes lead to a significant speedup in example gen for
azure-native.
2021-07-16 15:45:00 -07:00
James Nugent e6981898eb
Update package import handling for Node.js (#7474)
* Allow non-pulumi imports for Node.js

Currently the code generator is assuming that Node.js dependencies are
following a naming scheme that is prefixed with `pulumi/`. If this is
not the case the generated import statement is incorrect.

This commit adds a map `ProviderNameToModuleName` to the language
definition that allows you to map the name of the extracted provider of
a dependency to a module name that the generator now uses to create the
import statement.

* Prepend "pulumi" to import names in Node.js SDK

It is common when writing multi-language components to have a module
name which conflicts with a provider name. This can produce unusable
code, since you cannot simultaneously import a package as `aws` and have
a namespace `aws`, for example.

This commit makes this situation much less likely, by renaming the
imported identifier for providers to `pulumiX` where it would
previously have been `x`.

This has an unfortunate side effect of making the examples in the
documentation slightly uglier, since import statements for third-party
packages are now of the form `import * as pulumiAws from "@pulumi/aws"`.
I don't see a way to discern whether code generation is for SDKs vs
examples however, and short of plumbing that through, I don't see a way
around this, so test expectations are updated accordingly.

Co-authored-by: Ben Schiborr <bschiborr@apple.com>
2021-07-16 10:27:20 -07:00
Pat Gavlin 9b23badedd
[codegen/hcl2] Improve ConversionFrom perf. (#7545)
- Lazily produce conversion failure diagnostics. This lowers the
  allocation volume and cuts down on execution time by avoiding the
  conversion of source and dest types to strings.
- Add a fast path for union conversions that checks if the source type
  is identical to any of the union's element types. Type equality
  checks are generally much faster than type conversion checks.

These changes lead to a significant speedup in codegen time in
azure-native.
2021-07-16 09:56:26 -07:00
Komal 490e9cf477
[codegen/nodejs] - Dynamic provider config getters (#7530) 2021-07-16 09:52:45 -07:00
Komal 01321187a9
[codegen/python] - Fix required vs default values in provider config (#7536) 2021-07-16 09:48:03 -07:00
Pat Gavlin d5e6d2206b
[codegen/go] Improve codegen performance. (#7546)
- Track which languages have been imported for a package. If a language
  has already been imported, do not re-run its importers.
- Track which package contexts have been loaded in the Go code
  generator, and do not reload a context that already exists.

These changes shave a profound amount of time off of codegen in
azure-native, speeding things up by a factor of 5.
2021-07-16 07:50:35 -07:00