Commit graph

58 commits

Author SHA1 Message Date
Ian Wahbe 2e5fedff54
Switch from golint to revive (#8010) 2021-09-21 10:00:44 -07:00
Pat Gavlin 236ce54269
[schema] Add the Pulumi Package metaschema. (#7952)
The Pulumi Package metaschema is a JSON schema definition that describes
the format of a Pulumi Package schema. The metaschema can be used to
validate certain basic properties of a Pulumi Package schema, including
(but not limited to):

- data types (e.g. is this property a string?)
- data formats (e.g. is this string property a valid regex?)
- object shapes (e.g. is this object missing required properties?)

The schema binder has been updated to use the metaschema as its first
validation pass.

In addition to its use in the binder, the metaschema has its own page in
the developer documentation. This page is generated using a small tool,
jsonschema2md.go.
2021-09-20 12:00:42 -07:00
Ian Wahbe 7f90e12886
Validate Name, Version and Enviroment (#7896)
* Validate Name, Version and Enviroment

For the full path:
Package.Name
Package.Version
Package.Property.Default

* Update tests

* Update CHANGELOG_PENDING.md

* Add more versions to tests

* Add another "Version" field

* Even more "version" tags

* One more "version" tag added

* Update test results from codegen

* Fix py codegen tests

* Fix doc test

* Remove `version` validation

* Unformat json files

* Fail only on errors
2021-09-17 12:12:22 -07:00
Ian Wahbe 5e65204463
Fixup for 7874 (#7921)
* Fix some nits from 7874

This was a premature merge

* Fix #7940

We don't surface recursion warnings if there is no child where `replaceOnChanges` is set.
2021-09-10 14:56:56 -07:00
Ian Wahbe 08b428ae96
Add replaceOnChange to schema (#7874)
* Add replaceOnChange to schema

* replaceOnChange at generate time for resources

* ReplaceOnChanges sees through optional types

* Correctly deal with map,array,object,resource type

This is responding to PR clarifications from @justinvp and @lblackstone.

* Update CHANGELOG_PENDING.md

* Detect recursively defined objects

* Display recursion warning

* Check which recursive structures fail

* Add internal logic tests for replaceOnChanges

* Add tests
2021-09-07 22:23:30 -07:00
Pat Gavlin 76ee1b8ccf
[codegen/schema] Add a schema checker (#7865)
- Change the schema package to report semantic errors as diagnostics
  rather than Go errors
- Add a `pulumi schema check` command to the CLI for static checking of
  package schemas

The semantic checker can be extended in the future to add support for
target-specific checks.
2021-08-30 19:29:24 -07:00
Ian Wahbe bee803a970
Merge pull request #7824 from pulumi/iwahbe/7775/allow-hyphen-in-schema-path
Allow hyphen in schema path
2021-08-24 14:48:32 -07:00
Anton Tayanovskyy eef208633c
5758 for Go (#7784)
* Go support for 5758 - resurrect stale PR

* Fix listStorageAccountKeys test

* Check err so linter is satisfied

* Use all the examples

* Accept codegen results

* Regenerate with PULUMI_IGNORE_AMBIENT_PLUGINS=1

* Compile and test generated code as part of the test suite

* Add a CHANGELOG entry

* Remove temp test marker

* Shorten output type name

* Simplify code

* Add issue link

* Accept more codegen changes

* Use the suggested format for linking an issue
2021-08-23 16:46:09 -04:00
Ian Wahbe 9c02fe610f Allow hyphen in schema path
Fixes: https://github.com/pulumi/pulumi/issues/7775
2021-08-23 13:29:25 -07:00
Komal 24802142f2
[programgen] - Fix a panic in property binding (#7718) 2021-08-09 14:24:08 +03:00
Pat Gavlin d5e6d2206b
[codegen/go] Improve codegen performance. (#7546)
- Track which languages have been imported for a package. If a language
  has already been imported, do not re-run its importers.
- Track which package contexts have been loaded in the Go code
  generator, and do not reload a context that already exists.

These changes shave a profound amount of time off of codegen in
azure-native, speeding things up by a factor of 5.
2021-07-16 07:50:35 -07:00
Pat Gavlin 4d1ca1596b
[codegen/schema] YAML {un,}marshaling support. (#7509)
These changes add support for unmarshaling and marshaling package
schemas using YAML instead of JSON. Language-specific data is
canonically JSON. Users of the `*Spec` types will need to update the
types of the the their `Language` values to use the new
`schema.RawMessage` type instead of `json.RawMessage`: the former has
support for YAML while the latter does not.
2021-07-13 16:41:40 -07:00
Justin Van Patten e169844652
Fix lint error (#7481) 2021-07-09 16:05:03 -07:00
Pat Gavlin 8cf588d408
[codegen] Add more type tests. (#7475)
- Add another regression test for #7454
- Add tests for singly-nested plain collections
- Fix a bug when binding {"type": "object"}.
2021-07-09 13:33:15 -07:00
Pat Gavlin d07b325138
[codegen] Add type name generation tests. (#7470)
The inputs and expected outputs for the tests are encoded using a
schema. Each property present in the schema forms a testcase; the
expected outputs for each language are stored in each property's
`Language` field with the language name "test". Expected outputs can be
regenerated using `PULUMI_ACCEPT`.
2021-07-09 10:23:10 -07:00
Pat Gavlin 7b1d6ec1ac
Reify Input and Optional types in the schema type system. (#7059)
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.

With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.
2021-06-24 09:17:55 -07:00
Justin Van Patten 258fb00bc2
[codegen/schema] Fix intermittent failing test (#7304)
The `bad-methods-2.json` test ensures there is an error when trying to use a function as a method twice.

The schema looks like:

```
"methods": {
    "bar": "xyz:index:Foo/bar",
    "baz": "xyz:index:Foo/bar"
}
```

And the expected error is:

> function xyz:index:Foo/bar for method baz is already a method

However, when the schema is unmarshalled into a map, the keys in the map are unordered (just as JSON keys in objects are unordered), so occaisonally we'd see an error mentioning method `bar` rather than `baz`:

> function xyz:index:Foo/bar for method bar is already a method

To address this, we'll fix the portion of the code that is generating the error to walk the map in a deterministic order, and we'll also return the list of methods in the same deterministic order as well.
2021-06-16 07:14:53 -07:00
Justin Van Patten de871cec8e
Add schema support for methods (#7141) 2021-06-10 09:47:25 -07:00
James Nugent 601396c673
Include props which have underlying strings on Go SDK providers (#7230)
This commit modifies the work in #7058 to permit properties which do not
pass the test of being strings directly, but which have an underlying
type of string.

When applied to `pulumi-aws`, this results in the following diff:

```
diff --git a/sdk/go/aws/provider.go b/sdk/go/aws/provider.go
index c32ad2367..8b4c9fd0a 100644
--- a/sdk/go/aws/provider.go
+++ b/sdk/go/aws/provider.go
@@ -21,6 +21,8 @@ type Provider struct {
 	AccessKey pulumi.StringPtrOutput `pulumi:"accessKey"`
 	// The profile for API operations. If not set, the default profile created with `aws configure` will be used.
 	Profile pulumi.StringPtrOutput `pulumi:"profile"`
+	// The region where AWS operations will take place. Examples are us-east-1, us-west-2, etc.
+	Region pulumi.StringPtrOutput `pulumi:"region"`
 	// The secret key for API operations. You can retrieve this from the 'Security & Credentials' section of the AWS console.
 	SecretKey pulumi.StringPtrOutput `pulumi:"secretKey"`
 	// The path to the shared credentials file. If not set this defaults to ~/.aws/credentials.
```

The primary purpose this is desirable is to expose Region from instances
of the AWS provider.
2021-06-08 13:05:31 -07:00
James Nugent 15004e88cd
Omit non-string provider properties from code-gen (#7058)
Following pulumi/pulumi-terraform-bridge#347, properties are generated
for all provider config matching the inputs. Unfortunately this does not
work for complex values and non-string primitives generally (not only in
bridged providers) since values are JSON serialized.

While a proper solution to this is designed, it's sufficient for now to
stop generating non-string properties, which this commit does.
2021-05-21 10:37:48 -07:00
Mikhail Shilkov 1378c45849 Discriminated unions in schema and programgen 2021-05-05 10:01:22 +02:00
Pat Gavlin 6a33b4b7ee
Distinguish between inputty and plain args types. (#6811)
These changes fix a regression introduced by #6686 that caused the SDK
code generators for .NET, Python, and Typescript to omit definitions for
plain object types. This regression occurred because #6686 drew a
clearer line between types used as resource arguments and types used
as function arguments, but conflated "resource arguments" with "inputty
types". This caused the code generators to generate inputty types for
any types used as resource arguments, even those that are used for
plainly-typed properties.

Fixes #6796.
2021-04-19 16:40:39 -07:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Justin Van Patten 572c74826c
Add schema & codegen support for plain properties (#6481)
This change adds schema and codegen support for plain properties which
are emitted typed as the plain type rather than wrapped as an `Input`.
Plain properties require a prompt value and do not accept a value that
is `Output`.
2021-03-10 07:08:08 -08:00
Mikhail Shilkov 0f9273fc7b
[codegen] Add support for default values in enum properties (#6017) 2020-12-30 12:46:01 +01:00
Pat Gavlin 4b90205f3f
[codegen/python/sdk]: package/module registration (#5806)
Generate ResourcePackage and ResourceModule implementations and
registrations. A ResourcePackage is generated for any module that
includes a provider resource (which should be the root module only), and
a ResourceModule is generated for any module that includes a resource.

Note that version information is currently omitted. We should fix this
up before enabling resource reference deserialization end-to-end.
2020-11-24 12:43:23 -08:00
Pat Gavlin f527e06b21
Allow schemas to reference to provider types. (#5788)
These changes extend the type reference parser in the schema package to
accept references of the form "(package/version/schema.json)?#/provider".
These references refer to the package's provider type, which is
otherwise not referenceable, as it is not present in the "resources"
array.
2020-11-20 09:16:45 -08:00
Levi Blackstone 3586ab5d85
[codegen/python] Handle external schema refs (#5684) 2020-11-19 13:56:28 -07:00
Vivek Lakshmanan 43e48950df Adding support for go enum generation 2020-11-18 00:07:42 -08:00
Levi Blackstone 5188ec4c9f
[codegen/schema] Add support for external schema refs (#5606) 2020-10-29 17:41:12 -06:00
Komal 7039e67284
[codegen/nodejs] - Enums (#5472) 2020-10-07 18:13:10 -07:00
Komal b531038d33
[codegen/schema] - Add deprecation messages for enum values (#5521) 2020-10-07 09:32:00 -07:00
Levi Blackstone 9363f606b6
[codegen/schema] Add IsComponent to resource schema (#5395)
Add support for ComponentResources to the schema by
adding a new IsComponent field to the Resource spec.
2020-09-23 18:23:46 -06:00
Levi Blackstone 3ea59ebbb8
[codegen/schema]: Add ResourceType to support resource refs (#5325) 2020-09-22 11:09:27 -06:00
Komal a334b60193
Only allow type-matching enum values (#5417)
* Only allow type-matching enum values

* More tests and fixes
2020-09-21 12:23:04 -07:00
Komal 92a7d717a4
[codegen/schema] - Expand ObjectTypeSpec to ComplexTypeSpec (#5379)
Co-authored-by: Mikhail Shilkov <github@mikhail.io>
2020-09-16 13:47:40 -07:00
Levi Blackstone 3286aae20d
[codegen] Make schema and package available to HCL2 codegen (#5114) 2020-08-05 17:27:17 -06:00
Lee Briggs 3b0462f0e0
Merge branch 'master' into jaxxstorm/provider-info 2020-07-20 17:35:47 -07:00
Pat Gavlin d07c3698e5
[codegen/{hcl2,schema}] Abstract package loading. (#4975)
Instead of requiring a plugin host for package loading in the HCL2
binder, define a much narrower interface that exposes the ability to
fetch the schema for a package at a specific version. This interface is
defined in the schema package, which also exposes a caching loader that
is backed by provider plugins.

These changes also add some convenience methods to `*schema.Package` for
fast access to particular resources and functions.

Related to #1635.
2020-07-07 14:45:18 -07:00
Lee Briggs af2580c10a
Fix incorrect packagespec field name 2020-07-02 09:40:31 -07:00
Lee Briggs c25cbf0fc8
switch terminology to plugin download URL
After a discussion about terminology, the better phrasing is to use
pluginDownloadURL
2020-07-02 09:29:19 -07:00
Lee Briggs f310c21b85
Add providerURL to package definition 2020-07-01 19:10:25 -07:00
Levi Blackstone 137f39151b
[codegen] Add Secret field to Property schema (#4943) 2020-07-01 14:17:49 -06:00
Pat Gavlin bd1c0e6d7b
[codegen/schema] Support default union types. (#4746)
Add support for union types that indicate a default type for targets
that do not support unions, or do not support unions in certain
positions (e.g. output properties). The NodeJS backend makes use of this
in combination with a new flag, `disableUnionOutputTypes`, to avoid
generating unions in output types.

These changes also refactor the various module ->
package/module/namespace mapping methods so that these entities can be
fetched by their language name rather than their token.
2020-06-02 18:15:21 -07:00
Pat Gavlin 726d43a5bd
[codegen/nodejs/sdk] Split required {in,out}puts. (#4736)
These changes are necessary in order to move tfgen over to the schema
code generator.

The schema generator for tfgen has an annoying behavior for nested types
in which it does not separate input and output types. Worse, the shape
of the type that results from a collision of input and output types is
order-dependent: whichever of the two was observed last wins (note that
the shape is still determenistic, as the order in which types are
recorded is always the same). As a result, the NodeJS code generator
needs to know the set of required properties for the other aspect of the
type in order to generate proper code.

Contributes to
https://github.com/pulumi/pulumi-terraform-bridge/issues/179.
2020-06-01 11:50:10 -07:00
Levi Blackstone 754a4ae51d
Fix NodeJS codegen to handle recursive types (#4723)
Track the input/output types that have been encountered
and don't recurse into previously seen types.
2020-05-28 16:37:39 -06:00
Mikhail Shilkov adfa8116fb
Introduce k8s compatibility mode to C# codegen (#4544)
Introduce k8s compatibility mode to C# codegen
2020-05-19 11:41:06 +02:00
Pat Gavlin e9dd0e5e69
[codegen/schema] Path unescape type tokens in refs (#4641)
Some type tokens contain characters that are not valid in a single
URL path element (`/` in particular). These type tokens may be
path-escaped, and should be unescaped before they are interpreted.
2020-05-14 13:35:57 -07:00
Mikhail Shilkov d91f658576
Codegen changes to prepare for C# Kubernetes schema (#4531)
Codegen changes to prepare for C# Kubernetes schema
2020-05-01 12:45:47 +02:00
Mikhail Shilkov b67c774fcc
C# name resolution (#4468) 2020-04-23 00:42:31 +02:00