Commit graph

6350 commits

Author SHA1 Message Date
Ian Wahbe c9b719c6c0
Revert "Emit schema.Package.Version when possible (#7938)" (#8035)
* Revert "Emit schema.Package.Version when possible (#7938)"

This reverts commit 906dce73a1.

* Remove dotnet SDK compile checks

* Add to changelog
2021-09-22 19:39:07 -07:00
Komal 212ac89366
[auto/python] - Fix a bug in printing stack. (#8032) 2021-09-22 16:49:03 -07:00
Komal aa79322b46
post-release (#8028) 2021-09-22 16:22:58 -04:00
Komal 0bcf475573
Update SDK version in pulumi/pkg (#8027) 2021-09-22 13:16:08 -07:00
Komal d482c51811
Prep for 3.13.0 release (#8026) 2021-09-22 13:09:22 -07:00
Anton Tayanovskyy 09aa2c677e
Fix master build (#8025)
* Fixing build failure

* Python manifest

* Dotnet manifest

* Node manifest

* Docs manifest

* Fix/restore test case running
2021-09-22 14:53:30 -04:00
Anton Tayanovskyy 49298fb433
Codegen testing upgrades (#7989)
* Multi-pass, in-place checks for SDK codegen tests; toward working Python checks

* Remove temp debug output

* Upgrade Node

* Update dotnet; need to follow up on version.txt quirks

* WIP

* Sounds like we can use non-github package names to ensure things are local

* Fix simple-enum-schema

* Fix dash-named-schema

* Fix nested-module

* Start building a test-running pass

* Infer skipping tests from skipping compiles

* Move tree schma tests to a proper place

* Address lint issues on Go code

* Build against local Go SDK

* Update pkg/codegen/internal/test/sdk_driver.go

Co-authored-by: Ian Wahbe <ian@wahbe.com>

* Make go tests work by copying them into the tree from go-extras

* Fix lint

* Fix bad merge

* Manifest-based file discovery

* Remove version-related TODO from dotnet codegen

* Add doc comment

* Do not overwrite go.mod if found from mixins

* Accept python codegen change

* Accept node codegen

* Ignore lint issue

* Accept docs changes

Co-authored-by: Ian Wahbe <ian@wahbe.com>
2021-09-22 13:55:20 -04:00
Pat Gavlin 134d7cb818
[apitype] Add a JSON schema for deployments. (#8002)
This schema can be used to validate the contents of a Pulumi deployment.
If a deployment validates against this schema, it should be considered
syntactically valid, though it may contain certain classes of semantic
errors (e.g. references to unknown resources in dependency lists,
dependency cycles, etc.).

This schema is not yet used for validation in practice and may not be
entirely accurate.

These changes also add this schema (and the schemas on which it depends)
to the developer documentation. jsonschema2md.go has been updated to
support multi-file schemas.
2021-09-21 21:37:06 -07:00
Evan Boyle adc0f2adeb
Use case insensitive checks when detecting duplicate type tokens (#8017) 2021-09-21 20:48:45 -07:00
Komal 3d42198991
Clean up diagnostic messages in event log (#7998)
* Don't escape characters in event log.

* Respect NO_COLOR
2021-09-21 17:22:39 -07:00
Anton Tayanovskyy c16970b488
Faster CI PR verification on Mac OS: run fewer tests, use fewer workers (#8004)
* Experiment with matrix exclude/include rules

* Check if this applies to current PR

* Try again on the right yml file

* Why is GHA trying to find self-hosted workers

* Revert all changes

* No-op change

* Try a system where PR verification uses only 1 worker

* Fix include directive so that mac job has the right variables

* Fix YAML syntax
2021-09-21 19:54:47 -04:00
Anton Tayanovskyy a9c9853e85
Cancel checks on outdated PR commits (#8016) 2021-09-21 18:17:06 -04:00
Pat Gavlin cbdecf2cd5
[testing] Add rapid generators for PropertyValues. (#8009)
And use those generators to test property value serialization and
deserialization paths.
2021-09-21 15:02:10 -07:00
Ian Wahbe 2e5fedff54
Switch from golint to revive (#8010) 2021-09-21 10:00:44 -07:00
Pat Gavlin 236ce54269
[schema] Add the Pulumi Package metaschema. (#7952)
The Pulumi Package metaschema is a JSON schema definition that describes
the format of a Pulumi Package schema. The metaschema can be used to
validate certain basic properties of a Pulumi Package schema, including
(but not limited to):

- data types (e.g. is this property a string?)
- data formats (e.g. is this string property a valid regex?)
- object shapes (e.g. is this object missing required properties?)

The schema binder has been updated to use the metaschema as its first
validation pass.

In addition to its use in the binder, the metaschema has its own page in
the developer documentation. This page is generated using a small tool,
jsonschema2md.go.
2021-09-20 12:00:42 -07:00
Ian Wahbe c338876b9f
iwahbe/7858/fix nodejs hypen imports (#7993)
* Add pascal name case

* Add test to prevent regressions

* Update CHANGELOG_PENDING.md

* Give node/tsc more memory

* Use camelcase instead of snake case

* Add clearer comments
2021-09-20 10:11:44 -07:00
Komal c7dbf4ad15
Rename PermaLink to Permalink (#7999) 2021-09-20 09:42:29 -07:00
Ian Wahbe 906dce73a1
Emit schema.Package.Version when possible (#7938)
* Emit schema.Package.Version when possible

* Update CHANGELOG_PENDING.md

* Correctly interpret python versions (I hope)

* Update PLUGIN_VERSION to the package version

* Modify tests to conform with master merge
2021-09-17 23:11:54 -07:00
Justin Van Patten df50c4492e
[sdk/python] Fix serializing output values within dicts (#7996)
Pass along the `keep_output_values` param when serializing values within dicts.
2021-09-17 20:53:29 -07:00
Justin Van Patten d812d16329
[sdk/go] Unmarshal output values in component provider (#7975)
This adds support for unmarshaling output values in the CopyTo helper used when copying inputs to an args struct
2021-09-17 19:46:06 -07:00
Ian Wahbe 7f90e12886
Validate Name, Version and Enviroment (#7896)
* Validate Name, Version and Enviroment

For the full path:
Package.Name
Package.Version
Package.Property.Default

* Update tests

* Update CHANGELOG_PENDING.md

* Add more versions to tests

* Add another "Version" field

* Even more "version" tags

* One more "version" tag added

* Update test results from codegen

* Fix py codegen tests

* Fix doc test

* Remove `version` validation

* Unformat json files

* Fail only on errors
2021-09-17 12:12:22 -07:00
Komal 452e3f76f1
Fix python lint issues (#7988) 2021-09-16 17:42:35 -07:00
Pat Gavlin 2dcf3806bd
[automation-api] Exclude tests from test_fast. (#7986)
The Automation API tests take long enough that thay don't really fit
into `test_fast`. These changes move those tests to `test_all`.
2021-09-16 17:33:33 -07:00
Justin Van Patten 4f3f366695
[sdk/python] Marshal output values (#7926)
This change adds support for marshaling outputs as output values in the Python SDK.
2021-09-15 21:49:23 -07:00
Justin Van Patten 1a4f36e97b
[sdk/go] Add RegisterInputType and register built-in types (#7928)
This change adds a `RegisterInputType` function (similar to the existing `RegisterOutputType`) that is used to register an input interface's type and its associated input type, and adds registrations for the built-in input types.

This will be used when copying inputs to an args struct for multi-lang components. When a field is typed as the input interface (e.g. `StringMapInput`) and doesn't need to be an `Output`, we can use this to lookup the non-Output type that implements the interface (e.g. `StringMap`) so it can be instantiated.

A subsequent change will update the Go SDK codegen to produce input type registrations for a provider's input types.
2021-09-15 21:12:49 -07:00
Justin Van Patten 0b9e41e8c4
[sdk/nodejs] Marshal output values (#7925)
This change adds support for marshaling outputs as output values in the Node.js SDK.
2021-09-15 18:25:26 -07:00
Ian Wahbe 054b3f9edc
Iwahbe/7881/thread replace on chages through sdks (#7967)
* Thread replaceOnChanges through the Go SDK

* Add replaceOnChanges to the .NET SDK

* Update CHANGELOG_PENDING.md

* Fix null error

* Update CHANGELOG_PENDING.md

Co-authored-by: Justin Van Patten <jvp@justinvp.com>
2021-09-15 14:29:13 -07:00
Justin Van Patten ed4b53d3ae
Add monitor feature for output values (#7870) 2021-09-15 14:16:00 -07:00
Ian Wahbe 67303e1b99
Run type checker against all languages (#7931)
We run the best static check we can on generated code, ensuring that it is valid. 

* Run type checker against all languages (not docs)

* Fix package location, add some deps for schemas

* More tests passing

* These tests finally work

* Make linter happy

* Fix tests for merge from master

* Opt out of input-collision(nodejs) test

* Get more visibility into testing nodejs

* Fix type assumption

* Specify ts-node version

* Retrofit typescript dependencies for node14

* Give each go instance it's own module

* Attempt to diagnose remote go mod init failure

* Provide root for go mod init

* Make linter happy
2021-09-15 09:49:36 -07:00
Ian Wahbe 3b7d78c126
Use json.Unmarshal instead of custom parser (#7954)
* Use json.Unmarshal instead of custom parser

* Update CHANGELOG_PENDING.md
2021-09-13 10:49:57 -07:00
Justin Van Patten 0c0684af5c
Initial support for (un)marshaling output values (#7861)
This change expands the definition of `resource.Output` in the Go SDK with additional information about the output, i.e. dependencies and secretness, and adds support in the core Go RPC code for (un)marshaling output values.

Output values are marshaled as special objects ala archives, assets, and resource refs and are unmarshaled as `resource.Output` values.

Subsequent PRs will add:
 - A monitor feature for output values, which will initially be disabled by default but available to turn on via an envvar
 - Support for (un)marshaling output values in each language SDKs
 - A way for providers to indicate support for receiving output values
 - E2E tests
 - Turn the monitor feature on by default (w/ env var to disable) (Note: the current plan is to initially scope this to only be used when marshaling inputs to a multi-language component)
2021-09-13 09:05:31 -07:00
T-Vova e696fb6c50
Implemented filebase64() support for Code Generator (#7863)
* Added filebase64 support for Golang

* Fixed function signature

* Added filebase64 support for Typescript

* Added filebase64 support for Python

* Added filebase64 support for Dotnet, fixed Sha1

* Fixed helper method list

Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 23:09:28 +01:00
Ian Wahbe 5e65204463
Fixup for 7874 (#7921)
* Fix some nits from 7874

This was a premature merge

* Fix #7940

We don't surface recursion warnings if there is no child where `replaceOnChanges` is set.
2021-09-10 14:56:56 -07:00
Anton Tayanovskyy 151fdff906
Lower the BrokenDynamicProvider regression test from integration to mock (#7951) 2021-09-10 17:25:48 -04:00
Benjamin Schiborr 77867f9ce4
feat: Improve error messages for (un)marshalling (#7936)
Currently whenever an issue occurs in `UnmarshalProperties` and
`MarshalProperties` the offending property is hidden and very difficult
to track down.

This commit changes the behavior. For `Assets` and `Archives` the error
message now includes the URI and for other properties it includes the
key of the `PropertyMap`.
2021-09-10 13:18:08 -07:00
T-Vova 87e4c92c66
Implemented sha1() support for Code Generator (#7834)
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 21:40:38 +03:00
Anton Patsev 6143d13732
Fix license badge in readme (#7941) 2021-09-10 14:51:47 +03:00
T-Vova 913173a603
Revert "Fixed "unknown property '0'" error (#7912)" (#7939)
This reverts commit 8f40edc9e0.
2021-09-09 13:36:49 -10:00
Anton Tayanovskyy 6270f925d6
Add CodegenUtilities to C# SDK in prep for 5758 (#7934) 2021-09-09 17:18:39 -04:00
T-Vova 8f40edc9e0
Fixed "unknown property '0'" error (#7912)
Co-authored-by: Pat Gavlin <pat@pulumi.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-09 22:12:19 +03:00
T-Vova 90c0fa30af
Implemented file reading support for Go code generator (#7806)
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-09 22:10:25 +03:00
Paul Stack a70cf66fae
Upgrade pulumi docker image to use NodeJS 14.x (#7904)
Fixes: #7878

also, we stopp building the pulumi actions docker container as we are
now EOL as per 31st August
2021-09-09 20:49:22 +03:00
Pat Gavlin 2296dc791b Prepare for v3.12.0 release 2021-09-08 14:41:25 -07:00
Anton Tayanovskyy 1647b2f021
Retry http downloads that intermittently fail builds (#7916)
* Retry http downloads that intermittently fail builds

* Address PR feedback
2021-09-08 17:37:57 -04:00
Anton Tayanovskyy c247f6e283
Fix 7913 Python codegen issue with double-quote inside docstrings (#7914)
* Introduce a test that showcases the invalid generated code

* Use shared printComment function

* Check for triple quote escaping

* Accept go

* Accept dotnet

* Accept nodejs

* Move codegen exampe into an existing schema

* Add CHANGELOG entry
2021-09-08 14:52:54 -04:00
Komal 1e235330d6
Don't fail if one subset fails (#7917) 2021-09-08 08:59:40 -07:00
Ian Wahbe 08b428ae96
Add replaceOnChange to schema (#7874)
* Add replaceOnChange to schema

* replaceOnChange at generate time for resources

* ReplaceOnChanges sees through optional types

* Correctly deal with map,array,object,resource type

This is responding to PR clarifications from @justinvp and @lblackstone.

* Update CHANGELOG_PENDING.md

* Detect recursively defined objects

* Display recursion warning

* Check which recursive structures fail

* Add internal logic tests for replaceOnChanges

* Add tests
2021-09-07 22:23:30 -07:00
Ian Wahbe 2f0b370a67
Fix CHANGELOG_PENDING.md spelling (#7919)
One of these is mine. Sorry
2021-09-07 16:08:38 -07:00
Anton Tayanovskyy 27b1404d9e
Fix lint (#7915)
* Fix lint

* Fix lint of pkg folder
2021-09-07 16:41:17 -04:00
Anton Tayanovskyy d7b9c01999
Fix 7862 (#7887)
* Fix 7862

* Add a unit test that reproduces the timeout

* Add a CHANGELOG entry
2021-09-07 11:30:39 -04:00