[schema] Add the Pulumi Package metaschema. (#7952)

The Pulumi Package metaschema is a JSON schema definition that describes
the format of a Pulumi Package schema. The metaschema can be used to
validate certain basic properties of a Pulumi Package schema, including
(but not limited to):

- data types (e.g. is this property a string?)
- data formats (e.g. is this string property a valid regex?)
- object shapes (e.g. is this object missing required properties?)

The schema binder has been updated to use the metaschema as its first
validation pass.

In addition to its use in the binder, the metaschema has its own page in
the developer documentation. This page is generated using a small tool,
jsonschema2md.go.
This commit is contained in:
Pat Gavlin 2021-09-20 12:00:42 -07:00 committed by GitHub
parent c338876b9f
commit 236ce54269
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
15 changed files with 1792 additions and 18 deletions

View file

@ -12,6 +12,10 @@
- [codegen] - Packages include `Package.Version` when provided.
[#7938](https://github.com/pulumi/pulumi/pull/7938)
- [schema] The syntactical well-formedness of a package schema is now described
and checked by a JSON schema metaschema.
[#7952](https://github.com/pulumi/pulumi/pull/7952)
### Bug Fixes
- [codegen/schema] - Correct validation for Package

View file

@ -3,13 +3,13 @@ SPHINXBUILD ?= sphinx-build
SOURCEDIR = .
BUILDDIR = _build
html: Makefile graphics
html: Makefile graphics generated_markdown
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
pdf: Makefile graphics
pdf: Makefile graphics generated_markdown
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: graphics clean Makefile
.PHONY: graphics generated_markdown clean Makefile
SVG_FILES = \
providers/resource-lifecycle.svg \
@ -27,5 +27,13 @@ SVG_FILES = \
graphics: Makefile $(SVG_FILES)
MD_FILES = \
providers/metaschema.md
generated_markdown: Makefile $(MD_FILES)
providers/metaschema.md: utils/jsonschema2md.go ../pkg/codegen/schema/pulumi.json
go run ./utils/jsonschema2md.go < ../pkg/codegen/schema/pulumi.json >providers/metaschema.md
clean:
rm -rf $(BUILDDIR)/*

5
developer-docs/go.mod Normal file
View file

@ -0,0 +1,5 @@
module github.com/pulumi/pulumi/developer-docs
go 1.16
require github.com/santhosh-tekuri/jsonschema/v5 v5.0.0

2
developer-docs/go.sum Normal file
View file

@ -0,0 +1,2 @@
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0 h1:TToq11gyfNlrMFZiYujSekIsPd9AmsA2Bj/iv+s4JHE=
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0/go.mod h1:FKdcjfQW6rpZSnxxUvEA5H/cDPdvJ/SZJQLWWXWGrZ0=

View file

@ -22,6 +22,7 @@ architecture/resource-registration
:caption: Resource Providers
providers/implementers-guide
providers/metaschema
```
```{toctree}

View file

@ -135,12 +135,13 @@ TODO: write this up
## Schema
TODO: document the Pulumi schema model.
- configuration
- types
- resources
- functions
Each provider constitutes the implementation of a single Pulumi package. Each Pulumi
package has an associated schema that describes the package's
[configuration](#configuration), [resources](#resources), [functions](#functions),
and data types. The schema is primarily used to facilitate programmatic generation of
per-language SDKs for the Pulumi package, but is also used for importing resources,
program code generation, and more. Schemas may be expressed using JSON or YAML, and
must validate against the [metaschema](metaschema.md).
## Provider Lifecycle

View file

@ -0,0 +1,801 @@
# Pulumi Package Metaschema
A description of the schema for a Pulumi Package
`object`
## Properties
---
### `attribution`
Freeform text attribution of derived work, if required.
`string`
---
### `config`
The package's configuration variables.
`object`
#### Properties
---
##### `required`
A list of the names of the package's required configuration variables.
`array`
Items: `string`
---
##### `variables`
A map from variable name to propertySpec that describes a package's configuration variables.
`object`
Additional properties: [Property Definition](#property-definition)
---
---
### `description`
The description of the package. Descriptions are interpreted as Markdown.
`string`
---
### `functions`
A map from token to functionSpec that describes the set of functions defined by this package.
`object`
Property names: [Token](#token)
Additional properties: [Function Definition](#function-definition)
---
### `homepage`
The package's homepage.
`string`
---
### `keywords`
The list of keywords that are associated with the package, if any.
`array`
Items: `string`
---
### `language`
Additional language-specific data about the package.
`object`
---
### `license`
The name of the license used for the package's contents.
`string`
---
### `logoUrl`
The URL of the package's logo, if any.
`string`
---
### `meta`
Format metadata about this package.
`object`
#### Properties
---
##### `moduleFormat` (_required_)
A regex that is used by the importer to extract a module name from the module portion of a type token. Packages that use the module format "namespace1/namespace2/.../namespaceN" do not need to specify a format. The regex must define one capturing group that contains the module name, which must be formatted as "namespace1/namespace2/...namespaceN".
`string`
Format: `regex`
---
---
### `name` (_required_)
The unqualified name of the package (e.g. "aws", "azure", "gcp", "kubernetes", "random")
`string`
Pattern: `^[^0-9][-a-zA-Z0-9]*$`
---
### `pluginDownloadUrl`
The URL to use when downloading the provider plugin binary.
`string`
---
### `provider`
The provider type for this package.
[Resource Definition](#resource-definition)
---
### `repository`
The URL at which the package's sources can be found.
`string`
---
### `resources`
A map from type token to resourceSpec that describes the set of resources and components defined by this package.
`object`
Property names: [Token](#token)
Additional properties: [Resource Definition](#resource-definition)
---
### `types`
A map from type token to complexTypeSpec that describes the set of complex types (i.e. object, enum) defined by this package.
`object`
Property names: [Token](#token)
Additional properties: [Type Definition](#type-definition)
---
### `version`
The version of the package. The version must be valid semver.
`string`
Pattern: `^v?(?P<major>0|[1-9]\d*)\.(?P<minor>0|[1-9]\d*)\.(?P<patch>0|[1-9]\d*)(?:-(?P<prerelease>(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$`
---
## Alias Definition
`object`
### Properties
---
#### `name`
The name portion of the alias, if any
`string`
---
#### `project`
The project portion of the alias, if any
`string`
---
#### `type`
The type portion of the alias, if any
`string`
---
## Array Type
A reference to an array type. The "type" property must be set to "array" and the "items" property must be present. No other properties may be present.
`object`
### Properties
---
#### `items` (_required_)
The element type of the array
[Type Reference](#type-reference)
---
#### `type` (_required_)
Constant: `"array"`
---
## Enum Type Definition
Describes an enum type
`object`
### Properties
---
#### `enum` (_required_)
The list of possible values for the enum
`array`
Items: [Enum Value Definition](#enum-value-definition)
---
#### `type` (_required_)
The underlying primitive type of the enum
`string`
Enum: `"boolean"` | `"integer"` | `"number"` | `"string"`
---
## Enum Value Definition
`object`
### Properties
---
#### `deprecationMessage`
Indicates whether or not the value is deprecated.
`string`
---
#### `description`
The description of the enum value, if any. Interpreted as Markdown.
`string`
---
#### `name`
If present, overrides the name of the enum value that would usually be derived from the value.
`string`
---
#### `value` (_required_)
The enum value itself
`boolean` | `integer` | `number` | `string`
---
## Function Definition
Describes a function.
`object`
### Properties
---
#### `deprecationMessage`
Indicates whether or not the function is deprecated
`string`
---
#### `description`
The description of the function, if any. Interpreted as Markdown.
`string`
---
#### `inputs`
The bag of input values for the function, if any.
[Object Type Details](#object-type-details)
---
#### `language`
Additional language-specific data about the function.
`object`
---
#### `outputs`
The bag of output values for the function, if any.
[Object Type Details](#object-type-details)
---
## Map Type
A reference to a map type. The "type" property must be set to "object" and the "additionalProperties" property may be present. No other properties may be present.
`object`
### Properties
---
#### `additionalProperties`
The element type of the map. Defaults to "string" when omitted.
[Type Reference](#type-reference)
---
#### `type` (_required_)
Constant: `"object"`
---
## Named Type
A reference to a type in this or another document. The "$ref" property must be present. The "type" property is ignored if it is present. No other properties may be present.
`object`
### Properties
---
#### `$ref` (_required_)
The URI of the referenced type. For example, the built-in Archive, Asset, and Any
types are referenced as "pulumi.json#/Archive", "pulumi.json#/Asset", and "pulumi.json#/Any", respectively.
A type from this document is referenced as "#/types/pulumi:type:token".
A type from another document is referenced as "path#/types/pulumi:type:token", where path is of the form:
"/provider/vX.Y.Z/schema.json" or "pulumi.json" or "http[s]://example.com/provider/vX.Y.Z/schema.json"
A resource from this document is referenced as "#/resources/pulumi:type:token".
A resource from another document is referenced as "path#/resources/pulumi:type:token", where path is of the form:
"/provider/vX.Y.Z/schema.json" or "pulumi.json" or "http[s]://example.com/provider/vX.Y.Z/schema.json"
`string`
Format: `uri-reference`
---
#### `type`
ignored; present for compatibility with existing schemas
`string`
---
## Object Type Definition
`object`
All of:
- [Object Type Details](#object-type-details)
### Properties
---
#### `type`
Constant: `"object"`
---
## Object Type Details
Describes an object type
`object`
### Properties
---
#### `properties`
A map from property name to propertySpec that describes the object's properties.
`object`
Additional properties: [Property Definition](#property-definition)
---
#### `required`
A list of the names of an object type's required properties. These properties must be set for inputs and will always be set for outputs.
`array`
Items: `string`
---
## Primitive Type
A reference to a primitive type. A primitive type must have only the "type" property set.
`object`
### Properties
---
#### `type` (_required_)
The primitive type, if any
`string`
Enum: `"boolean"` | `"integer"` | `"number"` | `"string"`
---
## Property Definition
Describes an object or resource property
`object`
All of:
- [Type Reference](#type-reference)
### Properties
---
#### `const`
The constant value for the property, if any. The type of the value must be assignable to the type of the property.
`boolean` | `number` | `string`
---
#### `default`
The default value for the property, if any. The type of the value must be assignable to the type of the property.
`boolean` | `number` | `string`
---
#### `defaultInfo`
Additional information about the property's default value, if any.
`object`
##### Properties
---
###### `environment` (_required_)
A set of environment variables to probe for a default value.
`array`
Items: `string`
---
###### `language`
Additional language-specific data about the default value.
`object`
---
---
#### `deprecationMessage`
Indicates whether or not the property is deprecated
`string`
---
#### `description`
The description of the property, if any. Interpreted as Markdown.
`string`
---
#### `language`
Additional language-specific data about the property.
`object`
---
#### `replaceOnChanges`
Specifies whether a change to the property causes its containing resource to be replaced instead of updated (default false).
`boolean`
---
#### `secret`
Specifies whether the property is secret (default false).
`boolean`
---
## Resource Definition
Describes a resource or component.
`object`
All of:
- [Object Type Details](#object-type-details)
### Properties
---
#### `aliases`
The list of aliases for the resource.
`array`
Items: [Alias Definition](#alias-definition)
---
#### `deprecationMessage`
Indicates whether or not the resource is deprecated
`string`
---
#### `description`
The description of the resource, if any. Interpreted as Markdown.
`string`
---
#### `inputProperties`
A map from property name to propertySpec that describes the resource's input properties.
`object`
Additional properties: [Property Definition](#property-definition)
---
#### `isComponent`
Indicates whether or not the resource is a component.
`boolean`
---
#### `methods`
A map from method name to function token that describes the resource's method set.
`object`
Additional properties: `string`
---
#### `requiredInputs`
A list of the names of the resource's required input properties.
`array`
Items: `string`
---
#### `stateInputs`
An optional objectTypeSpec that describes additional inputs that mau be necessary to get an existing resource. If this is unset, only an ID is necessary.
[Object Type Details](#object-type-details)
---
## Token
`string`
Pattern: `^[^0-9][-a-zA-Z0-9]*:([^0-9][a-zA-Z0-9._/]*)?:[^0-9][a-zA-Z0-9._/]*$`
## Type Definition
Describes an object or enum type.
`object`
One of:
### Properties
---
#### `description`
The description of the type, if any. Interpreted as Markdown.
`string`
---
#### `language`
Additional language-specific data about the type.
`object`
---
## Type Reference
A reference to a type. The particular kind of type referenced is determined based on the contents of the "type" property and the presence or absence of the "additionalProperties", "items", "oneOf", and "$ref" properties.
`object`
One of:
### Properties
---
#### `plain`
Indicates that when used as an input, this type does not accept eventual values.
`boolean`
---
## Union Type
A reference to a union type. The "oneOf" property must be present. The union may additional specify an underlying primitive type via the "type" property and a discriminator via the "discriminator" property. No other properties may be present.
`object`
### Properties
---
#### `discriminator`
Informs the consumer of an alternative schema based on the value associated with it
`object`
##### Properties
---
###### `mapping`
an optional object to hold mappings between payload values and schema names or references
`object`
Additional properties: `string`
---
###### `propertyName` (_required_)
PropertyName is the name of the property in the payload that will hold the discriminator value
`string`
---
---
#### `oneOf` (_required_)
If present, indicates that values of the type may be one of any of the listed types
`array`
Items: [Type Reference](#type-reference)
---
#### `type`
The underlying primitive type of the union, if any
`string`
Enum: `"boolean"` | `"integer"` | `"number"` | `"string"`
---

View file

@ -0,0 +1,384 @@
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"os"
"regexp"
"sort"
"strings"
"github.com/santhosh-tekuri/jsonschema/v5"
)
var punctuationRegexp = regexp.MustCompile(`[^\w\- ]`)
// ref: https://github.com/gjtorikian/html-pipeline/blob/main/lib/html/pipeline/toc_filter.rb
func gfmHeaderAnchor(header string) string {
header = strings.ToLower(header)
header = punctuationRegexp.ReplaceAllString(header, "")
return "#" + strings.ReplaceAll(header, " ", "-")
}
func fprintf(w io.Writer, f string, args ...interface{}) {
_, err := fmt.Fprintf(w, f, args...)
if err != nil {
log.Fatal(err)
}
}
func toJSON(v interface{}) string {
bytes, err := json.Marshal(v)
if err != nil {
log.Fatal(err)
}
return string(bytes)
}
func schemaItems(schema *jsonschema.Schema) *jsonschema.Schema {
if schema.Items2020 != nil {
return schema.Items2020
}
if items, ok := schema.Items.(*jsonschema.Schema); ok {
return items
}
return nil
}
type converter struct {
w io.Writer
rootLocation string
defs map[string]*jsonschema.Schema
}
func (c *converter) printf(f string, args ...interface{}) {
fprintf(c.w, f, args...)
}
func (c *converter) inlineDef(schema *jsonschema.Schema) bool {
return schema.Description == "" &&
schema.Title == "" &&
schema.Format == "" &&
len(schema.Properties) == 0 &&
len(schema.AllOf) == 0 &&
len(schema.AnyOf) == 0 &&
len(schema.OneOf) == 0 &&
schema.If == nil &&
schema.PropertyNames == nil &&
len(schema.PatternProperties) == 0 &&
schema.Items == nil &&
schema.AdditionalItems == nil &&
len(schema.PrefixItems) == 0 &&
schema.Items2020 == nil &&
schema.Contains == nil &&
schema.Pattern == nil
}
func (c *converter) recordDef(schema *jsonschema.Schema) {
if schema != nil && strings.HasPrefix(schema.Location, c.rootLocation) {
if _, has := c.defs[schema.Location]; !has {
c.defs[schema.Location] = schema
c.collectDefs(schema)
}
}
}
func (c *converter) recordDefs(schemas []*jsonschema.Schema) {
for _, schema := range schemas {
c.recordDef(schema)
}
}
func (c *converter) collectDefs(schema *jsonschema.Schema) {
c.recordDef(schema.Ref)
c.recordDef(schema.RecursiveRef)
c.recordDef(schema.DynamicRef)
c.recordDef(schema.Not)
c.recordDefs(schema.AllOf)
c.recordDefs(schema.AnyOf)
c.recordDefs(schema.OneOf)
c.recordDef(schema.If)
c.recordDef(schema.Then)
c.recordDef(schema.Else)
for _, schema := range schema.Properties {
c.collectDefs(schema)
}
c.recordDef(schema.PropertyNames)
for _, schema := range schema.PatternProperties {
c.collectDefs(schema)
}
if child, ok := schema.AdditionalProperties.(*jsonschema.Schema); ok {
c.recordDef(child)
}
for _, dep := range schema.Dependencies {
if schema, ok := dep.(*jsonschema.Schema); ok {
c.recordDef(schema)
}
}
for _, schema := range schema.DependentSchemas {
c.recordDef(schema)
}
c.recordDef(schema.UnevaluatedProperties)
switch items := schema.Items.(type) {
case *jsonschema.Schema:
c.recordDef(items)
case []*jsonschema.Schema:
c.recordDefs(items)
}
if child, ok := schema.AdditionalItems.(*jsonschema.Schema); ok {
c.recordDef(child)
}
c.recordDefs(schema.PrefixItems)
c.recordDef(schema.Items2020)
c.recordDef(schema.Contains)
c.recordDef(schema.UnevaluatedItems)
}
func (c *converter) schemaTitle(schema *jsonschema.Schema) string {
if schema.Title != "" {
return schema.Title
}
return "`" + schema.Location + "`"
}
func (c *converter) refLink(ref *jsonschema.Schema) string {
dest := ref.Location
if strings.HasPrefix(ref.Location, c.rootLocation) {
dest = gfmHeaderAnchor(c.schemaTitle(ref))
}
return fmt.Sprintf("[%v](%v)", c.schemaTitle(ref), dest)
}
func (c *converter) ref(ref *jsonschema.Schema) string {
if !c.inlineDef(ref) {
return c.refLink(ref)
}
if ref.Ref != nil {
return c.ref(ref.Ref)
}
if len(ref.Constant) != 0 {
return c.schemaConstant(ref)
}
if len(ref.Enum) != 0 {
return c.schemaEnum(ref)
}
return c.schemaTypes(ref)
}
func (c *converter) schemaTypes(schema *jsonschema.Schema) string {
types := schema.Types
if len(types) == 1 {
return fmt.Sprintf("`%v`", types[0])
}
var sb strings.Builder
for i, t := range types {
if i != 0 {
fprintf(&sb, " | ")
}
fprintf(&sb, "`%v`", t)
}
return sb.String()
}
func (c *converter) convertSchemaTypes(schema *jsonschema.Schema) {
types := schema.Types
switch len(types) {
case 0:
// Nothing to do
case 1:
c.printf("\n%v\n", c.schemaTypes(schema))
default:
c.printf("\n%v\n", c.schemaTypes(schema))
}
}
func (c *converter) convertSchemaStringValidators(schema *jsonschema.Schema) {
if schema.Format != "" {
c.printf("\nFormat: `%v`\n", schema.Format)
}
if schema.Pattern != nil {
c.printf("\nPattern: `%v`\n", schema.Pattern)
}
}
func (c *converter) convertSchemaRef(schema *jsonschema.Schema) {
if schema.Ref != nil {
c.printf("\n%v\n", c.refLink(schema.Ref))
}
}
func (c *converter) schemaConstant(schema *jsonschema.Schema) string {
return fmt.Sprintf("`%s`", toJSON(schema.Constant[0]))
}
func (c *converter) convertSchemaConstant(schema *jsonschema.Schema) {
if len(schema.Constant) != 0 {
c.printf("\nConstant: %v\n", c.schemaConstant(schema))
}
}
func (c *converter) schemaEnum(schema *jsonschema.Schema) string {
var sb strings.Builder
for i, v := range schema.Enum {
if i != 0 {
sb.WriteString(" | ")
}
fprintf(&sb, "`%s`", toJSON(v))
}
return sb.String()
}
func (c *converter) convertSchemaEnum(schema *jsonschema.Schema) {
if len(schema.Enum) != 0 {
c.printf("\nEnum: %v\n", c.schemaEnum(schema))
}
}
func (c *converter) convertSchemaLogic(schema *jsonschema.Schema) {
if len(schema.AllOf) != 0 {
c.printf("\nAll of:\n")
for _, ref := range schema.AllOf {
c.printf("- %v\n", c.ref(ref))
}
}
if len(schema.AnyOf) != 0 {
c.printf("\nAny of:\n")
for _, ref := range schema.AllOf {
c.printf("- %v\n", c.ref(ref))
}
}
if len(schema.OneOf) != 0 {
c.printf("\nOne of:\n")
for _, ref := range schema.AllOf {
c.printf("- %v\n", c.ref(ref))
}
}
if schema.If != nil {
c.printf("\nIf %v", c.ref(schema.If))
if schema.Then != nil {
c.printf(", then %v", c.ref(schema.Then))
}
if schema.Else != nil {
c.printf(", else %v", c.ref(schema.Else))
}
c.printf("\n")
}
}
func (c *converter) convertSchemaObject(schema *jsonschema.Schema, level int) {
if schema.PropertyNames != nil {
c.printf("\nProperty names: %v\n", c.ref(schema.PropertyNames))
}
if additionalProperties, ok := schema.AdditionalProperties.(*jsonschema.Schema); ok {
c.printf("\nAdditional properties: %v\n", c.ref(additionalProperties))
}
required := map[string]bool{}
for _, name := range schema.Required {
required[name] = true
}
properties := make([]string, 0, len(schema.Properties))
for name, schema := range schema.Properties {
if schema.Always != nil && !*schema.Always {
continue
}
properties = append(properties, name)
}
sort.Strings(properties)
if len(properties) != 0 {
c.printf("\n%v Properties\n", strings.Repeat("#", level+1))
c.printf("\n---\n")
for _, name := range properties {
c.printf("\n%s `%s`", strings.Repeat("#", level+2), name)
if required[name] {
c.printf(" (_required_)")
}
c.printf("\n")
c.convertSchema(schema.Properties[name], level+2)
c.printf("\n---\n")
}
}
}
func (c *converter) convertSchemaArray(schema *jsonschema.Schema) {
if items := schemaItems(schema); items != nil {
c.printf("\nItems: %v\n", c.ref(items))
}
}
func (c *converter) convertSchema(schema *jsonschema.Schema, level int) {
if schema.Description != "" {
c.printf("\n%s\n", schema.Description)
}
c.convertSchemaTypes(schema)
c.convertSchemaConstant(schema)
c.convertSchemaEnum(schema)
c.convertSchemaStringValidators(schema)
c.convertSchemaRef(schema)
c.convertSchemaLogic(schema)
c.convertSchemaArray(schema)
c.convertSchemaObject(schema, level)
}
func (c *converter) convertRootSchema(schema *jsonschema.Schema) {
c.collectDefs(schema)
c.printf("# %s\n", c.schemaTitle(schema))
c.convertSchema(schema, 1)
defs := make([]*jsonschema.Schema, 0, len(c.defs))
for _, def := range c.defs {
defs = append(defs, def)
}
sort.Slice(defs, func(i, j int) bool {
return c.schemaTitle(defs[i]) < c.schemaTitle(defs[j])
})
for _, def := range defs {
if !c.inlineDef(def) {
c.printf("\n## %s\n", c.schemaTitle(def))
c.convertSchema(def, 2)
}
}
}
func main() {
schemaBytes, err := ioutil.ReadAll(os.Stdin)
if err != nil {
log.Fatal(err)
}
compiler := jsonschema.NewCompiler()
compiler.ExtractAnnotations = true
compiler.LoadURL = func(s string) (io.ReadCloser, error) {
if s == "blob://stdin" {
return ioutil.NopCloser(bytes.NewReader(schemaBytes)), nil
}
return jsonschema.LoadURL(s)
}
schema, err := compiler.Compile("blob://stdin")
if err != nil {
log.Fatal(err)
}
converter := converter{
w: os.Stdout,
rootLocation: schema.Location,
defs: map[string]*jsonschema.Schema{},
}
converter.convertRootSchema(schema)
}

View file

@ -1,6 +1,6 @@
{
"name": "py_tests",
"version": "0.0.1",
"name": "py_tests",
"version": "0.0.1",
"functions": {
"madeup-package:codegentest:funcWithConstInput": {
"description": "Codegen demo with const inputs",

View file

@ -197,13 +197,6 @@
}
]
},
"plant:tree/v1:VarietiesSupported": {
"type": "array",
"items": {
"$ref": "#/types/plant:tree/v1:RubberTreeVariety"
},
"description": "The rubber tree varieties supported"
},
"plant:tree/v1:Farm": {
"type": "string",
"enum": [

View file

@ -0,0 +1,511 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/pulumi/pulumi/blob/master/pkg/codegen/schema.json",
"title": "Pulumi Package Metaschema",
"description": "A description of the schema for a Pulumi Package",
"type": "object",
"properties": {
"name": {
"description": "The unqualified name of the package (e.g. \"aws\", \"azure\", \"gcp\", \"kubernetes\", \"random\")",
"type": "string",
"pattern": "^[a-zA-Z][-a-zA-Z0-9_]*$"
},
"version": {
"description": "The version of the package. The version must be valid semver.",
"type": "string",
"pattern": "^v?(?P<major>0|[1-9]\\d*)\\.(?P<minor>0|[1-9]\\d*)\\.(?P<patch>0|[1-9]\\d*)(?:-(?P<prerelease>(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\\.(?:0|[1-9]\\d*|\\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\\+(?P<buildmetadata>[0-9a-zA-Z-]+(?:\\.[0-9a-zA-Z-]+)*))?$"
},
"description": {
"description": "The description of the package. Descriptions are interpreted as Markdown.",
"type": "string"
},
"keywords": {
"description": "The list of keywords that are associated with the package, if any.",
"type": "array",
"items": {
"type": "string"
}
},
"homepage": {
"description": "The package's homepage.",
"type": "string"
},
"license": {
"description": "The name of the license used for the package's contents.",
"type": "string"
},
"attribution": {
"description": "Freeform text attribution of derived work, if required.",
"type": "string"
},
"repository": {
"description": "The URL at which the package's sources can be found.",
"type": "string"
},
"logoUrl": {
"description": "The URL of the package's logo, if any.",
"type": "string"
},
"pluginDownloadUrl": {
"description": "The URL to use when downloading the provider plugin binary.",
"type": "string"
},
"meta": {
"description": "Format metadata about this package.",
"type": "object",
"properties": {
"moduleFormat": {
"description": "A regex that is used by the importer to extract a module name from the module portion of a type token. Packages that use the module format \"namespace1/namespace2/.../namespaceN\" do not need to specify a format. The regex must define one capturing group that contains the module name, which must be formatted as \"namespace1/namespace2/...namespaceN\".",
"type": "string",
"format": "regex"
}
},
"required": ["moduleFormat"]
},
"config": {
"description": "The package's configuration variables.",
"type": "object",
"properties": {
"variables": {
"description": "A map from variable name to propertySpec that describes a package's configuration variables.",
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/propertySpec"
}
},
"required": {
"description": "A list of the names of the package's required configuration variables.",
"type": "array",
"items": {
"type": "string"
}
}
}
},
"types": {
"description": "A map from type token to complexTypeSpec that describes the set of complex types (i.e. object, enum) defined by this package.",
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/complexTypeSpec"
},
"propertyNames": {
"$ref": "#/$defs/token"
}
},
"provider": {
"description": "The provider type for this package.",
"$ref": "#/$defs/resourceSpec"
},
"resources": {
"description": "A map from type token to resourceSpec that describes the set of resources and components defined by this package.",
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/resourceSpec"
},
"propertyNames": {
"$ref": "#/$defs/token"
}
},
"functions": {
"description": "A map from token to functionSpec that describes the set of functions defined by this package.",
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/functionSpec"
},
"propertyNames": {
"$ref": "#/$defs/token"
}
},
"language": {
"description": "Additional language-specific data about the package.",
"type": "object"
}
},
"additionalProperties": false,
"required": [
"name"
],
"$defs": {
"token": {
"title": "Token",
"type": "string",
"$comment": "In the regex below, the 'module' portion of the token is optional. However, a missing module component creates a '::', which breaks URNs ('::' is the URN delimiter). We have many test schemas that use an empty module component successfully, as they never create URNs; while these are _probably_ the only places that need updating, it might be possible that there are module-less type tokens in the wild elsewhere and we may need to remain compatible with those tokens.",
"pattern": "^[a-zA-Z][-a-zA-Z0-9_]*:([^0-9][a-zA-Z0-9._/]*)?:[^0-9][a-zA-Z0-9._/]*$"
},
"typeSpec": {
"title": "Type Reference",
"description": "A reference to a type. The particular kind of type referenced is determined based on the contents of the \"type\" property and the presence or absence of the \"additionalProperties\", \"items\", \"oneOf\", and \"$ref\" properties.",
"type": "object",
"properties": {
"plain": {
"description": "Indicates that when used as an input, this type does not accept eventual values.",
"type": "boolean"
}
},
"oneOf": [
{
"title": "Primitive Type",
"description": "A reference to a primitive type. A primitive type must have only the \"type\" property set.",
"type": "object",
"properties": {
"type": {
"description": "The primitive type, if any",
"type": "string",
"enum": ["boolean", "integer", "number", "string"]
},
"additionalProperties": false,
"items": false,
"oneOf": false,
"$ref": false
},
"required": ["type"]
},
{
"title": "Array Type",
"description": "A reference to an array type. The \"type\" property must be set to \"array\" and the \"items\" property must be present. No other properties may be present.",
"type": "object",
"$comment": "An array type must have the \"type\" property set.",
"properties": {
"type": {
"const": "array"
},
"items": {
"description": "The element type of the array",
"$ref": "#/$defs/typeSpec"
},
"additionalProperties": false,
"oneOf": false,
"$ref": false
},
"required": ["type", "items"]
},
{
"title": "Map Type",
"description": "A reference to a map type. The \"type\" property must be set to \"object\" and the \"additionalProperties\" property may be present. No other properties may be present.",
"type": "object",
"properties": {
"type": {
"const": "object"
},
"additionalProperties": {
"description": "The element type of the map. Defaults to \"string\" when omitted.",
"$ref": "#/$defs/typeSpec"
},
"items": false,
"oneOf": false,
"$ref": false
},
"required": ["type"]
},
{
"title": "Named Type",
"description": "A reference to a type in this or another document. The \"$ref\" property must be present. The \"type\" property is ignored if it is present. No other properties may be present.",
"type": "object",
"properties": {
"type": {
"description": "ignored; present for compatibility with existing schemas",
"type": "string"
},
"$ref": {
"description": "The URI of the referenced type. For example, the built-in Archive, Asset, and Any\ntypes are referenced as \"pulumi.json#/Archive\", \"pulumi.json#/Asset\", and \"pulumi.json#/Any\", respectively.\nA type from this document is referenced as \"#/types/pulumi:type:token\".\nA type from another document is referenced as \"path#/types/pulumi:type:token\", where path is of the form:\n \"/provider/vX.Y.Z/schema.json\" or \"pulumi.json\" or \"http[s]://example.com/provider/vX.Y.Z/schema.json\"\nA resource from this document is referenced as \"#/resources/pulumi:type:token\".\nA resource from another document is referenced as \"path#/resources/pulumi:type:token\", where path is of the form:\n \"/provider/vX.Y.Z/schema.json\" or \"pulumi.json\" or \"http[s]://example.com/provider/vX.Y.Z/schema.json\"",
"type": "string",
"format": "uri-reference"
},
"additionalProperties": false,
"items": false,
"oneOf": false
},
"required": ["$ref"]
},
{
"title": "Union Type",
"description": "A reference to a union type. The \"oneOf\" property must be present. The union may additional specify an underlying primitive type via the \"type\" property and a discriminator via the \"discriminator\" property. No other properties may be present.",
"type": "object",
"properties": {
"type": {
"description": "The underlying primitive type of the union, if any",
"type": "string",
"enum": ["boolean", "integer", "number", "string"]
},
"oneOf": {
"description": "If present, indicates that values of the type may be one of any of the listed types",
"type": "array",
"items": {
"$ref": "#/$defs/typeSpec"
},
"minItems": 2
},
"discriminator": {
"description": "Informs the consumer of an alternative schema based on the value associated with it",
"type": "object",
"properties": {
"propertyName": {
"description": "PropertyName is the name of the property in the payload that will hold the discriminator value",
"type": "string",
"minLength": 1
},
"mapping": {
"description": "an optional object to hold mappings between payload values and schema names or references",
"type": "object",
"additionalProperties": {
"type": "string"
}
}
},
"required": [
"propertyName"
]
},
"additionalProperties": false,
"items": false,
"$ref": false
},
"required": ["oneOf"]
}
]
},
"propertySpec": {
"title": "Property Definition",
"description": "Describes an object or resource property",
"type": "object",
"allOf": [
{ "$ref": "#/$defs/typeSpec" }
],
"properties": {
"description": {
"description": "The description of the property, if any. Interpreted as Markdown.",
"type": "string"
},
"const": {
"description": "The constant value for the property, if any. The type of the value must be assignable to the type of the property.",
"type": ["boolean", "number", "string"]
},
"default": {
"description": "The default value for the property, if any. The type of the value must be assignable to the type of the property.",
"type": ["boolean", "number", "string"]
},
"defaultInfo": {
"description": "Additional information about the property's default value, if any.",
"type": "object",
"properties": {
"environment": {
"description": "A set of environment variables to probe for a default value.",
"type": "array",
"items": {
"type": "string"
}
},
"language": {
"description": "Additional language-specific data about the default value.",
"type": "object"
}
},
"required": ["environment"]
},
"deprecationMessage": {
"description": "Indicates whether or not the property is deprecated",
"type": "string"
},
"language": {
"description": "Additional language-specific data about the property.",
"type": "object"
},
"secret": {
"description": "Specifies whether the property is secret (default false).",
"type": "boolean"
},
"replaceOnChanges": {
"description": "Specifies whether a change to the property causes its containing resource to be replaced instead of updated (default false).",
"type": "boolean"
}
}
},
"complexTypeSpec": {
"title": "Type Definition",
"description": "Describes an object or enum type.",
"type": "object",
"properties": {
"description": {
"description": "The description of the type, if any. Interpreted as Markdown.",
"type": "string"
},
"language": {
"description": "Additional language-specific data about the type.",
"type": "object"
}
},
"oneOf": [
{
"title": "Object Type Definition",
"type": "object",
"allOf": [
{ "$ref": "#/$defs/objectTypeSpec" }
],
"properties": {
"type": {
"const": "object"
}
}
},
{ "$ref": "#/$defs/enumTypeSpec" }
]
},
"objectTypeSpec": {
"title": "Object Type Details",
"description": "Describes an object type",
"type": "object",
"properties": {
"properties": {
"description": "A map from property name to propertySpec that describes the object's properties.",
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/propertySpec"
}
},
"required": {
"description": "A list of the names of an object type's required properties. These properties must be set for inputs and will always be set for outputs.",
"type": "array",
"items": {
"type": "string"
}
}
}
},
"enumTypeSpec": {
"title": "Enum Type Definition",
"description": "Describes an enum type",
"type": "object",
"properties" :{
"type": {
"description": "The underlying primitive type of the enum",
"type": "string",
"enum": ["boolean", "integer", "number", "string"]
},
"enum": {
"description": "The list of possible values for the enum",
"type": "array",
"items": {
"title": "Enum Value Definition",
"type": "object",
"properties": {
"name": {
"description": "If present, overrides the name of the enum value that would usually be derived from the value.",
"type": "string"
},
"description": {
"description": "The description of the enum value, if any. Interpreted as Markdown.",
"type": "string"
},
"value": {
"description": "The enum value itself",
"type": ["boolean", "integer", "number", "string"]
},
"deprecationMessage": {
"description": "Indicates whether or not the value is deprecated.",
"type": "string"
}
},
"required": ["value"]
}
}
},
"required": ["type", "enum"]
},
"resourceSpec": {
"title": "Resource Definition",
"description": "Describes a resource or component.",
"type": "object",
"allOf": [
{ "$ref": "#/$defs/objectTypeSpec" }
],
"properties": {
"description": {
"description": "The description of the resource, if any. Interpreted as Markdown.",
"type": "string"
},
"inputProperties": {
"description": "A map from property name to propertySpec that describes the resource's input properties.",
"type": "object",
"additionalProperties": {
"$ref": "#/$defs/propertySpec"
}
},
"requiredInputs": {
"description": "A list of the names of the resource's required input properties.",
"type": "array",
"items": {
"type": "string"
}
},
"stateInputs": {
"description": "An optional objectTypeSpec that describes additional inputs that mau be necessary to get an existing resource. If this is unset, only an ID is necessary.",
"$ref": "#/$defs/objectTypeSpec"
},
"aliases": {
"description": "The list of aliases for the resource.",
"type": "array",
"items": {
"title": "Alias Definition",
"type": "object",
"properties": {
"name": {
"description": "The name portion of the alias, if any",
"type": "string"
},
"project": {
"description": "The project portion of the alias, if any",
"type": "string"
},
"type": {
"description": "The type portion of the alias, if any",
"type": "string"
}
}
}
},
"deprecationMessage": {
"description": "Indicates whether or not the resource is deprecated",
"type": "string"
},
"isComponent": {
"description": "Indicates whether or not the resource is a component.",
"type": "boolean"
},
"methods": {
"description": "A map from method name to function token that describes the resource's method set.",
"type": "object",
"additionalProperties": {
"type": "string"
}
}
}
},
"functionSpec": {
"title": "Function Definition",
"description": "Describes a function.",
"type": "object",
"properties": {
"description": {
"description": "The description of the function, if any. Interpreted as Markdown.",
"type": "string"
},
"inputs": {
"description": "The bag of input values for the function, if any.",
"$ref": "#/$defs/objectTypeSpec"
},
"outputs": {
"description": "The bag of output values for the function, if any.",
"$ref": "#/$defs/objectTypeSpec"
},
"deprecationMessage": {
"description": "Indicates whether or not the function is deprecated",
"type": "string"
},
"language": {
"description": "Additional language-specific data about the function.",
"type": "object"
}
}
}
}
}

View file

@ -16,8 +16,10 @@ package schema
import (
"bytes"
_ "embed" //nolint: golint
"encoding/json"
"fmt"
"io"
"math"
"net/url"
"os"
@ -29,6 +31,7 @@ import (
"github.com/blang/semver"
"github.com/hashicorp/hcl/v2"
"github.com/pkg/errors"
"github.com/santhosh-tekuri/jsonschema/v5"
"gopkg.in/yaml.v3"
"github.com/pulumi/pulumi/sdk/v3/go/common/resource/plugin"
@ -1270,6 +1273,22 @@ func (m *RawMessage) UnmarshalYAML(node *yaml.Node) error {
return nil
}
//go:embed pulumi.json
var metaSchema []byte
var MetaSchema *jsonschema.Schema
func init() {
compiler := jsonschema.NewCompiler()
compiler.LoadURL = func(u string) (io.ReadCloser, error) {
if u == "blob://pulumi.json" {
return io.NopCloser(bytes.NewReader(metaSchema)), nil
}
return jsonschema.LoadURL(u)
}
MetaSchema = compiler.MustCompile("blob://pulumi.json")
}
// TypeSpec is the serializable form of a reference to a type.
type TypeSpec struct {
// Type is the primitive or composite type, if any. May be "bool", "integer", "number", "string", "array", or
@ -1500,6 +1519,39 @@ func errorf(path, message string, args ...interface{}) *hcl.Diagnostic {
}
}
func validateSpec(spec PackageSpec) (hcl.Diagnostics, error) {
bytes, err := json.Marshal(spec)
if err != nil {
return nil, err
}
var raw interface{}
if err = json.Unmarshal(bytes, &raw); err != nil {
return nil, err
}
if err = MetaSchema.Validate(raw); err == nil {
return nil, nil
}
validationError, ok := err.(*jsonschema.ValidationError)
if !ok {
return nil, err
}
var diags hcl.Diagnostics
var appendError func(err *jsonschema.ValidationError)
appendError = func(err *jsonschema.ValidationError) {
if err.InstanceLocation != "" && err.Message != "" {
diags = diags.Append(errorf("#"+err.InstanceLocation, "%v", err.Message))
}
for _, err := range err.Causes {
appendError(err)
}
}
appendError(validationError)
return diags, nil
}
// bindSpec converts a serializable PackageSpec into a Package. This function includes a loader parameter which
// works as a singleton -- if it is nil, a new loader is instantiated, else the provided loader is used. This avoids
// breaking downstream consumers of ImportSpec while allowing us to extend schema support to external packages.
@ -1520,6 +1572,13 @@ func errorf(path, message string, args ...interface{}) *hcl.Diagnostic {
func bindSpec(spec PackageSpec, languages map[string]Language, loader Loader) (*Package, hcl.Diagnostics, error) {
var diags hcl.Diagnostics
// Validate the package against the metaschema.
validationDiags, err := validateSpec(spec)
if err != nil {
return nil, nil, fmt.Errorf("validating spec: %w", err)
}
diags = diags.Extend(validationDiags)
// Validate that there is a name
if spec.Name == "" {
diags = diags.Append(errorf("#/name", "no name provided"))

View file

@ -36,6 +36,7 @@ require (
github.com/pkg/errors v0.9.1
github.com/pulumi/pulumi/sdk/v3 v3.3.1
github.com/rjeczalik/notify v0.9.2
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0
github.com/sergi/go-diff v1.1.0
github.com/shirou/gopsutil v3.21.7+incompatible
github.com/shurcooL/httpfs v0.0.0-20190707220628-8d4bc4ba7749 // indirect

View file

@ -585,6 +585,8 @@ github.com/ryanuber/go-glob v1.0.0 h1:iQh3xXAumdQ+4Ufa5b25cRpC5TYKlno6hsv6Cb3pkB
github.com/ryanuber/go-glob v1.0.0/go.mod h1:807d1WSdnB0XRJzKNil9Om6lcp/3a0v4qIHxIXzX/Yc=
github.com/sabhiram/go-gitignore v0.0.0-20180611051255-d3107576ba94 h1:G04eS0JkAIVZfaJLjla9dNxkJCPiKIGZlw9AfOhzOD0=
github.com/sabhiram/go-gitignore v0.0.0-20180611051255-d3107576ba94/go.mod h1:b18R55ulyQ/h3RaWyloPyER7fWQVZvimKKhnI5OfrJQ=
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0 h1:TToq11gyfNlrMFZiYujSekIsPd9AmsA2Bj/iv+s4JHE=
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0/go.mod h1:FKdcjfQW6rpZSnxxUvEA5H/cDPdvJ/SZJQLWWXWGrZ0=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=

View file

@ -577,6 +577,8 @@ github.com/ryanuber/go-glob v1.0.0 h1:iQh3xXAumdQ+4Ufa5b25cRpC5TYKlno6hsv6Cb3pkB
github.com/ryanuber/go-glob v1.0.0/go.mod h1:807d1WSdnB0XRJzKNil9Om6lcp/3a0v4qIHxIXzX/Yc=
github.com/sabhiram/go-gitignore v0.0.0-20180611051255-d3107576ba94 h1:G04eS0JkAIVZfaJLjla9dNxkJCPiKIGZlw9AfOhzOD0=
github.com/sabhiram/go-gitignore v0.0.0-20180611051255-d3107576ba94/go.mod h1:b18R55ulyQ/h3RaWyloPyER7fWQVZvimKKhnI5OfrJQ=
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0 h1:TToq11gyfNlrMFZiYujSekIsPd9AmsA2Bj/iv+s4JHE=
github.com/santhosh-tekuri/jsonschema/v5 v5.0.0/go.mod h1:FKdcjfQW6rpZSnxxUvEA5H/cDPdvJ/SZJQLWWXWGrZ0=
github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
github.com/sergi/go-diff v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
github.com/sergi/go-diff v1.1.0/go.mod h1:STckp+ISIX8hZLjrqAeVduY0gWCT9IjLuqbuNXdaHfM=