Commit graph

83 commits

Author SHA1 Message Date
Pat Gavlin f21eda521f
[codegen] Rename the PCL package. (#8103)
It's just confusing that PCL lives in a package named `hcl2`.
2021-09-29 20:11:56 -07:00
T-Vova e696fb6c50
Implemented filebase64() support for Code Generator (#7863)
* Added filebase64 support for Golang

* Fixed function signature

* Added filebase64 support for Typescript

* Added filebase64 support for Python

* Added filebase64 support for Dotnet, fixed Sha1

* Fixed helper method list

Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 23:09:28 +01:00
T-Vova 87e4c92c66
Implemented sha1() support for Code Generator (#7834)
Co-authored-by: Justin Van Patten <jvp@justinvp.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-10 21:40:38 +03:00
T-Vova 913173a603
Revert "Fixed "unknown property '0'" error (#7912)" (#7939)
This reverts commit 8f40edc9e0.
2021-09-09 13:36:49 -10:00
T-Vova 8f40edc9e0
Fixed "unknown property '0'" error (#7912)
Co-authored-by: Pat Gavlin <pat@pulumi.com>
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-09-09 22:12:19 +03:00
T-Vova 065ae27b91
Fixed nil pointer error with BindExpressionText and typeExpr (#7766)
Co-authored-by: Vova Ivanov <jetvova@gmail.com>
2021-08-17 14:22:06 +03:00
Pat Gavlin 1409d88438
[codegen/*] Add join and toBase64 functions. (#5012)
Co-authored-by: Komal Ali <komal@pulumi.com>
2021-07-28 20:41:23 -07:00
Komal fddeea8a88
[codegen] Unify program codegen testing (#7657)
* ProgramGen test driver

* Refactor to avoid exporting hcl2 entities
2021-07-27 18:00:39 -07:00
Pat Gavlin 9b23badedd
[codegen/hcl2] Improve ConversionFrom perf. (#7545)
- Lazily produce conversion failure diagnostics. This lowers the
  allocation volume and cuts down on execution time by avoiding the
  conversion of source and dest types to strings.
- Add a fast path for union conversions that checks if the source type
  is identical to any of the union's element types. Type equality
  checks are generally much faster than type conversion checks.

These changes lead to a significant speedup in codegen time in
azure-native.
2021-07-16 09:56:26 -07:00
Pat Gavlin 80d35758d5
[codegen/pcl] Allocate fewer type instances. (#7548)
When converting a `schema.InputType` to a `model.Type`, calculate the
resolved form of the type in the schema type system rather than the
model type system. The results are semantically identical, but the
number of type objects that are allocated is much smaller b/c
`model.NewOutputType` no longer allocates.

This deserves a little more explanation.

In order to prevent nested outputs and/or promises, `model.NewOutputType`
calculates the resolved form of its argument prior to allocating a new
`OutputType` value. Calculating the resolved form of the argument is a
no-op if the argument is already fully resolved. Therefore, passing in a
fully-resolved schema type prevents `model.NewOutputType` from
calulating the resolved form, and `model.NewOutputType` will only
allocate the `OutputType` itself instead of the `OutputType` and the
resolved form of any eventuals present in its argument.

This has a _very important_ knock-on benefit: the schema -> model type
translator ensures that given a `schema.Type` instance `T` it will
always return the same `model.Type` instance `U`. This termendously
speeds up type equality checks for complex types, as they will now be
referentially identical.

This change alone gives a significant speedup in azure-native code
generation.
2021-07-16 07:49:24 -07:00
Pat Gavlin 7b1d6ec1ac
Reify Input and Optional types in the schema type system. (#7059)
These changes support arbitrary combinations of input + plain types
within a schema. Handling plain types at the property level was not
sufficient to support such combinations. Reifying these types
required updating quite a bit of code. This is likely to have caused
some temporary complications, but should eventually lead to
substantial simplification in the SDK and program code generators.

With the new design, input and optional types are explicit in the schema
type system. Optionals will only appear at the outermost level of a type
(i.e. Input<Optional<>>, Array<Optional<>>, etc. will not occur). In
addition to explicit input types, each object type now has a "plain"
shape and an "input" shape. The former uses only plain types; the latter
uses input shapes wherever a plain type is not specified. Plain types
are indicated in the schema by setting the "plain" property of a type spec
to true.
2021-06-24 09:17:55 -07:00
Pat Gavlin af87128fb5
Add an option to skip resource typechecking. (#7305)
More testing has found further issues with incorrect attribute shapes
generated by tf2pulumi. This knob will allow us to continue to generate
examples until those bugs are fixed.
2021-06-15 10:22:57 -07:00
Pat Gavlin 5867fa84ce
[codegen/pcl] Allow missing object properties. (#7297)
Add an option to allow missing object properties. This will prevent us
from losing examples once resource typechecking is fixed (it is
currently unintentionally disabled because the resource inputs object
type has an unexpected shape).
2021-06-15 09:09:16 -07:00
Mikhail Shilkov 1378c45849 Discriminated unions in schema and programgen 2021-05-05 10:01:22 +02:00
pulumi-bot 73a66f48ea [breaking] Changing the version of go.mod in sdk / pkg to be v3 2021-04-14 19:32:18 +01:00
Mikhail Shilkov 9a837db992 Fix a lint error 2021-02-23 11:17:02 +01:00
Mikhail Shilkov c007f1f2f8 Avoid stack overflow in Azure NextGen program examples 2021-02-04 15:44:20 +01:00
Pat Gavlin 249140242e
Add support for provider-side preview. (#5443)
These changes add support for provider-side previews of create and
update operations, which allows resource providers to supply output
property values for resources that are being created or updated during a
preview.

If a plugin supports provider-side preview, its create/update methods
will be invoked during previews with the `preview` property set to true.
It is the responsibility of the provider to fill in any output
properties that are known before returning. It is a best practice for
providers to only fill in property values that are guaranteed to be
identical if the preview were instead an update (i.e. only those output
properties whose values can be conclusively determined without
actually performing the create/update operation should be populated).
Providers that support previews must accept unknown values in their
create and update methods.

If a plugin does not support provider-side preview, the inputs to a
create or update operation will be propagated to the outputs as they are
today.

Fixes #4992.
2020-10-09 13:13:55 -07:00
Mikhail Shilkov 98f04cd2fc
Ensure top-level precedence to avoid flaky tests (#5508) 2020-10-05 19:50:51 +02:00
Luke Hoban a2a64e55d0
Support recursive object types in pkg/codegen model (#5404)
Fixes #5228.
2020-09-18 17:29:21 -07:00
Komal ff219a2381
Use input/output classes in program gen (#5086) 2020-08-06 15:09:23 -07:00
Levi Blackstone 3286aae20d
[codegen] Make schema and package available to HCL2 codegen (#5114) 2020-08-05 17:27:17 -06:00
Pat Gavlin 00fbbb6544
[codegen/hcl2] Fix nested apply rewrites (#5013)
Rewrites that should produce nested applies due to functions that return
eventual types were instead producing a single top-level apply. These
changes fix that by considering a function that produces an eventual
value as inspecting eventual values.
2020-07-16 10:26:10 -07:00
Pat Gavlin 4961567ecf
[codegen/hcl2] Add a template trivia constructor (#5009)
Just what it says on the tin. Currently it's not possible to create a
valid value of this type because the `bytes` field is unexported. This
constructor fixes that.
2020-07-14 12:07:44 -07:00
Pat Gavlin 86599a06a6
[codegen/hcl2] Fix applies on relative traversals. (#5008)
The apply rewrite for relative traversals did not consider whether or
not the receiver was eventually-typed, and did not properly check
whether or not the relative traversal itself was eventually-typed. These
changes correct those mistakes.
2020-07-14 12:07:35 -07:00
Komal 0ede17ced2
Remove duplicated methods (#4985) 2020-07-08 16:50:46 -07:00
Pat Gavlin d07c3698e5
[codegen/{hcl2,schema}] Abstract package loading. (#4975)
Instead of requiring a plugin host for package loading in the HCL2
binder, define a much narrower interface that exposes the ability to
fetch the schema for a package at a specific version. This interface is
defined in the schema package, which also exposes a caching loader that
is backed by provider plugins.

These changes also add some convenience methods to `*schema.Package` for
fast access to particular resources and functions.

Related to #1635.
2020-07-07 14:45:18 -07:00
Pat Gavlin 2d530e8038
[hcl2] Fix tokenless body item printing. (#4974)
In general, each item in an HCL2 body must be followed by a trailing
newline. The printer did not properly insert these newlines for body
items without any associated tokens/trivia, or with trivia that did not
include a trailing new line.

Related to #1635.
2020-07-07 13:46:26 -07:00
Pat Gavlin a8a20ecb4b
[codegen/*] Add support for explicit secrets. (#4927)
- Add a new builtin function, secret, that marks its input value as
  secret
- Add support for this function to the various code generators

Fixes #4924.
2020-06-30 11:35:24 -07:00
Pat Gavlin 69ba47cff2
[codegen/*] Add support for resource options. (#4925)
The PCL binder has supported resource options for some time, but these
options haven't been used or processed by the various code generators.
These options--particularly the parent and provider options0--are
critical for import codegen. These changes implement the basic set of
options, and add a note about fleshing out the rest as necessary.

One component of these changes is a new rewriter that rewrites property
references into property paths that are understood by the Pulumi engine.
This rewriter is used to preprocess the contents of the `ignoreChanges`
resource option.

These changes also hack around a weakness in the HCL2 type system:
In Go, references to resources should be typed as `hcl2.ResourceType`.
Unfortunately, this breaks the existing collection semantics associated
with resources. Because of this, the Go code generator does not have
enough information to know that it should generate a `[]pulumi.Resource`
for lists of resources. These changes hack around that limitation using
a Go-specific opaque type and some hardcoded comparisons in
`argumentTypeName`.

Fixes #4923.
2020-06-29 16:33:52 -07:00
Evan Boyle 94cd67b564
[Go Program Gen] Improved type handling and invokes (#4749) 2020-06-05 18:52:00 -07:00
Pat Gavlin d77cda98fc
[codegen/hcl2] Fixes for RewriteConversions. (#4743)
- Typecheck in all cases where a type may have changed
- Do not perform literal conversions if the type is already correct
- Perform literal conversions before checking to see if a call to
  `__convert` is required. This catches cases such as string literals
  passed where ints are required. Without this change, that form in
  particular generates a bare number literal rather than a number
  literal wrapped in a `__convert`.
2020-06-02 12:00:35 -07:00
Mikhail Shilkov cf8b6c1237
Convert primitive types in C# program gen (#4686) 2020-05-29 12:38:53 +02:00
Pat Gavlin e973ec0491
[codegen/hcl2] Fix assignability from dynamic. (#4704)
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.

This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:

```
if t.AssignableFrom(model.DynamicType) {
}
```

Fixes #4703.
2020-05-26 09:05:42 -07:00
Pat Gavlin cb8066d74a
[codegen/hcl2] Fix ForExpression.Evaluate. (#4701)
This expression was not evaluated correctly when producing a dictionary.
2020-05-26 09:05:35 -07:00
Pat Gavlin 05032d6850 [codegen/hcl2] Fix assignability from dynamic.
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.

This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:

```
if t.AssignableFrom(model.DynamicType) {
}
```

Fixes #4703.
2020-05-26 08:21:09 -07:00
Pat Gavlin 5330c97684
[codegen/hcl2/model] Remove type caches. (#4683) 2020-05-22 10:47:34 +01:00
Evan Boyle 8dd329f57f
Go basic program gen (#4667) 2020-05-21 10:23:33 -07:00
Mikhail Shilkov adfa8116fb
Introduce k8s compatibility mode to C# codegen (#4544)
Introduce k8s compatibility mode to C# codegen
2020-05-19 11:41:06 +02:00
Pat Gavlin 088751a4a8
[codegen/hcl2] Refactor package schema loading. (#4661)
- Move the implementation of loadPackageSchema into a method on
  PackageCache
- Protect the cache with synchronization primitives to enable
  concurrency in downstream consumers
- Use jsoniter to deserialize schemas
2020-05-18 17:35:11 -07:00
Mikhail Shilkov 5b410ec440
Resolve referenced packages for every TF program (#4636) 2020-05-15 19:18:07 +02:00
Pat Gavlin 647b6627a2
[codegen/hcl2] Fix descent in RewriteConversions. (#4614)
Descend into anonymous function expressions, the operands to conditional
expressions, and the value in for expressions.
2020-05-13 08:25:26 -07:00
Pat Gavlin 33258326e0
[codegen/hcl2] Add a conversion insertion pass. (#4594)
Add a rewriter that reifies implicit conversions into a call to the
`__convert` intrinsic. Code generators can recognize this intrinsic and
use it to generate appropriate conversion code.

Part of this work involves redesigning the type annotations system.
Annotations are now only applicable to opaque and object types. Instead
of inspecting annotations directly, code generators should use
`hcl2.GetSchemaForType` to extract the `schema.Type` for a `model.Type`.
2020-05-11 11:17:36 -07:00
Pat Gavlin 255620ea1e
[codegen/nodejs] Await invokes in async contexts. (#4598)
If we are generating code into an async context (e.g. an async main),
await calls to invoke rather than leaving them as promises. This results
in more idiomatic code withing such contexts.
2020-05-11 10:21:56 -07:00
Pat Gavlin fc00c16e31
[codegen/hcl2] Fix iterator apply rewrites. (#4589)
There are two expressions in HCL2 that are used to iterate over
collections:

- Splat expressions, e.g. `foo.*.bar`, and
- For expressions, e.g. `[for v in foo: v.bar]`

In both of these cases, the parts of the expression that are not the
collection being iterated behave like callbacks, and must be treated as
such by the apply rewriter.
2020-05-07 12:15:40 -07:00
Pat Gavlin 8fded5f29f
[codegen/hcl2] Fix template control tokens. (#4584)
Token detection was broken for conditional and for expressions that
represent template control sequences. The code originally attempted to
determine whether or not a conditional or for expression was a control
sequence by inspecting the expression's parent. Unfortunately, that
approach is unable to distinguish between expressions that are control
sequences and those that are merely template parts. These changes
instead inspect the first token of the expression for a template control
token (i.e. `%{`): if such a token is found, the expression is detected
as a template control sequence.
2020-05-07 09:12:00 -07:00
Pat Gavlin 96300f12d9
[codegen/*] Improve range type binding + codegen. (#4552)
- Determine variable types for ranged resources by typechecking an
  equivalent expression
- Detect top-level await in NodeJS and generate an async main
- Fix `pulumi.all` generation for NodeJS
- Fix a bug in the lowering of relative traversals in Python
2020-05-04 15:04:35 -07:00
Pat Gavlin 8c4c46052d Fix the entries() signature 2020-05-01 13:10:56 -07:00
Pat Gavlin 50b1049337 Fix a misspelling 2020-05-01 11:04:44 -07:00
Pat Gavlin 21634768aa [hcl2] Fix package loading and trivia detection
- Load packages referenced by all node types, not just resource nodes
- Fix trivia detection
2020-05-01 11:00:58 -07:00