Non-string provider inputs must be projected as JSON formatted strings. The current codegen simply calls `json.dumps` for such properties, but this does not work for the new input types, which aren't JSON serializable.
To address this, make use of the new `pulumi.runtime.to_json` utility function, which is capable of serializing raw dicts and input types as JSON.
We currently emit array types as `List[T]` for Python, but `List[T]` is invariant, which causes type checkers like mypy to produce errors when values like `["foo", "bar"]` are passed as args typed as `List[pulumi.Input[str]]`. Instead, we should move to using `Sequence[T]` which is covariant, and does not have this problem.
We actually already do this for `Dict` vs. `Mapping`, emitting map types as `Mapping[str, T]` rather than `Dict[str, T]` because `Mapping[str, T]` is covariant. This change makes us consistent for array types.
* add initial pull-request workflow
* run SDK test all
* add SDK tests
* fixup make targets
* add dist target
* revert back to 5 updates
* disable test
* add issue for test disabling
These changes add initial support for the construction of remote
components. For now, this support is limited to the NodeJS SDK;
follow-up changes will implement support for the other SDKs.
Remote components are component resources that are constructed and
managed by plugins rather than by Pulumi programs. In this sense, they
are a bit like cloud resources, and are supported by the same
distribution and plugin loading mechanisms and described by the same
schema system.
The construction of a remote component is initiated by a
`RegisterResourceRequest` with the new `remote` field set to `true`.
When the resource monitor receives such a request, it loads the plugin
that implements the component resource and calls the `Construct`
method added to the resource provider interface as part of these
changes. This method accepts the information necessary to construct the
component and its children: the component's name, type, resource
options, inputs, and input dependencies. It is responsible for
dispatching to the appropriate component factory to create the
component, then returning its URN, resolved output properties, and
output property dependencies. The dependency information is necessary to
support features such as delete-before-replace, which rely on precise
dependency information for custom resources.
These changes also add initial support for more conveniently
implementing resource providers in NodeJS. The interface used to
implement such a provider is similar to the dynamic provider interface
(and may be unified with that interface in the future).
An example of a NodeJS program constructing a remote component resource
also implemented in NodeJS can be found in
`tests/construct_component/nodejs`.
This is the core of #2430.
An extra constructor overload was recently added to pass undefined state from
`get` for resources that do not have any state inputs (notably Kubernetes
resources). This ended up breaking PaC's `validateResourceOfType`, which relies
on type inference in common usage to determine the resource's args type based
on the signature of the constructor.
This constructor overload isn't necessary. Instead, we can remove it and modify
how the constructor is called inside `get`. This also makes it so we're not
exposing details about `get`'s implementation in the public API.
Changed the codegen in 6fd72dc0 but missed a condition that
is causing incorrect code in pulumi-kubernetes. This change
correctly generates inputs in both conditional branches.
If PyName(name) != PyNameLegacy(name), we panic, unless name is in an exclusion list.
Once all providers have been updated to this codegen (and we've added to `useLegacyName` and `excludeFromPanic` as appropriate), we can go back and remove the panic behavior along with the `excludeFromPanic` behavior.
If there's a config subpackage, we had explicitly added it to the map of modules because there *are* config properties, so we know it will be non-empty. This change ensures the config subpackage is imported in the top-level package's __init__.py and that the config subpackage itself has an __init__.py emitted.
Resource doc changes for Python:
- Types are included in constructor/function args
- The property names for input/output types are now always snake_case, regardless of the generated mapping tables, to match the new input/output classes
- Some other minor tweaks to function/constructor signatures (e.g. removed the `__props__` arg, as it's not meant to be used directly; use `@staticmethod` for static `get` methods).
Fixes: #5134
This ensures that `pulumi history` has been deprecated in favor of
the new `pulumi stack history` command. The deprecated command will
be removed in v3.0.0 of Pulumi
Fixes: #5126
In the newServiceSecretsManager, we were loading and saving the project stack
without understanding if any changes had actually been made - e.g. changing the
stack to be a new secrets provider
This has now been guarded against, tests added to understand when it will
actually be saved and we no longer get unnecessary config sorting when interacting
with a default stack AND no longer get an empty config map in our config
file when we initiate a new stack with a default secrets provider
* Use a package map where appropriate
* Fix imports
* handle NoneType for literal value expressions in go program gen
* add operator tests for c#, python, and node
Co-authored-by: evanboyle <evan@pulumi.com>
Fixes: #5104
This reverts a code change that was checking initially for the
SecretsProvider of the currentStack being an empty string. When it
was an empty string, we were checking the backend type and we were
setting a serviceSecretsManager. This wasn't correct, the logic
needed to check for an empty SecretsProvider AND an empty EncryptionSalt
The important part is that it needed to check for the EncryptionSalt
to make sure it wasn't a passphrase secrets manger
We make several calls to `os/user`, which uses CGO and means
cross-compilation is not possible. This replaces `os/user` with the
`luser` package, which is a drop-in replacement which does not use `CGO`
Certain operations in `engine/diff` mutate engine events during display.
This mutation can occur concurrently with the serialization of the event
for persistence, which causes a panic in the CLI. These changes fix the
offending code and add code that copies each engine event before
persisteing it in order to guard against future issues.
Rewrites that should produce nested applies due to functions that return
eventual types were instead producing a single top-level apply. These
changes fix that by considering a function that produces an eventual
value as inspecting eventual values.
Just what it says on the tin. Currently it's not possible to create a
valid value of this type because the `bytes` field is unexported. This
constructor fixes that.
The apply rewrite for relative traversals did not consider whether or
not the receiver was eventually-typed, and did not properly check
whether or not the relative traversal itself was eventually-typed. These
changes correct those mistakes.
In general, a package/module name in these targets is derived from the
module portion of a type token. If the type token is not already in an
expected form--namely, all lowercase--the generated package/module names
will also be in unexpected forms. These changes normalize the module
names to lowercase s.t. the generated package/module names conform to
expectations.
- NodeJS: fix get + constructor generation for types with required args
and no state type.
- Go: fix type + import generation for properties in the root module.
- NodeJS: fix get + constructor generation for types with required args
and no state type.
- Go: fix type + import generation for properties in the root module.
Previously, streamInvoke was only supported by
the query command. Copied the implementation
into the resource monitor, which will allow
streaming invoke commands to run during updates.
Also fixed a bug with cancellation of streaming
invokes. The check was comparing against a
hardcoded string, which did not match the actual
error string. Instead, we can rely on the error code.
Add a new package, `codegen/importer`, that can generate definitions for
resource states in PCL or TS/Python/C#/Go. The pipeline is relatively
simple: given a list of resource states, generate a PCL program in
memory, bind it, and pass it to the language-specific code generator.
This builds upon the existing PCL IR, and can be used with the currently
supported code generators.
Related to #1635.
* Use the correct format for package name when module name is empty.
* Add an exclusion for Docker's Image component resource when generating the Python formal params.
* Prefix the type name with the package name when linking to Python function names.
Instead of requiring a plugin host for package loading in the HCL2
binder, define a much narrower interface that exposes the ability to
fetch the schema for a package at a specific version. This interface is
defined in the schema package, which also exposes a caching loader that
is backed by provider plugins.
These changes also add some convenience methods to `*schema.Package` for
fast access to particular resources and functions.
Related to #1635.
In general, each item in an HCL2 body must be followed by a trailing
newline. The printer did not properly insert these newlines for body
items without any associated tokens/trivia, or with trivia that did not
include a trailing new line.
Related to #1635.
- Remove `Info` from `Source`. This method was not used.
- Remove `Stack` from `EvalSource`. This method was not used.
- Remove `Type` and `URN` from `Step`. These values are available via
`Res().URN.Type()` and `Res().URN`, respectively. This removes the
possibility of inconsistencies between the type, URN, and state of the
resource associated with a `Step`.
- Remove URN from StepEventMetadata.
The k8s SDK logs deprecations warnings as part of the
provider logic, and includes a flag to suppress these
warnings. Including them in the SDKs breaks this logic.
The PCL binder has supported resource options for some time, but these
options haven't been used or processed by the various code generators.
These options--particularly the parent and provider options0--are
critical for import codegen. These changes implement the basic set of
options, and add a note about fleshing out the rest as necessary.
One component of these changes is a new rewriter that rewrites property
references into property paths that are understood by the Pulumi engine.
This rewriter is used to preprocess the contents of the `ignoreChanges`
resource option.
These changes also hack around a weakness in the HCL2 type system:
In Go, references to resources should be typed as `hcl2.ResourceType`.
Unfortunately, this breaks the existing collection semantics associated
with resources. Because of this, the Go code generator does not have
enough information to know that it should generate a `[]pulumi.Resource`
for lists of resources. These changes hack around that limitation using
a Go-specific opaque type and some hardcoded comparisons in
`argumentTypeName`.
Fixes#4923.
The docs generator previously assumed that the opts parameter
for every resource was of the CustomResource type. This is
incorrect for the YAML and Helm overlays, which are
ComponentResources. This should be handled more generally
once our schema supports ComponentResources, but this fixes
the docs for now.
For the NodeJS k8s SDK, rather than falling back to default values
using the || operator, use the nullish coalescing operator (??).
This avoid situations where the primary value is set to false,
and then is overridden by the default value.
Use the schema package's Markdown parser and walk its AST to extract
examples.
These changes also rename StripNonRelevantExamples to FilterExamples.
This is preparatory work for #4159 and #4632.
In particular, use the parser to filter and extract examples. This also
sets up support for entity references in documentation that can be used
in order to render language-specific names for resources, functions,
types, and properties.
Related to #4632 and #4159.
Several users reported cases where error messages would
cause a panic if they contained accented characters. I wasn't
able to reproduce this failure locally, but tracked down the
panic to logging gRPC calls. The Message field is typed as
a string, which requires all of the characters to be valid UTF-8.
This change runs each log string through the strings.ToValidUTF8
function, which will replace any invalid characters with the
"unknown" character. This should prevent the the logger from
panicking.
Fixes: #4827
Diff:
```
if encrypted is not None:
- warnings.warn("Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"", DeprecationWarning)
+ warnings.warn("Rename PostgreSQL role resource attribute \"encrypted\" to \"encrypted_password\"", DeprecationWarning)
pulumi.log.warn("encrypted is deprecated: Rename PostgreSQL role resource attribute "encrypted" to "encrypted_password"")
```
When running `pulumi up`, after the preview, we prompt asking whether to proceed with the update. If you type to filter the options and then hit an arrow key a couple times, the CLI panics. This is a bug in the `survey` library we depend on. The issue has been fixed in the library upstream; this change updates our dependency.
Automatically create a virtual environment and install dependencies in it with `pulumi new` and `pulumi policy new` for Python templates.
This will save a new `virtualenv` runtime option in `Pulumi.yaml` (`PulumiPolicy.yaml` for policy packs):
```yaml
runtime:
name: python
options:
virtualenv: venv
```
`virtualenv` is the path to a virtual environment that Pulumi will use when running `python` commands.
Existing projects are unaffected and can opt-in to using this by setting `virtualenv`, otherwise, they'll continue to work as-is.
If extra files are present, make sure that appropriate modules are
created as necessary and that extra sources are exported from each
module's `index.ts`.
If the type of a configuration variable is a token type (i.e. a type
provided by an overlay) with the underlying type string, call
`config.get` rather than `config.getObject`. This fixes a regression in
the `aws.config.region` variable.
- Split out common functionality into generateModuleContextMap method
- Add LanguageResource and LanguageProperty structs for use by downstream
codegen tools
- Add LanguageResources function for use by downstream codegen tools
Add support for union types that indicate a default type for targets
that do not support unions, or do not support unions in certain
positions (e.g. output properties). The NodeJS backend makes use of this
in combination with a new flag, `disableUnionOutputTypes`, to avoid
generating unions in output types.
These changes also refactor the various module ->
package/module/namespace mapping methods so that these entities can be
fetched by their language name rather than their token.
- Typecheck in all cases where a type may have changed
- Do not perform literal conversions if the type is already correct
- Perform literal conversions before checking to see if a call to
`__convert` is required. This catches cases such as string literals
passed where ints are required. Without this change, that form in
particular generates a bare number literal rather than a number
literal wrapped in a `__convert`.
If the source file for a resource or function definition would collide
with a reserved filename (e.g. `index.ts`), append an underscore to its
filename (e.g. `index_.ts`).
These changes are necessary in order to move tfgen over to the schema
code generator.
The schema generator for tfgen has an annoying behavior for nested types
in which it does not separate input and output types. Worse, the shape
of the type that results from a collision of input and output types is
order-dependent: whichever of the two was observed last wins (note that
the shape is still determenistic, as the order in which types are
recorded is always the same). As a result, the NodeJS code generator
needs to know the set of required properties for the other aspect of the
type in order to generate proper code.
Contributes to
https://github.com/pulumi/pulumi-terraform-bridge/issues/179.
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.
This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:
```
if t.AssignableFrom(model.DynamicType) {
}
```
Fixes#4703.
Types like output(T), promise(T), and union(T_0, ..., T_N) should be
assignable from dynamic if they contain an element type that is
assignable from dynamic.
This allows consumers to use the following code to check if some type
behaves like the dynamic type w.r.t. conversions:
```
if t.AssignableFrom(model.DynamicType) {
}
```
Fixes#4703.
- Move the implementation of loadPackageSchema into a method on
PackageCache
- Protect the cache with synchronization primitives to enable
concurrency in downstream consumers
- Use jsoniter to deserialize schemas
# This is the 1st commit message:
Use patched azure gocloud library
Fixes#4642
# The commit message #2 will be skipped:
# sync with pulumi-master branch
* Remove code that was hiding Go as a language option for k8s overlay resources. Mark input args type for Functions as optional in Go.
* Show a special note in the Go function's snippet when the function name does not match other languages.
* Make sure the draft PR stays assigned to the original author. Add some more comments.
* Remove logic to use the Go ModuleToPackage map to then lookup the relevant C# namespace. Make getLanguageModuleName a method of modContext since passing in a package is not needed anymore.
Some type tokens contain characters that are not valid in a single
URL path element (`/` in particular). These type tokens may be
path-escaped, and should be unescaped before they are interpreted.
- Add support for constant values in unions (e.g. `"foo" | "bar"`)
- Generate `Input<T | U>` instead of `Input<T> | Input<U>`
- Emit deprecation messages for resources
- Handle nil state inputs
- Allow packages to provide a README
- Remove sync invoke support
These changes are part of
https://github.com/pulumi/pulumi-terraform-bridge/issues/179, and
restore functional parity with the current `tfgen` generator.
* Generate language package details in the index pages.
* Add a new DocLanguageHelper interface method to get to a module per-language.
* Add a new workflow file for automatically creating draft docs PRs for previewing resource docs for AWS and Kubernetes as a result of changes in the resource docs generator.
Add a rewriter that reifies implicit conversions into a call to the
`__convert` intrinsic. Code generators can recognize this intrinsic and
use it to generate appropriate conversion code.
Part of this work involves redesigning the type annotations system.
Annotations are now only applicable to opaque and object types. Instead
of inspecting annotations directly, code generators should use
`hcl2.GetSchemaForType` to extract the `schema.Type` for a `model.Type`.
If we are generating code into an async context (e.g. an async main),
await calls to invoke rather than leaving them as promises. This results
in more idiomatic code withing such contexts.
* Add a new template for examples section. Extract the examples section into a structured format for custom template processing.
* Update the description IFF we were able to extract examples sections from it.
* Update doc comments and add missing file header for the newly added file.
* Make the example description readable. Add a check for empty example sections.
* Add a chooser right below the Example Usage header. Remove javascript as a language.
* Allow an empty new-line between short-codes boundaries.
Future proof this test by depending on `@pulumi/pulumi` 2.0. `@pulumi/pulumi` 1.x depends on the native `grpc` package. In the future, when we run on future versions of Node.js, prebuilt native binaries of `grpc` might not be available which could result in this test failing.
In `@pulumi/pulumi` 2.0, we have moved to using `@grpc/grpc-js`, a pure JavaScript implementation, which doesn’t have this problem.
There are two expressions in HCL2 that are used to iterate over
collections:
- Splat expressions, e.g. `foo.*.bar`, and
- For expressions, e.g. `[for v in foo: v.bar]`
In both of these cases, the parts of the expression that are not the
collection being iterated behave like callbacks, and must be treated as
such by the apply rewriter.
Token detection was broken for conditional and for expressions that
represent template control sequences. The code originally attempted to
determine whether or not a conditional or for expression was a control
sequence by inspecting the expression's parent. Unfortunately, that
approach is unable to distinguish between expressions that are control
sequences and those that are merely template parts. These changes
instead inspect the first token of the expression for a template control
token (i.e. `%{`): if such a token is found, the expression is detected
as a template control sequence.
Fixes: #4585
This was happening by default in the test framework and then again
in the go language runtime
Also we were replacing a module and then running go mod tidy - this
is unnecessary - go mod download is sufficient here to ensure we
have the correct dependency we need. The tidy may have been removing
the replacements added
* Adjust the args type names for k8s overlay resources.
* Revert to using input type link for kubernetes non-overlay resources.
* Use the correct module name to lookup the C# namespace for resources that belong to a module.
* Helm's arg types don't use the version number.
Specifically, handle index and relative traversal expressions, and clean
up the code a little bit.
This should also help us pick up more `pulumi.interpolate` calls in TS.
With the addition of Python examples, doc comments may now contain
triple quotes. These must be escaped in order to avoid malformed source
files.
Fixes#4568.
- Determine variable types for ranged resources by typechecking an
equivalent expression
- Detect top-level await in NodeJS and generate an async main
- Fix `pulumi.all` generation for NodeJS
- Fix a bug in the lowering of relative traversals in Python
* Restore the API type links for C#.
* Also restore them in function.tmpl.
* Add package details to the Functions template as well. Add a global template function to detect if the APIDocLinks has links for a language. Don't generate C# API doc links for k8s.
This code was accidentally removed in a prior PR.
These changes also remove an assert; the situation the assertion guards
against is valid and occurs when dealing with traversals inside of
splat expressions.
Unlike most languages with interpolated strings, Python's formatted
string literals do not allow the nesting of quotes. For example,
this expression is not legal Python:
f"Foo {"bar"} baz"
If an interpolation requires quotes, those quotes nust differ from the
quotes used by the enclosing literal. We can fix the previous example
by rewriting it with single quotes:
f"Foo {'bar'} baz"
However, this presents a problem if there are more than two levels of
nesting, as Python only has two kinds of quotes (four if the outermost
string uses """ or '''): in this case, the expression becomes
unspellable, and must be assigned to a local that is then used in place
of the original expression. So this:
f"Foo {bar[f'index {baz["qux"]}']} zed"
becomes this:
index = "qux"
f"Foo {bar[f'index {baz[index]}']}"
To put it bluntly, Python code generation reqiures register allocation,
but for quotes. These changes implement exactly that.
These changes also include a fix for traversals that access values that
are dictionaries rather than objects, and must use indexers rather than
attributes.
Mapping happens at package scope, not type scope. This is what causes
some of the surprising mixes of camel and snake casing in our packages.
In order to accommodate this, build the case mapping tables up front,
and refer to them when performing case mapping.
The tokens that make up the "key" portion of an index traversal
(e.g. `"foo"` in `a["foo"]`) are structured like those that make up a
block label: an open quote token, a string literal token, and a close
quote token. The token mapper did not account for that fact, and instead
recorded the key token as the open quote. These changes correct that
error, and adjust the code in `literalText` to allow for
properly-escaped and quoted strings where necessary.
If a single process is going to bind and generate multiple programs, it
is useful to be able to cache package schemas in order to avoid the
(large) overhead of deserializing schemas multiple times.
Use strings.Builder instead of raw slices. We could go further here and
collapse to a single, ~right-sized bulder at the cost of complexity in
the `stateAcronym -> stateLowerOrNumber` transition.
Also, eliminate the pyName function, as it is redundant.
When a community member sends a pull request to the repo, travis can't
decrypt some environment variables which include decrypting the GCP
credentials.
In addition to this, there are a few tests that run that require the
`PULUMI_ACCESS_TOKEN` which can't be used.
This PR does 2 things:
- Adds a check to the GCP credentials decryption which checks the source
of the PR (whether it's from within the local repo, or a fork)
- Sets an environment variable: `COMMUNITY_PRS` which is used downstream
during some tests. If the var exists, some tests are skipped
This should enhance the experience for community pull requests
significantly
Fixes#4508
* Add a new DocLangHelper interface method to generate function names per language. Use the functionNames override map in Go to correctly generate a function name.
* Add kong to the title lookup map.
* Check if item was found in map.
* Use title case for the nodejs Function args type name. Use title case for the function name in the index page.
* Also fix the result name for a Function in nodejs to use title case.
* Use the module name to lookup the package.
* Use the title-case name for the Function name in the index page. Fix the language value passed to the lang chooser in function.tmpl.
* Use title case for nested type titles.
* Add title lookup for GitHub.
Some of the apply rewriter's assumptions were broken by the richer
expressions available in HCL2. These changes fix those broken
assumptions, in particular the assumption that only scope traversal
expressions are sources of eventual values.
When writing the snapshot to the filestate bucket, we can retry in the
event of an error, which helps users who are experiencing issues around
write rates to GCS
In particular, ensure that they are keyword-safe. This affects
`aws:lambda:*` in particular: prior to these changes, we were generating
code into `pulumi_aws.lamdba`, which is not referencable due to its use
of the `lamdba` keyword. With these changes, we generate code into
`pulumi_aws.lambda_`.
There is also a small fix for multi-line non-formatted strings: these
strings do not need escaped braces.
When referencing `secretOutputNames` in from another stack, spurious
diffs can often be created because the secret output slice was not
ordered.
This PR orders the slice before it's added to the propertymap, ensuring
the order always remains the same
This probably seems like a trivial change, but while debugging #4258 it
was apparent that an error could have come from 3 different places. This
rewords an error message to make it slightly clearer what the error is.
* Fix output property names in Python codegen.
Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.
Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.
The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.
* Fix Python input property names.
- Fix input property names and forms for invokes. Previously we
generated a dict; now we generate properly-named args.
- Fix nested property names for resources.
* PR feedback
Fixes: #4444
Before:
```
$ pulumi stack
Current stack is 47BE2956-D665-4EC3-9AE6-4D4A1C417074:
Managed by demo-mbp
No updates yet; run 'pulumi up'
Current stack resources (0):
No resources currently in this stack
Use `pulumi stack select` to change stack; `pulumi stack ls` lists known ones
```
After:
```
$ pulumi stack --show-name
47BE2956-D665-4EC3-9AE6-4D4A1C417074
```
- Fix input property names and forms for invokes. Previously we
generated a dict; now we generate properly-named args.
- Fix nested property names for resources.
* [codegen/go] Fix accessors on struct ptr outputs
The accesor methods on nestred struct Ptr outputs were previously not accepting pointer typed inputs as they should, and would thus always panic if used.
The "simple" fix would be to just accept the pointer type and blindly dereference it. But this doesn't seem like the right experience - it would make these accessors very unsafe to use in practice.
Instead, this PR implements the accessors on pointer-typed outputs as nil-coaslescing, always lifting the output type into a pointer type and flowing a nil value into the result type. This ensures the accessor will not nil-deref, and that user code can handle the `nil` value itself (or use `.Apply` directly to implement more specialized behaviour).
Before:
```go
// Name of your S3 bucket.
func (o BuildStorageLocationPtrOutput) Bucket() pulumi.StringOutput {
return o.ApplyT(func(v BuildStorageLocation) string { return v.Bucket }).(pulumi.StringOutput)
}
```
After:
```go
// Name of your S3 bucket.
func (o BuildStorageLocationPtrOutput) Bucket() pulumi.StringPtrOutput {
return o.ApplyT(func(v *BuildStorageLocation) *string {
if v == nil {
return nil
}
return &v.Bucket
}).(pulumi.StringPtrOutput)
}
```
However, due to the decision to have this more usable behaviour, this is a breaking change, as some/many accessors now return a pointer type when they previously did not.
Fixespulumi/pulumi-azure#530.
* Mark nested property types as requiring ptr types
* Add CHANGELOG
* More fixes
Some property names are mapped from their `camelCase` Pulumi name to a
`snake_case` Python name. This mapping is irregular, and only occurs for
resources properties and function calls.
Note that there's still more work to do here: this only fixes names on
the output side; the input side is still broken for nested resource
proprerties and function calls.
The underlying design--annotated types in `hcl2/model`--may need some
additional work in the future, but I _believe_ it's good enough for now.
* Optimize titles and descriptions
* Move regexp to file level variable
* wip
* Add title tags and description to module and pkg index pages
* Remove block_external_search_index from index.tmpl
* Improve resource descriptions
* Add more providers to look up map
* Improve module level descriptions
Clean up
cleanup
* Update the function.tmpl with the auto-generated disclaimer. Pass title tag and meta description for the function template as well.
* Fix whitespace in the package_details template..
* Account for empty module names for package-level resources. Move logic for generating title tag and description into separate methods for Resources and Functions.
* Add a test for the generated resource and function title tag.
Co-authored-by: Praneet Loke <1466314+praneetloke@users.noreply.github.com>
* Add ids to headers so they can be linked to
* Make header ids more general
Rather than including the identifier in the name of the id, use a simpler more generic id.
* H3=>H2 for Modules/Resources/Functions/Package Details headings
This way, on individual resource/function pages, it doesn't make it look like "Package Details" is a sub heading under "Supporting Types" in the right-hand "ON THIS PAGE".
* Add auto-generated disclaimer for the resource.tmpl as well.
* Allow specifying the languages to show in the lang chooser. Hide the go language for k8s overlay resources.
* Add another exclusion for generating the C# constructor param for k8s overlay resources.
* Replace hard-coded exclusions for skipping input properties with checking for ConstValue attribute of input properties.
- Define `null` in Pulumi HCL2
- Bind Pulumi HCL2 in topological order s.t. variable types can be
properly computed
- Fix resources that range over bools and numbers
- Add element, length, lookup, readFile, and split functions
- Do not rewrite function signatures with input types during binding
- Fix splat expression binding for non-lists
- Add support for evaluating expressions
- Add support for operator precedence to code generators
- Add support for constants to the HCL2 IR
- Add support for generating ranged resources in Python
- Add support for generating conditional resource in Node and Python
- Fix various naming issues in Python
After importing some resources, and running a second update with the
import still applied, an unexpected replace would occur. This wouldn't
happen for the vast majority of resources, but for some it would.
It turns out that the resources that trigger this are ones that use a
different format of identifier for the import input than they do for the
ID property.
Before this change, we would trigger an import-replacement when an
existing resource's ID property didn't match the import property, which
would be the case for the small set of resources where the input
identifier is different than the ID property.
To avoid this, we now store the `importID` in the statefile, and
compare that to the import property instead of comparing the ID.
* Add a method to the DocLanguageHelper interface to get the API doc link for Pulumi types.
* Add a test for Go API doc link methods.
* Use GetDocLinkForPulumiType in gen_function as well.
* Make `async:true` the default for `invoke` calls (#3750)
* Switch away from native grpc impl. (#3728)
* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)
* Only retry as long as we get unavailable back. Anything else continues. (#3769)
* Handle all errors for now. (#3781)
* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)
* Upgrade all paths for sdk and pkg to v2
* Backport C# invoke classes and other recent gen changes (#4288)
Adjust C# generation
* Replace IDeployment with a sealed class (#4318)
Replace IDeployment with a sealed class
* .NET: default to args subtype rather than Args.Empty (#4320)
* Adding system namespace for Dotnet code gen
This is required for using Obsolute attributes for deprecations
```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```
* Fix the nullability of config type properties in C# codegen (#4379)
- Parentheses were not handled properly
- Literals inside of template control sequences were not handled
properly
These changes also improve test coverage for the printers.
* Fix k8s input type links for constructor args.
* Skip using property case maps for k8s Python.
* Fix the namespace of C# constructor args type and resource type.
This is required for using Obsolute attributes for deprecations
```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```
* Use the correct module names for languages.
* Add Kubernetes to the title lookup map.
* Add output properties to the map only if filtered output properties is non-zero length.
* Add a check for the Python code gen to prevent infinte recursion. Remove the custom logic for k8s for generating Python property case maps.
* Fix bug with C# links for k8s constructor resources.
* Apply a style on the deprecation message.
* Use the display name when rendering a property type that does not have a link.
* Fix the python type string generator for docs to inspect union types.
* Update cleanTypeString to account for nodejs package names.
* Add missing lookup names for packages.
Pulumi HCL2 IR:
- Add support for invokes
- Add support for resource options, incl. ranged resources
- Allow the apply rewriter to ignore promise-typed values
- Add tests for the binder
- Add support functions for TF: entries and range
NodeJS codegen:
- Simplify for expression codegen
- Add support for invoke codegen
- Add support for entries and range functions
- Add tests
Python codegen:
- Implement codegen for most expression types
- Add support for invoke codegen
- Add tests
* Add new templates for generating index files. Remove the now irrelevant code.
* Add comment for generating index files by deduping module names. Use the path format of a module name as the link.
* Add a trailing slash for module links in the index page.
* Move the categories rendered in the index file into their own template file. Handle kubernetes provider specific module name conversion while generating parent modules.
* Fix issue with the k8s provider resource being generated as a module rather than a resource on the package-level index page.
* Show the deprecation message of a resource at the top of the document.
* Move k8s specific things to its own file.
* Add title attribute to the list items. Show the last part of a module name, in case it contains path separators
* Lookup the package name from the Go language info object for k8s then use that to lookup the C# namespace.
* Move the logic for cleaning a property type string for display names to a separate function.
* Export the title function from go and dotnet code generators for use in the resource doc generator. Use the title function from the respective lang code gens to match the correct title case used there. Fix issue with the Kubernetes namespace not being stripped for C# property type strings.
* Skip including apiVersion and kind as input properties for Kubernetes until pulumi-kubernetes#1062 is merged.
* Fix an issue with stripping the module name from type doc links in nodejs.
* Remove unused code that visited object types. Update the nested types code to account for types that may reference themselves. Added glog logging statements. Save the package-level language info objects, so we can use them later.
* Check if a type token has already been added as an input or an output type when collecting nested types.
* Rename appearsIn to typeUsage.
* Scan provider resource also using the special-case function if package is k8s.
* Simplify the display names for Pulumi types by removing the Pulumi prefix.
- config variables are now specified as `config <name> <type>`
- output variables are now specified as `output <name> <type>`
- apply arguments are assigned better names by the apply rewriters
These changes add preliminary (read: incomplete) support for
representing Pulumi programs using HCL2. Language-specific code
generators can use this representation as a basis for understanding the
semantics of a Pulumi program.
Some properties have constant values, such as the
apiVersion and kind for the Kubernetes provider. Add
a Const field to the Property schema to support this.
Instead of always running `npm install`, if the Policy Pack is Python, emit instructions on how to install dependencies. Also minor cleanup/refactoring.
This was still in pkg and wasn't accessible from the scripts location.
This code should be in the go/common location of sdk so that it can be
accessed from within our build scripts
Without this change, we were not able to get access to the Version number
for Python and none of our Python builds have been publishing binaries
* Decode the C# language info from the schema package and set it in the dotnet lang helper for docs. Pass the qualifier for C# property type strings based on whether it is an input or an output property.
* Don't pass a module name for Pulumi core types used in TS constructor and Function params.
* Add tests for generating doc links for nodejs types.
* Add test for confirming input doc link for C# type.
* Fix the C# type name for InvokeOptions.
The initial config represents any config that was specified programmatically to the Policy Pack, for Policy Packs that support programmatic configuration like AWSGuard.
* Add tests for the examples processing.
* Update the schema-based code generators to strip out non-relevant code examples wrapped in short-codes.
* Extract language relevant examples for Functions too.
* Simpler way of extracting the surrounding text from the description.
* Add some more new-line characters to clean-up the examples sections.
* Update the C# invoke signature in the Functions template used by resource docs generator.
* Add a new function to the DocLanguageHelper interface for resource docs generator to generate language-specific property names.
* Add a new DisplayName property to the propertyType struct for simpler display names without module names.
* Update templates to use the DisplayName property of propertyType. Strip the namespace/module name from type names for use as display names.
* Generate Go package maps at the beginning, so that the resource docs can use the appropriate package context for generating a property type string name.
* Temporarily block the resource docs from search indexing.
* Use PascalCase for nested type names in Python.
* Fix bug with Python property name casing for nested types by checking the property case maps.
* Fix rename stack message when attempting to move organizations
* Include the URL to change stack ownership
Co-authored-by: Lee-Ming Zen <lee@pulumi.com>
Rather than shelling out to the `rm` and `cp` commands,
use Go's os utils to perform these operations. This will
work on any platform rather than just Linux.
This package contains a driver for generating source code from HCL2
expressions. The driver is based on the fmt package's support for custom
formatters.
These changes implement `GetRequiredPlugins` for Go using a registry
mechanism and an alternate entry point for `pulumi.Run`. Packages that
require plugins are expected to register themselves with the Pulumi SDK.
When `pulumi.Run` is used and the `PULUMI_PLUGINS` envvar is truthy, the
program will dump a JSON-encoded description of its required plugins to
stdout. The language host then uses this description to respond to
These changes add a package for type checking and modeling HCL2
configurations. It is made up of three primary components:
1. A static type system
2. A semantic representation of HCL2 expressions and a binder from HCL2
native syntax to this representation
3. A semantic representation of HCL2 structural elements and binders
from HCL2 native syntax to this representation.
The type system is described in the "Extended Types" section of the
specification. The semantic representations of expressions and
structural elements are documented in their implementations.
* Generate the constructor params for Python along with other languages.
* Remove redundant py_function_param nested template. Declare a new type for defining property characteristics rather than using inlining formal params. Generate the Lookup functions for all languages similar to the constructor params with linking enabled.
* Fix bug with generating the input arg type name for Functions in Go.
* Add prefix for args param of a Go-based Resource Function.
* Input args for Go-based Functions use Lookup*Args and not Get*Args.
* Turns out that args for Go-based Functions use a different prefix based on whether the function is a package-level or module-level Function.
* Update the Python list and dictionary type names for the resource doc generator.
* Add a separate function for Python doc helper return a type string representing dictionaries that are simple maps and don't have a known nested element type.
* Also remove the Examples property from the resourceDocArgs type since the resource Comment will contain the multi-lang examples on their surrounded by a shortcode.
The integration test framework currently supports using `dep` for dependency management.
However, `dep` has no native ability to manage "yarn link"-style dependencies on locally available packages.
This is a necessary scenario for testing in most repos though, as (e.g.) examples in the Kubernetes repo need to test against the locally available version of `pulumi-kubernetes`.
The best we can do is a trick of (a) deleting the vendored copy of the locally available dependency (b) copying the locally available dependency into the vendor folder (c) deleting the nested vendor folder in the new copy of the locally available dependency.
* Added a new template for Functions. Implement the genFunction method for generating the docs for Functions.
* Rename type resourceArgs to resourceDocArgs. Minor updates the resource template.
* Generate nested types for Functions.
* Unexport types that don't need to be exported. Create the doc language helper objects in an init function and reuse them rather than recreating them every time. Update genNestedTypes to work with schema functions or resources.
* Fixed bug in nested type generation for Functions. Fixed bug in generating input and output doc links for nested types.
The changes in #4004 caused old provider configuration to be used even when a provider was different between inputs and outputs, in the case that the diff returned DiffUnkown.
To better handle that case, we compute a more accurate (but still conservative) DiffNone or DiffSome so that we can ensure we conservatively update to a new provider when needed, but retain the performance benefit of not creating and configuring a new provider as much as possible.
Part of https://github.com/pulumi/pulumi-aws/issues/814.
