Commit graph

55 commits

Author SHA1 Message Date
Lee Briggs 8c314ec39a
Add brew make targets to sdk makefiles 2020-05-13 20:42:01 -07:00
Pat Gavlin 9b1b9cca24
Makefile: properly encode dependencies (#4541)
This allows for the use of `make -j`, which speeds up a full build
dramatically at the cost of rather incomprehensible logs.
2020-05-04 14:26:52 -07:00
CyrusNajmabadi 66bd3f4aa8
Breaking changes due to Feature 2.0 work
* Make `async:true` the default for `invoke` calls (#3750)

* Switch away from native grpc impl. (#3728)

* Remove usage of the 'deasync' library from @pulumi/pulumi. (#3752)

* Only retry as long as we get unavailable back.  Anything else continues. (#3769)

* Handle all errors for now. (#3781)


* Do not assume --yes was present when using pulumi in non-interactive mode (#3793)

* Upgrade all paths for sdk and pkg to v2

* Backport C# invoke classes and other recent gen changes (#4288)

Adjust C# generation

* Replace IDeployment with a sealed class (#4318)

Replace IDeployment with a sealed class

* .NET: default to args subtype rather than Args.Empty (#4320)

* Adding system namespace for Dotnet code gen

This is required for using Obsolute attributes for deprecations

```
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'ObsoleteAttribute' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
Iam/InstanceProfile.cs(142,10): error CS0246: The type or namespace name 'Obsolete' could not be found (are you missing a using directive or an assembly reference?) [/Users/stack72/code/go/src/github.com/pulumi/pulumi-aws/sdk/dotnet/Pulumi.Aws.csproj]
```

* Fix the nullability of config type properties in C# codegen (#4379)
2020-04-14 09:30:25 +01:00
Pat Gavlin 2f22c1c59c
HCL2 updates (#4309)
Pulumi HCL2 IR:
- Add support for invokes
- Add support for resource options, incl. ranged resources
- Allow the apply rewriter to ignore promise-typed values
- Add tests for the binder
- Add support functions for TF: entries and range

NodeJS codegen:
- Simplify for expression codegen
- Add support for invoke codegen
- Add support for entries and range functions
- Add tests

Python codegen:
- Implement codegen for most expression types
- Add support for invoke codegen
- Add tests
2020-04-06 19:43:16 -07:00
evanboyle d93017dbde Revert "move sdk/nodejs/cmd/pulumi-language-nodejs -> pkg/cmd/pulumi-language-nodejs"
This reverts commit a73044247e.
2020-03-18 17:12:55 -07:00
evanboyle 30df499838 move pkg/version -> sdk/go/common/version 2020-03-18 15:25:25 -07:00
evanboyle a73044247e move sdk/nodejs/cmd/pulumi-language-nodejs -> pkg/cmd/pulumi-language-nodejs 2020-03-18 12:25:07 -07:00
CyrusNajmabadi 9151d48ee3
Fix typing for 'Lifted<T>' to work better across versions of TS (#3658) 2019-12-13 11:18:19 -08:00
Justin Van Patten 7278a7429c
Don't remove tests from @pulumi/pulumi npm package (#3532)
The test files are currently included in the npm package, and we have packages that depend on the test files currently, so when installing the linkable `@pulumi/pulumi` package locally, don't delete the tests.
2019-11-19 21:29:53 +00:00
evanboyle 9e7d8a6ed6 Fixing Istanbul usage 2019-11-05 11:17:07 -08:00
CyrusNajmabadi df06b8fc9b
Add publishing to nuget support (#3416) 2019-10-29 20:14:49 -07:00
CyrusNajmabadi e558296afa
Add a helper function to easily convert async functions to sync functions (#2943) 2019-07-16 12:12:21 -07:00
Alex Clemmer 3bc86055c2 Add pulumi-analyzer-package scripts in publishing pipeline 2019-07-16 00:58:33 -07:00
CyrusNajmabadi 55bb3b2486
Simplify API for passing providers to a ComponentResource. (#2609) 2019-06-06 16:20:12 -07:00
CyrusNajmabadi 02f46811db
Add a SxS test for @pulumi/pulumi to help catch when we make breaking changes to core types. (#2610) 2019-03-29 12:27:42 -07:00
CyrusNajmabadi be1c3eb05b
Add support for serializing bigints (#2581) 2019-03-22 15:33:37 -07:00
CyrusNajmabadi 0beab1bc37
test_all for the sdk should run test_fast tests. (#2485) 2019-02-26 14:12:43 -08:00
Matt Ellis 6c7092ff65 Pass -count=1 to disable result caching on go 1.10 and above 2018-11-08 14:11:52 -08:00
Matt Ellis 992b048dbf Adopt golangci-lint and address issues
We run the same suite of changes that we did on gometalinter. This
ended up catching a few new issues, some of which were addressed and
some of which were baselined.
2018-11-08 14:11:47 -08:00
joeduffy 162157c1a7 Add a Dockerfile for the Pulumi CLI
This introduces a Dockerfile for the Pulumi CLI. This makes it
easier to develop and test the engine in a self-contained environment,
in addition to being suitable for running the actual CLI itself.

For instance,

    $ docker run pulumi/pulumi -e "PULUMI_ACCESS_TOKEN=x" up

will run the Pulumi program mounted under the /app volume. This will
be used in some upcoming CI/CD scenarios.

This uses multi-stage builds, and Debian Stretch as the base, for
relatively fast and lean build times and resulting images. We are
intentional about restoring dep packages independent of the actual
source code so that we don't end up needlessly re-depping, which can
consume quite a bit of time. After fixing
https://github.com/pulumi/pulumi/issues/1986, we should explore an
Alpine base image option.

I made the decision to keep this image scoped to just the Go builds.
Therefore, none of the actual SDK packages themselves are built, just
the engine, CLI, and language plugins for Node.js, Python, and Go.
It's possible to create a mega-container that has all of these full
environments so that we can rebuild them too, but for now I figured
it was better to rely on package management for them.

Another alternative would have been to install released binaries,
rather than building them. To keep the useful flow for development,
however, I decided to go the build route for now. If we build at the
same hashes, the resulting binaries "should" be ~identical anyhow.

I've created a pulumi/pulumi Docker Hub repo that we can publish this
into. For now, there is no CI publishing of the image.

This fixes pulumi/pulumi#1991.
2018-09-29 11:48:21 -07:00
Thomas Schersach c01023737e Copy missing pulumi-resource-pulumi-nodejs binary in node lang host dist target 2018-09-13 11:18:06 -07:00
Thomas Schersach 62463ab3bc Added dist target for make, will help with Homebrew (#1731)
* Added dist target for make, will help with Homebrew

* Try to install go dependencies before building

* Make sure dep ensure is called before trying to build SDKs

* Removed dep ensure from dist initial step
2018-08-08 13:00:42 -07:00
Matt Ellis dcb5ae1e1f Stop including native serialization modules
We retained these modules to support using v0.11.X and earlier
versions of @pulumi/pulumi, which required a native module to do
closure serialization. 0.12.X does not need this, so lets stop
including it.
2018-06-04 14:27:01 -07:00
CyrusNajmabadi 5387e78cfa
Support async function serialization. (#1311) 2018-05-03 12:25:52 -07:00
Matt Ellis 409477b951 Invoke node directly from the language host
Instead of using a shell script to jump from the language host into
node, just invoke node directly. This makes our start-up path a little
simpler to understand and indirectly fixes pulumi/home#156, where we
would fail on Windows if the `-exec` script was in a folder that had
spaces in it (due to a subtle interaction between how go launches cmd
files and how cmd.exe parses arguments).
2018-05-02 11:16:58 -07:00
Matt Ellis ca2fbca13f Continue to add native modules to NODE_PATH
While we no longer use the native runtime module, older versions of
@pulumi/pulumi still require it. Let's continue to have the launcher
put the native module location on the `$PATH`. And we'll include them
in the SDK for a while longer.

Fixes #1177
2018-04-13 14:26:32 -07:00
Sean Gillespie 682f908e77
Implement scope chain free variable lookup in pure TypeScript (#1139)
* Implement closure scope chain analysis in pure TypeScript

This change makes use of four V8 intrinsics to avoid having to use a
native module to inspect the scope chains of live Function objects. This
unfortunately leads to the limitation of not allowing captures of 'this'
in arrow functions, but that is something we are willing to live with
for now.

* Remove native module build and restore from the Makefile

* CR feedback: Be a little more efficient when scanning the scope chain

* Nuke everything related to custom Node versions and the native Node module

* CR feedback: rename native.ts -> v8.ts, document some interfaces in v8.ts
2018-04-10 10:04:11 -07:00
Matt Ellis 5c4a31f692 Adopt new version strategy
Our previous strategy of just using `git describe --tags --dirty` to
compute a version caused issues. The major one was that since version
sort lexigrapically, git's strategy of having a commit count without
leading zeros lead to cases where 0.11.0-dev-9 was "newer than"
0.11.0-dev-10 which is not what you want at all.

With this change, we compute a version by first seeing if the commit
is tagged, and if so, we use that tag. Otherwise, we take the closest
tag and to it append the unix timestamp of the commit and then append
a git hash.

Because we use the commit timestamp, things will sort correctly again.

Part of pulumi/home#174
2018-03-15 18:06:04 -07:00
CyrusNajmabadi 5b244dbdb1
Use a class for Output serialization to ensure that .apply exists on it. (#1040)
Also, rename/cleanup a bunch of serialization code.

Also, generate better environment names in the serialized closure code. Thsi code should be much easier to make sense of as hte names will better track to the original names in the user code.

Also, dedupe simple non-capturing functions. This helps ensure we don't spit out N copies of __awaiter (one per file it is declared in).
2018-03-12 16:27:00 -07:00
CyrusNajmabadi e7c0e4cdaa
Make many fixes to closure serialization (#944)
Make many fixes to closure serialization

Primary things that i've done as part of this change:

    Added support for cyclic objects.
    Properly serialize objects that are shared across different function. previously you would get multiple copies, now you properly reference the same copy.
    Remove the usages of 'hashes' for functions. Because we track identity of objects, we no longer need them.
    Serialize properties of functions (if they have any).
    Handle Objects/Functions with different __proto__s than normal. i.e. classes/constructors. but also anything the user may have done themselves to the object.
    Handle generator functions.
    Handle functions with 'computed' names.
    Handle functions with 'symbol' names.
    Handle serializing Promises as Promises.
    Removed the dual Closure/AsyncClosure tree. One existed solely so we could have a tree without promises (for use in testing maybe?). Because this all exists in a part of our codebase that is entirely async, it's fine to have promises in the tree, and to await them when serializing the Closure to a string.
    Handle serializing class-constructors and methods. Including properly handling 'super' calls.
2018-03-01 00:32:01 -08:00
joeduffy a045e2fb1e Implement more of the Python runtime
This change includes a lot more functionality.  Enough to actually
run the webserver-py example through previews, updates, and destroys!

* Actually wire up the gRPC connections to the engine/monitor.

* Move the Node.js and Python generated Protobuf/gRPC files underneath
  the actual SDK directories to simplify this generally.  No more
  copying during `make` and, in fact, this was required to give a smoother
  experience with good packages/modules for the Python's SDK development.

* Build the Python egg during `make build`.

* Add support for program stacks.  Just like with the Node.js runtime,
  we will auto-parent any resources without explicit parents to a single
  top-level resource component.

* Add support for component resource output properties.

* Add get_project() and get_stack() functions for retrieving the current
  project and stack names.

* Properly use UNKNOWN sentinels.

* Add a set_outputs() function on Resource.  This is defined by the
  code-generator and allows custom logic for output property setting.
  This is cleaner than the way we do this in Node.js, and gives us a
  way to ensure that output properties are "real" properties, complete
  with member documentation.  This also gives us a hook to perform
  name demangling, which the code-generator typically controls anyway.

* Add package dependencies to setuptools.py and requirements.txt.
2018-02-24 08:58:34 -08:00
joeduffy 88dcdd8d2b Substitute ${VERSION} on Windows builds too
This change refactors the way we do ${VERSION} substitution in both
the Node.js SDK's version.js and package.json, so that it can work on
Windows.  This is required now that we are actually parsing semvers.
2018-02-20 14:37:28 -08:00
joeduffy ca3516d3e5 Fix language script merge 2018-02-18 08:08:15 -08:00
joeduffy 548c22d014 Reimplement GetRequiredPlugins in Go
This brings back the Node.js language plugin's GetRequiredPlugins
function, reimplemented in Go now that the language host has been
rewritten from JavaScript.  Fairly rote translation, along with
some random fixes required to get tests passing again.
2018-02-18 08:08:15 -08:00
joeduffy 5d16fc936a Add workspace.GetPluginPath, and use it
This change introduces a workspace.GetPluginPath function that probes
the central workspace cache of plugins for a matching plugin binary that
matches the desired kind, name, and, optionally, version.  It also permits
overriding this with $PATH for developer scenarios.

The analyzer, language, and resource plugin logic now uses this function
for deciding which binary path to load at runtime.
2018-02-18 08:08:15 -08:00
Sean Gillespie d3fb639823 Ship nativeruntime.node as part of the SDK
Fixes #356. Instead of downloading a node binary with our closure
serialization code linked-in, this PR instead publishes the
`nativeruntime.node` produced by the NodeJS SDK build as part of the SDK.

This has a number of advantages. First, it is vastly more easy to
develop closure.cc in this configuration. Second, we have the ability
to ship different `nativeruntime.node`s side-by-side, paving the way
for enabling future versions of Node. Third, we don't have to stay
in the business of shipping custom builds of Node, although we do still
need to ship a version of Node with minor modifications in order for
Windows to still work.
2018-02-16 18:12:33 -08:00
Joe Duffy 5d2f21d527
Merge pull request #926 from pulumi/swgillespie/custom-node
Download and use a custom Node binary containing the closure serialization native module
2018-02-13 19:16:16 -08:00
Sean Gillespie 05cb5368a4
Download and use a custom Node binary instead of linking against
private V8 APIs from within our native Node module.
2018-02-13 14:04:01 -08:00
Joe Duffy e0d3eae16f
Publish README and LICENSE in @pulumi/pulumi (#927) 2018-02-13 12:05:45 -08:00
Joe Duffy 6e5084218c
Use the scoped @pulumi name for linking layout (#920) 2018-02-12 16:16:16 -08:00
Sean Gillespie e87204d3e1
Move language host logic from Node to Go (#901)
* experimental: separate language host from node

* Remove langhost details from the NodeJS SDK runtime

* Cleanup

* Work around an issue where Node sometimes loads the same module twice in two different contexts, resulting in two distinct module objects. Some additional cleanup.

* Add some tests

* Fix up the Windows script

* Fix up the install scripts and Windows build

* Code review feedback

* Code review feedback: error capitalization
2018-02-10 02:15:04 +00:00
Joe Duffy 8ea0133d0e
Copy the package.json with its semver expanded (#864)
Our scripts currently copy the package.json that does *not* have
the expanded semver, so its version is simply "${VERSION}", and NPM
is very much not happy with that.  We can just stop copying the
package.json explicitly since it's inside of the bin/ directory.
2018-01-31 10:34:21 -08:00
Joe Duffy 16ade183d8
Add a manifest to checkpoint files (#630)
This change adds a new manifest section to the checkpoint files.
The existing time moves into it, and we add to it the version of
the Pulumi CLI that created it, along with the names, types, and
versions of all plugins used to generate the file.  There is a
magic cookie that we also use during verification.

This is to help keep us sane when debugging problems "in the wild,"
and I'm sure we will add more to it over time (checksum, etc).

For example, after an up, you can now see this in `pulumi stack`:

```
Current stack is demo:
    Last updated at 2017-12-01 13:48:49.815740523 -0800 PST
    Pulumi version v0.8.3-79-g1ab99ad
    Plugin pulumi-provider-aws [resource] version v0.8.3-22-g4363e77
    Plugin pulumi-langhost-nodejs [language] version v0.8.3-79-g77bb6b6
    Checkpoint file is /Users/joeduffy/dev/code/src/github.com/pulumi/pulumi-aws/.pulumi/stacks/webserver/demo.json
```

This addresses pulumi/pulumi#628.
2017-12-01 13:50:32 -08:00
Matt Ellis 46c35281ab Adopt new makefile system
See https://github.com/pulumi/home/pull/56 for more details.
2017-11-16 23:56:29 -08:00
Matthew Riley 418ff30725 Remove unused PROCCNT variable 2017-10-27 14:42:47 -07:00
CyrusNajmabadi d929c169de Enable tslinting of the nodejs sdk. (#433) 2017-10-18 15:03:56 -07:00
Matt Ellis fc14329cfc Stop using yarn scripts for building
We have Makefiles, so we might as well use them, instead of spliting
build logic across two systems.
2017-10-16 10:47:37 -07:00
Matt Ellis e7c3aaba69 Merge pull request #395 from pulumi/pulumi-service-interface
More engine refactoring
2017-10-11 13:44:36 -07:00
CyrusNajmabadi b713990b5e Enable 'use const' linter rule. (#405)
* Enable 'use const' linter rule.
2017-10-10 14:50:55 -07:00
Matt Ellis 6bab1dbad4 Pass $(YARNFLAGS) to all yarn invocations
This let's you set things like YARNFLAGS==--offline which is helpful
when you are on an airplane. Yarn can still pick up stuff that you had
pulled down recently from its local cache
2017-10-09 18:21:55 -07:00