Commit graph

320 commits

Author SHA1 Message Date
Luke Hoban f0a24079ce
Filter the list of templates shown by default (#3147)
If any templates are marked as `Important: true` then by default show only those templates along with an option to see additional templates.

Fixes #3094.
2019-08-27 17:56:49 -07:00
Luke Hoban 47dc3cfb8b
Print a Welcome to Pulumi message (#3145)
Present a warm welcome to users when they interactively login.

Also use this as an opportunity to present a "Tip of the Day" - which for now we will use to highlight auto-naming as this has been a common new user question.
2019-08-27 17:19:35 -07:00
Alex Clemmer 99d70e4610 Introduce MarshalOptions.{RejectAsset, RejectArchive}
Not all resource providers support Pulumi's Asset and Archive types. In
particular, the Kubernetes provider should reject any resource
definition that contains either of these types.

This commit will introduce two MarshalOptions that will make it easy for
the Kubernetes provider to guarantee that no properties of this type are
in a resource request, as it's deserializing the request from the
engine.
2019-08-26 15:19:14 -07:00
Mikhail Shilkov c270204387 Check project existence during pulumi new 2019-08-23 09:09:54 +02:00
Chris Smith eb0934970c
Add support for filtering stacks by organization, tag (#3108)
* Add support for filtering stacks by organization, tag

* Update CHANGELOG.md

* Address PR feedback

* Address even more PR feedback

* Support empty-string filters
2019-08-22 13:56:43 -07:00
Matt Ellis e349f3c094 Fix stack reference issue when running on a pre -beta.3 CLI
In #3071 we made change to the built in provider for `StackReference`
to return additional data about the names of outputs which were
secrets.  The SDKs uses this information to decide if it should mark a
stack output as secret when returning it's value from `getOutput`.

When using an older CLI which does not report this data, we hit an
error:

```
 TypeError: Cannot read property 'outputs' of undefined
```

This was caused by a refactoring error where we extracted a private
helper out of the StackReference class, but neglected to change one
instance of `this` to `sr`. While we do have tests that exercise this
feature, we would only see this bug when using an older CLI, and we
don't have automated tests here that use the neweset `@pulumi/pulumi`
with an older `pulumi` CLI
2019-08-22 11:55:02 -07:00
Matt Ellis 7696ed8e94 Prepare for 1.0.0-beta.3 release 2019-08-21 10:33:33 -07:00
Pat Gavlin 42fc75fffe
Fail read steps with missing resources. (#3123)
Just what it says on the tin.

Fixes #262.
2019-08-21 10:09:02 -07:00
Pat Gavlin 8745440c1b
Allow users to explicitly disable delete-before-replace. (#3118)
With these changes, a user may explicitly set `deleteBeforeReplace` to
`false` in order to disable DBR behavior for a particular resource. This
is the SDK + CLI escape hatch for cases where the changes in
https://github.com/pulumi/pulumi-terraform/pull/465 cause undesirable
behavior.
2019-08-20 15:51:02 -07:00
Matt Ellis 900046a6ed
Merge pull request #3089 from pulumi/ellismg/fix-2714
Fix issues in the filestate backend
2019-08-19 10:50:44 -07:00
Mikhail Shilkov 7f672a1c50 Changelog 2019-08-19 08:28:34 +02:00
Matt Ellis 828086d638 Do not attempt to load checkpoint before saving a snapshot
For historical reasons, we used to need to require to load an existing
checkpoint to copy some data from it into the snapshot when saving a
new snapshot. The need for this was removed as part of the general
work in #2678, but we continued to load the checkpoint and then just
disregard the data that was returned (unless there was an error and
that error was not FileNotFound, in which case we would fail).

Our logic for checking if something was FileNotFound was correct when
we wrote it, but when we adopted go-cloud in order to have our
filestate backend also write to blob storage backends like S3, we
forgot that we had checks like `os.IsNotExists()` floating around
which were now incorrect. That meant if the file did not exist for
some reason, instead of going along as planned, we'd error out now
with an error saying something wasn't found.

When we write a checkpoint, we first "backed up" the initial version
by renaming it to include a `.bak` suffix, then we write the new file
in place. However, this can run afoul of eventual consistency models
like S3, since there will be a period of time in which a caller may
observe that the object is missing, even after a new version is
written (based on my understanding of [S3's consistency
model](https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html#ConsistencyModel)

Since we no longer need to actually copy any information from the
previous checkpoint, we can simply remove the call entirely to load
it.

As a follow up, we need to audit places inside the filebased backend
that assume `os.*` functions are going to do what we want them to do,
since in general they will not.

Fixes #2714
2019-08-16 13:40:55 -07:00
Matt Ellis 342f8311a1 Fix renaming a freshly created stack using the local backend
Attempting to `pulumi stack rename` a stack which had been created but
never updated, when using the local backend, was broken because
code-paths were not hardened against the snapshot being `nil` (which
is the case for a stack before the initial deployment had been done).

Fixes #2654
2019-08-16 13:39:34 -07:00
Paul Stack f8db8e4209
Allow resource IDs to change on reresh steps (#3087)
* Allow resource IDs to change on reresh steps

This is a requirement for us to be able to move forward with
versions of the Terraform Azurerm provider. In v1.32.1, there was
a state migration that changed the ID format of the azure table
storage resource

We used to have a check in place for old ID being equal to new ID.
This has been changed now and we allow the change of ID to happen
in the RefreshStep

* Update pkg/resource/deploy/step.go

Co-Authored-By: Pat Gavlin <pat@pulumi.com>
2019-08-16 21:04:03 +03:00
Matt Ellis 88901627c4 Update CHANGELOG.md 2019-08-13 16:12:20 -07:00
Pat Gavlin fdfef5f334
Update the NodeJS version compat checks. (#3083)
- Unify the 1.x.y series and the 0.17.z series
- Fix the check s.t. post-1.0, only the major versions are required to
  match
2019-08-13 15:40:25 -07:00
Pat Gavlin 57b6e84645
Prepare for the v1.0.0-beta.1 release. (#3079)
- Update scripts/get-version and friends to understand the new -alpha
  and -beta tags
- Update the CHANGELOG
2019-08-13 11:41:32 -07:00
CyrusNajmabadi 1a698cbc9e
Fix crashes around secrets with 'undefined' value. (#3069) 2019-08-12 16:00:20 -07:00
Pat Gavlin 91af1a93c4
Fix a reentrancy issue in _sync_await. (#3056)
_sync_await was not reentrant with respect to _run_once: the latter
captures the length of the ready list before it iterates it, and the
former drains the ready list by reentering _run_once. Fix this by
tracking the length of the list before pumping the event loop and then
pushing cancelled handles on to the list as necessary after pumping the
loop.

These changes also fix an issue with `export`ing awaitables.

Fixes #3038.
2019-08-08 19:51:11 -07:00
Pat Gavlin 3c03ee3bdd
Fix invoke error reporting. (#3048)
- Report all failures
- Use appropriate member functions to access failure details

Fixes https://github.com/pulumi/pulumi-terraform/issues/396
2019-08-08 09:14:36 -07:00
Christian Nunciato 6800c41da4
Merge pull request #3036 from pulumi/cnunciato/actions-11
Send GitHub Actions CLI output to a file
2019-08-06 17:09:33 -07:00
Christian Nunciato e0b5bae352 Address feedback, update CHANGELOG
Signed-off-by: Christian Nunciato <c@nunciato.org>
2019-08-06 13:58:28 -07:00
Justin Van Patten eedc760689
Fix typo in CHANGELOG.md (#3029) 2019-08-05 17:16:22 -07:00
Pat Gavlin 2ab814fd09
Do not resolve missing outputs to inputs in preview. (#3014)
This can cause `apply`s to run on values that may change during an
update, which can lead to unexpected behavior.

Fixes #2433.
2019-08-05 12:44:04 -07:00
Pat Gavlin 8fd05fcd9f Prepare the CHANGELOG for v0.17.28. 2019-08-05 11:46:51 -07:00
Pat Gavlin 06034fab40
Only await input properties once. (#3024)
These changes fix a bug in the Python runtime that would cause any
awaitable input properties passed to a resource that are missing
from the resource's output properties to be awaited twice. The fix is
straightforward: rather than roundtripping an input property through
serialize/deserialize, just deserialized the already-serialized input
property.

Fixes #2940.
2019-08-03 10:29:19 -07:00
Luke Hoban 6ed4bac5af
Support additional cloud secrets providers (#2994)
Adds support for additional cloud secrets providers (AWS KMS, Azure KeyVault, Google Cloud KMS, and HashiCorp Vault) as the encryption backend for Pulumi secrets. This augments the previous choice between using the app.pulumi.com-managed secrets encryption or a fully-client-side local passphrase encryption.

This is implemented using the Go Cloud Development Kit support for pluggable secrets providers.

Like our cloud storage backend support which also uses Go Cloud Development Kit, this PR also bleeds through to users the URI scheme's that the Go CDK defines for specifying each of secrets providers - like `awskms://alias/LukeTesting?region=us-west-2` or `azurekeyvault://mykeyvaultname.vault.azure.net/keys/mykeyname`.

Also like our cloud storage backend support, this PR doesn't solve for how to configure the cloud provider client used to resolve the URIs above - the standard ambient credentials are used in both cases. Eventually, we will likely need to provide ways for both of these features to be configured independently of each other and of the providers used for resource provisioning.
2019-08-02 16:12:16 -07:00
Pat Gavlin 63eb7abb59
Make pulumi.runtime.invoke synchronous. (#3019)
These changes make the `pulumi.runtime.invoke` function invokable in a
synchronous manner. Because this function still needs to perform
asynchronous work under the covers--namely awaiting a provider URN and
ID if a provider instance is present in the `InvokeOptions`--this
requires some creativity. This creativity comes in the form of a helper
function, `_sync_await`, that performs a logical yield from the
currently running event, manually runs the event loop until the given
future completes, performs a logical resume back to the
currently executing event, and returns the result of the future.

The code in `_sync_await` is a bit scary, as it relies upon knowledge of
(and functions in) the internals of the `asyncio` package. The necessary
work performed in this function was derived from the implementations of
`task_step` (which pointed out the need to call `_{enter,leave}_task`)
and `BaseEventLoop.run_forever` (which illustrated how the event loop is
pumped). In addition to potential breaking changes to these internals,
the code may not work if a user has provided an alternative implementation
for `EventLoop`. That said, the code is a close enough copy of
`BaseEventLoop.run_forever` that it should be a reasonable solution.
2019-08-02 14:19:56 -07:00
Charlie Hills 654f5e2d35 Fixes python get_secret always returning None (#3022)
get_secret was looking up secrets by the fully resolved key, but should use the simple key.
2019-08-02 09:59:51 -07:00
CyrusNajmabadi d1376db975
Support arbitrary stack export values in python. (#3015) 2019-08-01 20:00:07 -07:00
Luke Hoban aac25eabc4
Add requireOutput to StackReference (#3007)
Provides an additional helper function to read outputs from a stack reference in case it is known that the stack output must be present. This is similar to the design for config.get and config.require.

Fixes #2343.
2019-08-01 11:27:32 -07:00
Mikhail Shilkov 653501758d Retry renaming a temporary folder during plugin installation (#3008)
A workaround for #2695

During the plugin installation, we create a temporary folder, unzip the binary, and then rename the folder to a permanent name. The rename fails 90% of the time with access denied. An immediate retry of renaming seems to always succeed.
2019-07-31 17:44:26 -07:00
Mikhail Shilkov d7a3987f47 Prevent poorly formatted strings in fatal error messages (#2995)
Fixes #1511
2019-07-31 17:42:59 -07:00
Luke Hoban 3d4c01abeb
Add Output.concat to Python (#3006)
Fixes #2366
2019-07-31 11:35:05 -07:00
Pat Gavlin 67ec74bdc5
Pass ignoreChanges to providers. (#3005)
These changes add support for passing `ignoreChanges` paths to resource
providers. This is intended to accommodate providers that perform diffs
between resource inputs and resource state (e.g. all Terraform-based
providers, the k8s provider when using API server dry-runs). These paths
are specified using the same syntax as the paths used in detailed diffs.

In addition to passing these paths to providers, the existing support
for `ignoreChanges` in inputs has been extended to accept paths rather
than top-level keys. It is an error to specify a path that is missing
one or more component in the old or new inputs.

Fixes #2936, #2663.
2019-07-31 11:39:07 -05:00
CyrusNajmabadi 7bdd590586
Add deprecation warnings. (#3004) 2019-07-30 15:51:44 -07:00
CyrusNajmabadi 96d7bedce4
Update CL (#2998) 2019-07-29 12:51:11 -07:00
James Nugent 12b5318730 Update CHANGELOG.md 2019-07-29 12:27:26 -05:00
Justin Van Patten bb43f63d51
Backport Changelog entries from the docs repo (#2992)
https://www.pulumi.com/docs/reference/changelog/ contains changelog entries for older versions, but hasn't been kept up-to-date. We'll be removing those from that page, after having moved the older entries to the CHANGELOG.md in this repo, which this commit does.
2019-07-26 19:33:07 -07:00
Luke Hoban 74854de490 Prepare for 0.17.26 2019-07-26 08:54:53 -07:00
CyrusNajmabadi 237f8d2222
Add python aliases support. (#2974) 2019-07-25 11:21:06 -07:00
Pat Gavlin fa05e5cb05
Migrate old providers without outputs. (#2973)
If we encounter a provider with old inputs but no old outputs when reading
a checkpoint file, use the old inputs as the old outputs. This handles the
scenario where the CLI is being upgraded from a version that did not
reflect provider inputs to provider outputs, and a provider is being
upgraded from a version that did not implement `DiffConfig` to a version
that does.

Fixes https://github.com/pulumi/pulumi-kubernetes/issues/645.
2019-07-23 13:39:21 -07:00
Pat Gavlin e305484c1a
Fix the CHANGELOG for v0.17.23 (#2972)
And link to the tracking issue for working around bugs exposed by
provider diffing.
2019-07-22 16:14:12 -07:00
Cameron Stokes cded857802 Fix bug link in changelog. (#2946)
Link to correct bug for custom timeouts entry.
2019-07-21 12:28:37 -05:00
Luke Hoban fa4da84669
Add get_object functions to Python SDK (#2959)
Fixes #2320
2019-07-20 07:04:18 -07:00
CyrusNajmabadi fa904a45e5
Fix changelog (#2964) 2019-07-19 16:46:00 -07:00
CyrusNajmabadi 61b0c74c21
Update changelog. (#2963) 2019-07-19 16:20:04 -07:00
Luke Hoban 563dff3d92
changelog: fix header for 0.17.24 release (#2960)
* changelog: fix header for 0.17.24 release

* More cleanup in CHANGELOG
2019-07-19 13:31:40 -07:00
CyrusNajmabadi 837155994b
Update changelog. (#2953) 2019-07-19 11:26:08 -07:00
Luke Hoban 3768e5c690
Python Dynamic Providers (#2900)
Dynamic providers in Python.

This PR uses [dill](https://pypi.org/project/dill/) for code serialization, along with a customization to help ensure deterministic serialization results.

One notable limitation - which I believe is a general requirement of Python - is that any serialization of Python functions must serialize byte code, and byte code is not safely versioned across Python versions.  So any resource created with Python `3.x.y` can only be updated by exactly the same version of Python.  This is very constraining, but it's not clear there is any other option within the realm of what "dynamic providers" are as a feature.  It is plausible that we could ensure that updates which only update the serialized provider can avoid calling the dynamic provider operations, so that version updates could still be accomplished.  We can explore this separately.

```py
from pulumi import ComponentResource, export, Input, Output
from pulumi.dynamic import Resource, ResourceProvider, CreateResult, UpdateResult
from typing import Optional
from github import Github, GithubObject

auth = "<auth token>"
g = Github(auth)

class GithubLabelArgs(object):
    owner: Input[str]
    repo: Input[str]
    name: Input[str]
    color: Input[str]
    description: Optional[Input[str]]
    def __init__(self, owner, repo, name, color, description=None):
        self.owner = owner
        self.repo = repo
        self.name = name
        self.color = color
        self.description = description

class GithubLabelProvider(ResourceProvider):
    def create(self, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).create_label(
            name=props["name"],
            color=props["color"],
            description=props.get("description", GithubObject.NotSet))
        return CreateResult(l.name, {**props, **l.raw_data}) 
    def update(self, id, _olds, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.edit(name=props["name"],
               color=props["color"],
               description=props.get("description", GithubObject.NotSet))
        return UpdateResult({**props, **l.raw_data})
    def delete(self, id, props):
        l = g.get_user(props["owner"]).get_repo(props["repo"]).get_label(id)
        l.delete()

class GithubLabel(Resource):
    name: Output[str]
    color: Output[str]
    url: Output[str]
    description: Output[str]
    def __init__(self, name, args: GithubLabelArgs, opts = None):
        full_args = {'url':None, 'description':None, 'name':None, 'color':None, **vars(args)}
        super().__init__(GithubLabelProvider(), name, full_args, opts)

label = GithubLabel("foo", GithubLabelArgs("lukehoban", "todo", "mylabel", "d94f0b"))

export("label_color", label.color)
export("label_url", label.url)
```


Fixes https://github.com/pulumi/pulumi/issues/2902.
2019-07-19 10:18:25 -07:00
Luke Hoban fc38d7d4d9
Handle mixed versions of Resources in parent hierarchy (#2942)
For new properties added to `Resource`, we need to make sure to handle cases where these are undefined as they may not be available on versions of `Resource` that come from older SDK versions, which could me side-by-side in a single Pulumi program execution.

Fixes #2938
2019-07-16 11:15:26 -07:00
Pat Gavlin 049d7cec4e Prepare the CHANGELOG for release. 2019-07-16 11:05:29 -07:00
Paul Stack 02ffff8840
Addition of Custom Timeouts (#2885)
* Plumbing the custom timeouts from the engine to the providers

* Plumbing the CustomTimeouts through to the engine and adding test to show this

* Change the provider proto to include individual timeouts

* Plumbing the CustomTimeouts from the engine through to the Provider RPC interface

* Change how the CustomTimeouts are sent across RPC

These errors were spotted in testing. We can now see that the timeout
information is arriving in the RegisterResourceRequest

```
req=&pulumirpc.RegisterResourceRequest{
           Type:                    "aws:s3/bucket:Bucket",
           Name:                    "my-bucket",
           Parent:                  "urn:pulumi:dev::aws-vpc::pulumi:pulumi:Stack::aws-vpc-dev",
           Custom:                  true,
           Object:                  &structpb.Struct{},
           Protect:                 false,
           Dependencies:            nil,
           Provider:                "",
           PropertyDependencies:    {},
           DeleteBeforeReplace:     false,
           Version:                 "",
           IgnoreChanges:           nil,
           AcceptSecrets:           true,
           AdditionalSecretOutputs: nil,
           Aliases:                 nil,
           CustomTimeouts:          &pulumirpc.RegisterResourceRequest_CustomTimeouts{
               Create:               300,
               Update:               400,
               Delete:               500,
               XXX_NoUnkeyedLiteral: struct {}{},
               XXX_unrecognized:     nil,
               XXX_sizecache:        0,
           },
           XXX_NoUnkeyedLiteral: struct {}{},
           XXX_unrecognized:     nil,
           XXX_sizecache:        0,
       }
```

* Changing the design to use strings

* CHANGELOG entry to include the CustomTimeouts work

* Changing custom timeouts to be passed around the engine as converted value

We don't want to pass around strings - the user can provide it but we want
to make the engine aware of the timeout in seconds as a float64
2019-07-16 00:26:28 +03:00
Pat Gavlin e1a52693dc
Add support for importing existing resources. (#2893)
A resource can be imported by setting the `import` property in the
resource options bag when instantiating a resource. In order to
successfully import a resource, its desired configuration (i.e. its
inputs) must not differ from its actual configuration (i.e. its state)
as calculated by the resource's provider.

There are a few interesting state transitions hiding here when importing
a resource:
1. No prior resource exists in the checkpoint file. In this case, the
   resource is simply imported.
2. An external resource exists in the checkpoint file. In this case, the
   resource is imported and the old external state is discarded.
3. A non-external resource exists in the checkpoint file and its ID is
   different from the ID to import. In this case, the new resource is
   imported and the old resource is deleted.
4. A non-external resource exists in the checkpoint file, but the ID is
   the same as the ID to import. In this case, the import ID is ignored
   and the resource is treated as it would be in all cases except for
   changes that would replace the resource. In that case, the step
   generator issues an error that indicates that the import ID should be
   removed: were we to move forward with the replace, the new state of
   the stack would fall under case (3), which is almost certainly not
   what the user intends.

Fixes #1662.
2019-07-12 11:12:01 -07:00
Matt Ellis 611c616468 Update CHANGELOG for 0.17.22 2019-07-12 10:32:39 -07:00
CyrusNajmabadi 73b4100899
Support the exported value of a stack being a Promise/Output itself (#2916) 2019-07-09 17:22:35 -07:00
Matt Ellis 858517a7ca Correctly push secretness up during deserialization in runtimes
There current RPC model for Pulumi allows secret values to be deeply
embedded in lists or maps, however at the language level, since we
track secrets via `Output<T>` we need to ensure that during
deserialization, if a list or a map contains a secret, we need to
instead treat it as if the entire list or map was a secret.

We have logic in the language runtimes to do this as part of
serialization. There were a few issues this commit addresses:

- We were not promoting secretness across arrays in either Node or
  Python
- For Python, our promotion logic was buggy and caused it to behave in
  a manner where if any value was secret, the output values of the
  object would be corrupted, because we'd incorrectly treat the
  outputs as a secret who's value was a map, instead of a map of
  values (some of which may be secret).

This caused very confusing behavior, because it would appear that a
resource creation call just did not set various output properties when
one or more of them ended up containing a secret.
2019-07-09 10:40:27 -07:00
Matt Ellis f11f4f7498
Merge pull request #2890 from Charliekenney23/only-print-emojis-in-interactive
Don't print emojis in non-interactive mode
2019-07-02 16:48:32 -07:00
Mikhail Shilkov bc542e2dc4 Allow specifying a local path to templates for pulumi new (#2884)
* Allow specifying a local path to templates for pulumi new

* Add CHANGELOG

* gofmt

* Add tests

* Linting error
2019-07-01 14:40:55 -07:00
Mikhail Shilkov e30e6208a0 Normalize Windows paths for directory archive (#2887)
* Normalize Windows paths for directory archive

* Changelog

* Remove the redundant check
2019-07-02 00:04:24 +03:00
Pat Gavlin 6e5c4a38d8
Defer all diffs to resource providers. (#2849)
Thse changes make a subtle but critical adjustment to the process the
Pulumi engine uses to determine whether or not a difference exists
between a resource's actual and desired states, and adjusts the way this
difference is calculated and displayed accordingly.

Today, the Pulumi engine get the first chance to decide whether or not
there is a difference between a resource's actual and desired states. It
does this by comparing the current set of inputs for a resource (i.e.
the inputs from the running Pulumi program) with the last set of inputs
used to update the resource. If there is no difference between the old
and new inputs, the engine decides that no change is necessary without
consulting the resource's provider. Only if there are changes does the
engine consult the resource's provider for more information about the
difference. This can be problematic for a number of reasons:

- Not all providers do input-input comparison; some do input-state
  comparison
- Not all providers are able to update the last deployed set of inputs
  when performing a refresh
- Some providers--either intentionally or due to bugs--may see changes
  in resources whose inputs have not changed

All of these situations are confusing at the very least, and the first
is problematic with respect to correctness. Furthermore, the display
code only renders diffs it observes rather than rendering the diffs
observed by the provider, which can obscure the actual changes detected
at runtime.

These changes address both of these issues:
- Rather than comparing the current inputs against the last inputs
  before calling a resource provider's Diff function, the engine calls
  the Diff function in all cases.
- Providers may now return a list of properties that differ between the
  requested and actual state and the way in which they differ. This
  information will then be used by the CLI to render the diff
  appropriately. A provider may also indicate that a particular diff is
  between old and new inputs rather than old state and new inputs.

Fixes #2453.
2019-07-01 12:34:19 -07:00
Charles Kenney b6d469d53e update changelog 2019-06-30 01:43:16 -04:00
Matt Ellis 29320af68b Release v0.17.21 2019-06-26 21:49:31 -07:00
Matt Ellis 881db4d72a Correctly flow secretness across POJO serliazation for stack outputs
Our logic to export a resource as a stack output transforms the
resource into a plain old object by eliding internal fields and then
just serializing the resource as a POJO.

The custom serialization logic we used here unwrapped an Output
without care to see if it held a secret. Now, when it does, we
continue to return an Output as the thing to be serialized and that
output is marked as a secret.

Fixes #2862
2019-06-26 15:16:07 -07:00
James Nugent 42593ad24c Update CHANGELOG.md 2019-06-25 20:46:25 +01:00
CyrusNajmabadi 7a1db9c24a
Update CL (#2868) 2019-06-24 11:54:40 -07:00
CyrusNajmabadi 8553683713
Update changelog. (#2866) 2019-06-23 09:48:14 -07:00
CyrusNajmabadi 7b8421f0b2
Fix crash when there were multiple duplicate aliases to the same resource. (#2865) 2019-06-23 02:16:18 -07:00
CyrusNajmabadi c674198e5a
Make our changelog consistent with the CHG format (#2863) 2019-06-21 11:00:19 -07:00
CyrusNajmabadi a2ad0c034c
Update changelog (#2854) 2019-06-21 00:16:21 -07:00
CyrusNajmabadi ef3cad6bf1
Reads should not cause resources to be displayed in our progress display (#2844) 2019-06-18 15:38:32 -07:00
Matt Ellis 93025e5f36 Use PULUMI_TEST_MODE to control test mode in Python
This mirrors the UX that we have in node where you can set
`PULUMI_TEST_MODE` to control if test mode is enabled from outside
your program.

Fixes #2818
2019-06-18 10:48:29 -07:00
CyrusNajmabadi 11a19a4990
Make it possible to get a StackReference output promptly (#2824) 2019-06-17 12:25:56 -07:00
Artem Yarmoluk f1b5fb6e0f
Backend setting in project config
Signed-off-by: Artem Yarmoluk <koolgen@gmail.com>
2019-06-13 20:02:03 +03:00
CyrusNajmabadi b9c57858ed
Update changelog. (#2819) 2019-06-12 15:47:17 -07:00
CyrusNajmabadi 54cbda80c1
Store information about a CustomResource's provider in __providers. (#2816) 2019-06-11 16:57:37 -07:00
Matt Ellis bc9448b1cf Fix crash in pulumi stack output when stack is empty 2019-06-11 12:58:29 -07:00
Matt Ellis 1e519ce150 Fix a bug when logging into bucket urls
Currently if you log into s3://bucket/subdirectory, Pulumi will write
files to s3://bucket/.pulumi and not s3://bucket/subdirectory/.pulumi,
this corrects the error.
2019-06-10 13:43:17 -07:00
Mike Metral 803533d7d4 docs(login): escape codeblocks, and add object store state instructions 2019-06-07 10:37:06 -07:00
CyrusNajmabadi 55bb3b2486
Simplify API for passing providers to a ComponentResource. (#2609) 2019-06-06 16:20:12 -07:00
Matt Ellis e8a5acf1f2 Prepare CHANGELOG.md for release 2019-06-06 10:35:24 -07:00
Pat Gavlin dfa120e6b4
Check for provider changes in mustWrite (#2805)
Recent changes to default provider semantics and the addition of
resource aliases allow a resource's provider reference to change even if
the resource itself is considered to have no diffs. `mustWrite` did not
expect this scenario, and indeed asserted against it. These changes
update `mustWrite` to detect such changes and require that the
checkpoint be written if and when they occur.

Fixes #2804.
2019-06-05 16:27:26 -07:00
Matt Ellis fdc4c64789 Update v0.17.15 release date 2019-06-05 09:39:11 -07:00
Matt Ellis 4e429592d6 Update CHANGELOG.md for release
[skip ci]
2019-06-04 11:39:19 -07:00
Sean Gillespie 2870518a64 Refine resource replacement logic for providers (#2767)
This commit touches an intersection of a few different provider-oriented
features that combined to cause a particularly severe bug that made it
impossible for users to upgrade provider versions without seeing
replacements with their resources.

For some context, Pulumi models all providers as resources and places
them in the snapshot like any other resource. Every resource has a
reference to the provider that created it. If a Pulumi program does not
specify a particular provider to use when performing a resource
operation, the Pulumi engine injects one automatically; these are called
"default providers" and are the most common ways that users end up with
providers in their snapshot. Default providers can be identified by
their name, which is always prefixed with "default".

Recently, in an effort to make the Pulumi engine more flexible with
provider versions, it was made possible for the engine to have multiple
default providers active for a provider of a particular type, which was
previously not possible. Because a provider is identified as a tuple of
package name and version, it was difficult to find a name for these
duplicate default providers that did not cause additional problems. The
provider versioning PR gave these default providers a name that was
derived from the version of the package. This proved to be a problem,
because when users upgraded from one version of a package to another,
this changed the name of their default provider which in turn caused all
of their resources created using that provider (read: everything) to be
replaced.

To combat this, this PR introduces a rule that the engine will apply
when diffing a resource to determine whether or not it needs to be
replaced: "If a resource's provider changes, and both old and new
providers are default providers whose properties do not require
replacement, proceed as if there were no diff." This allows the engine
to gracefully recognize and recover when a resource's default provider changes
names, as long as the provider's config has not changed.
2019-06-03 12:16:31 -07:00
Matt Ellis 0fbce2c9ce Update CHANGELOG.md 2019-06-03 09:31:50 -07:00
Luke Hoban 15e924b5cf
Support aliases for renaming, re-typing, or re-parenting resources (#2774)
Adds a new resource option `aliases` which can be used to rename a resource.  When making a breaking change to the name or type of a resource or component, the old name can be added to the list of `aliases` for a resource to ensure that existing resources will be migrated to the new name instead of being deleted and replaced with the new named resource.

There are two key places this change is implemented. 

The first is the step generator in the engine.  When computing whether there is an old version of a registered resource, we now take into account the aliases specified on the registered resource.  That is, we first look up the resource by its new URN in the old state, and then by any aliases provided (in order).  This can allow the resource to be matched as a (potential) update to an existing resource with a different URN.

The second is the core `Resource` constructor in the JavaScript (and soon Python) SDKs.  This change ensures that when a parent resource is aliased, that all children implicitly inherit corresponding aliases.  It is similar to how many other resource options are "inherited" implicitly from the parent.

Four specific scenarios are explicitly tested as part of this PR:
1. Renaming a resource
2. Adopting a resource into a component (as the owner of both component and consumption codebases)
3. Renaming a component instance (as the owner of the consumption codebase without changes to the component)
4. Changing the type of a component (as the owner of the component codebase without changes to the consumption codebase)
4. Combining (1) and (3) to make both changes to a resource at the same time
2019-05-31 23:01:01 -07:00
Praneet Loke 5ac446fffd
Add CI vars for Bitbucket Pipelines (#2783)
* Introduce a new package under ciutil for individual CI systems. Split-out each CI system with env var detection for each.

* Add Bitbucket Piplines env var detection.

* Update changelog with note about adding Bitbucket Pipelines detection.

* Rename the CI system structs.

* Move files from ciutil/systems to ciutil. Un-export some types that don't need visibility beyond the ciutil package.

* Un-export DetectSystem function and the System type.

* Add a test for CI systems which we only know by name and nothing else, i.e. those with just a baseCI implementation.
2019-05-30 17:35:41 -07:00
Pat Gavlin 2324eaaa59
Add StackReference to the Python SDK (#2786)
This commit adds StackReference to the Python SDK, which uses
read_resource to read the remote state of a a Pulumi stack.
2019-05-30 14:12:37 -07:00
Sean Gillespie eda5de0f88 Implementation of Read for Python (#2752)
This commit implements read_resource functionality for Python in a
manner identical to the NodeJS implementation. If an "id" option is
passed to a resource via ResourceOptions on construction, that resource
will be read and not created.
2019-05-30 11:04:47 -07:00
Justin Van Patten 5df2b15320
Update CHANGELOG.md (#2778) 2019-05-28 17:31:41 -07:00
CyrusNajmabadi b84b3d487d
Revert "Simplify signature of pulumi.all (#2527)" (#2775)
This reverts commit b32892e0c1.
2019-05-28 10:54:56 -07:00
Joe Duffy bf75fe0662
Suppress JSON outputs in preview correctly (#2771)
If --suppress-outputs is passed to `pulumi preview --json`, we
should not emit the stack outputs. This change fixes pulumi/pulumi#2765.

Also adds a test case for this plus some variants of updates.
2019-05-25 12:10:38 +02:00
Matt Ellis 0574d8cd6f Attempt to download plugins before doing a refresh
Like `preview`, `update` and `destroy` we should ensure any plugins
that are listed in the snapshot are present.

Fixes #2669
2019-05-24 16:12:22 -07:00
stack72 19bf51d89a cmd/whoami: Move the changelog entry to unreleased
This was originally part of an older release but it has since been
moved to unreleased as it was only recently merged
2019-05-23 12:37:19 +02:00
Paul Stack fa691d8c40
cli/whoami: Addition of the currently connected backend to whoami (#2686)
cli/whoami: Addition of the currently connected backend to whoami
2019-05-22 15:57:52 +03:00
stack72 6ab390a815 cli/whoami: Addition of the currently connected backend to whoami
This is an attempt towards #2684

I am not sure if this is too simplistic for now OR we need to
consider if this will break anyones automation as they maybe using
the output of that command as plain text

Before:

```
▶ pulumi whoami
stack72
```

After:

```
▶ pulumi whoami
User: stack72
Backend URL: https://app.pulumi.com/stack72
```
2019-05-22 14:45:04 +02:00
Matt Ellis 61bff0c3a4 Do not parse version from resource providers
Until we can come up with a solution for #2753, just ignore the
version that comes in as part of a resource monitor RPC.
2019-05-21 19:20:18 -07:00
Fraser Waters d5b45ac5ad Move windows path fix changelog entry to 17.13 (#2749)
Was merged into the 17.11 changelog
2019-05-21 18:11:44 -07:00
Matt Ellis 31bd463264 Gracefully handle the case where secrets_provider is uninitalized
A customer reported an issue where operations would fail with the
following error:

```
error: could not deserialize deployment: unknown secrets provider type
```

The problem here was the customer's deployment had a
`secrets_provider` section which looked like the following:

```
"secrets_providers": {
    "type": ""
}
```

And so our code to try to construct a secrets manager from this thing
would fail, as our registry does not contain any information about a
provider with an empty type.

We do two things in this change:

1. When serializing a deployment, if there is no secrets manager,
don't even write the `secrets_provider` block. This helps for cases
where we are roundtripping deployments that did not have a provider
configured (i.e. they were older stacks that did not use secrets)

2. When deserializing, if we see an empty secrets provider like the
above, interpret it to mean "this deployment has no secrets". We set
up a decrypter such that if it ends up haiving secrets, we panic
eagerly (since this is a logical bug in our system somewhere).
2019-05-21 17:11:54 -07:00
CyrusNajmabadi b32892e0c1
Simplify signature of pulumi.all (#2527) 2019-05-21 13:13:23 -04:00
CyrusNajmabadi 2246a97c17
Always normalize paths to forward slashes to properly work with gocloud (#2747) 2019-05-20 14:46:00 -04:00
Matt Ellis 7ca9721d23 Apply colorization to the ---outputs--- line
We were not actually calling our colorization routines, which lead to
printing this very confusing text:

```
<{%reset%}>    --outputs:--<{%reset%}>
```

When running updates with `--diff` or when drilling into details of a
proposed operation, like a refresh.
2019-05-20 11:43:02 -07:00
Matt Ellis 4f693af023 Do not pass arguments as secrets to CheckConfig/Configure
Providers from plugins require that configuration value be
strings. This means if we are passing a secret string to a
provider (for example, trying to configure a kubernetes provider based
on some secret kubeconfig) we need to be careful to remove the
"secretness" before actually making the calls into the provider.

Failure to do this resulted in errors saying that the provider
configuration values had to be strings, and of course, the values
logically where, they were just marked as secret strings

Fixes #2741
2019-05-17 16:42:29 -07:00
Matt Ellis 2cd4409c0d Fix a panic during property diffing
We have to actually return the value we compute instead of just
dropping it on the floor and treating the underlying values as
primitive.

I ran into this during dogfooding, the added test case would
previously panic.
2019-05-15 16:20:25 -07:00
Matt Ellis 137b517c13 Prepare for release 2019-05-15 12:04:43 -07:00
Matt Ellis b4cca4a108 Update CHANGELOG.md 2019-05-15 09:33:02 -07:00
Matt Ellis e453c7db9e Eagerly validate secrets-provider option
Validate the value is well formed much earlier so you don't end up
seeing you've picked a bad value in the middle of trying to create
your new stack. Update the helptext to list currently supported
values.

Fixes #2727
2019-05-15 00:02:29 -07:00
Matt Ellis c91ddf996b Do not prompt for passphrase multiple times
The change does two things:

- Reorders some calls in the CLI to prevent trying to create a secrets
  manager twice (which would end up prompting twice).

- Adds a cache inside the passphrase secrets manager such that when
  decrypting a deployment, we can re-use the one created earlier in
  the update. This is sort of a hack, but is needed because otherwise
  we would fail to decrypt the deployment, meaning that if you had a
  secret value in your deployment *and* you were using local
  passphrase encryption *and* you had not set PULUMI_CONFIG_PASSPHRASE
  you would get an error asking you to do so.

Fixes #2729
2019-05-14 23:35:27 -07:00
Matt Ellis f6c25b7d2d Update Changelog for 0.17.11 2019-05-13 17:38:19 -07:00
Matt Ellis 5a5d320b7d Wordsmith CHNAGELOG.md 2019-05-10 17:12:35 -07:00
Matt Ellis 16932873a4 Update CHANGELOG.md 2019-05-10 17:07:52 -07:00
Cameron Stokes 1927b1c908 ~CHANGELOG: Change date format of recent release (#2716)
Be consistent with previous date formats (and use a format that's less ambiguous).
2019-05-10 16:55:09 -07:00
Erin Krengel b0327b96db
Ekrengel/fix azure urls (#2709)
* Fix GetVSCInfo for various Azure urls
2019-05-08 09:25:59 -07:00
Praneet Loke f4112a4ea1
Add detection of Azure Pipelines (#2705)
* Add a var for PRNumber. Add an environment metadata key for PR number.

* Move the detection of PULUMI_CI_SYSTEM into vars.DetectVars(). Set the PRNumber CI property based on respective env vars from each CI system.

* Add Azure Pipelines build variables.

* Add tests for DetectVars.

* Added changelog entry for Azure Pipelines.

* Capture the value of env var being modified for the ciutil unit test, and restore their values at the end of them.

* Simplify the DetectVars function by moving the Pulumi CI system code into the switch-case expression.

* Rename the Pulumi CI system to Generic CI. Include the GenericCI system in the test case for DetectVars.
2019-05-07 11:49:13 -07:00
Justin Van Patten fedfc9b6b4
pulumi update => pulumi up (#2702)
We changed the `pulumi update` command to be `pulumi up` a while back
(`update` is an alias of `up`). This change just makes it so we refer to
the actual command, `pulumi up`, instead of the older `pulumi update`.
2019-05-06 14:00:18 -07:00
CyrusNajmabadi b146733a26
Update changelog. (#2699) 2019-05-04 21:58:17 -07:00
Matt Ellis 2efde733b2
Merge pull request #2694 from bigkraig/contrib
Minor contributing doc improvement
2019-05-02 16:53:35 -07:00
CyrusNajmabadi daca809d09
Fix local file:// stacks on Windows. (#2696) 2019-05-02 16:52:00 -07:00
Matt Ellis 769cd7ce07
Merge pull request #2615 from pulumi/ellismg/scout-node-12
Support Node 12
2019-05-02 10:25:00 -07:00
Kraig Amador 7dac095d98
Updated CHANGELOG 2019-05-02 10:20:48 -07:00
Luke Hoban 3062ba7c4f
Move CHANGELOG entry for pulumi preview --json (#2690)
This was first released in `0.17.9` not `0.17.8`.
2019-05-01 12:39:18 -07:00
Matt Ellis de78ddb4b6 Update CHANGELOG.md 2019-05-01 10:05:33 -07:00
Justin Van Patten cde23669e2
pulumi new: Suppress npm warnings (#2685)
Right now, when we run `npm install` as part of `pulumi new`, the
following warnings are emitted:

```
node-pre-gyp WARN Using needle for node-pre-gyp https download

...

npm WARN aws-typescript@ No description
npm WARN aws-typescript@ No repository field.
npm WARN aws-typescript@ No license field.
```

We can suppress these warnings by specifying `--loglevel=error` to the
`npm install` command.
2019-05-01 09:17:53 -07:00
Justin Van Patten 55b233fbdc
Update CHANGELOG for 0.17.9 (#2687) 2019-04-30 11:01:58 -07:00
Justin Van Patten 95d51efe6a
pulumi new: Ensure the stack is selected (#2683)
Ensure the stack is selected when using `pulumi new` with an existing
stack (e.g. a stack created on app.pulumi.com).
2019-04-29 13:32:15 -07:00
Fraser Waters 6a2c9ae4f8 Update CHANGELOG.md 2019-04-27 21:47:49 +01:00
joeduffy 250bcb9751 Add a --json flag to the preview command
This change adds a --json flag to the preview command, enabling
basic JSON serialization of preview plans. This effectively flattens
the engine event stream into a preview structure that contains a list
of steps, diagnostics, and summary information. Each step contains
the deep serialization of resource state, in addition to metadata about
the step, such as what kind of operation it entails.

This is a partial implementation of pulumi/pulumi#2390. In particular,
we only support --json on the `preview` command itself, and not `up`,
meaning that it isn't possible to serialize the result of an actual
deployment yet (thereby limiting what you can do with outputs, etc).
2019-04-25 17:36:31 -07:00
PLACE 70bc0436ed Add support for state in cloud object storage (S3, GCS, Azure) (#2455) 2019-04-24 20:55:39 -07:00
Mike Metral 24f08ddb0f Update CHANGELOG for 0.17.8 2019-04-23 13:21:56 -07:00
Luke Hoban 0550f71a35
Add an ignoreChanges resource option (#2657)
Fixes #2277.

Adds a new ignoreChanges resource option that allows specifying a list of property names whose values will be ignored during updates. The property values will be used for Create, but will be ignored for purposes of updates, and as a result also cannot trigger replacements.

This is a feature of the Pulumi engine, not of the resource providers, so no new logic is needed in providers to support this feature. Instead, the engine simply replaces the values of input properties in the goal state with old inputs for properties marked as ignoreChanges.

Currently, only top level properties may be specified in ignoreChanges. In the future, this could be extended to support paths to nested properties (including into array elements) with a JSONPath/JMESPath syntax.
2019-04-22 13:54:48 -07:00
Joe Duffy 3b93199f7a Use Outputs instead of merged Inputs+Outputs (#2659)
Fixes #2650.

We have historically relied on merging inputs and outputs in several places in the engine. This used to be necessary, as discussed in #2650 (comment), but our core engine model has moved away from depending on this. However, we still have a couple places we do this merge, and those places have triggered several severe issues recently in subtle cases.

We believe that this merging should no longer be needed for a correct interpretation of the current engine model, and indeed that doing the merge actively violates the contract with providers. In this PR we remove the remaining places where this input + output merge was being done. In all three cases, we use just the Outputs, which for most providers will already include the same values as the inputs - but correctly as determined by the provider itself.
2019-04-22 13:52:36 -07:00
Matt Ellis f78c2f58f6 Update CHANGELOG.md for release 2019-04-17 08:19:52 -07:00
Joe Duffy 644d5dc916
Enable unit testing for Pulumi programs (#2638)
* Enable unit testing for Pulumi programs

This change enables rudimentary unit testing of your Pulumi programs, by introducing a `PULUMI_TEST_MODE` envvar that, when set, allows programs to run without a CLI. That includes

* Just being able to import your Pulumi modules, and test ordinary functions -- which otherwise would have often accidentally triggered the "Not Running in a CLI" error message
* Being able to verify a subset of resource properties and shapes, with the caveat that outputs are not included, due to the fact that this is a perpetual "dry run" without any engine operations occurring

In principle, this also means you can attach a debugger and step through your code.

* Finish the unit testing features

This change

1) Incorporates CR feedback, namely requiring that test mode be
   explicitly enabled for any of this to work.

2) Implements Python support for the same capabilities.

3) Includes tests for both JavaScript and Python SDKs.

* Add a note on unit testing to the CHANGELOG

* Use Node 8 friendly assert API

* Embellish the CHANGELOG entry a bit
2019-04-16 22:20:01 -07:00
Levi Blackstone 2532ce0527
Update changelog for 0.17.6 release (#2639) 2019-04-11 11:53:17 -06:00
Alex Clemmer fac6944781 Warn instead of error when refresh'd resource is unhealthy
Fixes #2633.

Currently when a user runs `refresh` and a resource is in a state of
error, the `refresh` will fail and the resource state will not be
persisted. This can make it vastly harder to incrementally fix
infrastructure. The issue mentioned above explains more of the
historical context, as well as some specific failure modes.

This commit resolves this issue by causing refresh to *not* report an
error in this case, and instead to simply log a warning that the
`refresh` has recognized that the resource is in an unhealthy state
during state sync.
2019-04-10 16:43:33 -07:00
Matt Ellis 44292747d4 Collect logs from Google Cloud Functions
This change adds an operations provider for GCP. Right now, it can
just collect logs from google cloud functions, similar to `gcloud
functions logs read`
2019-04-08 09:44:05 -07:00
Joe Duffy cc7a657628
Make the Python SDK's Config.name argument optional (#2624)
This makes the Python SDK's Config constructor name argument
optional, similar to what we do in the Node.js SDK. If not supplied,
the current project name is used as the default.
2019-04-04 16:55:01 -07:00
CyrusNajmabadi 1f51ec00fc
Revert 'Simplify API for passing providers to a ComponentResource (#2602)' (#2606) 2019-03-28 18:31:03 -07:00
CyrusNajmabadi 7193622183
Simplify API for passing providers to a ComponentResource (#2602) 2019-03-28 15:04:07 -07:00
Matt Ellis 2ace3e7b0c Correctly handle FileArchives when the filename contains a dot
Our logic for how we handled `.tar.gz` archives meant that any other
type of file that had a dot in the filename would not be detected
correctly.

Fixes #2589
2019-03-28 13:26:07 -07:00
CyrusNajmabadi b1c9386926
Update changelog (#2594) 2019-03-26 16:53:37 -07:00
CyrusNajmabadi 992952b54f
Revert "Better present interpolated strings during preview. (#2564)" (#2593) 2019-03-26 16:36:43 -07:00
Matt Ellis ccd958777c Don't print error prefix when a confirmation prompt is declined
Use `result.Result` in more places, so when a confirmation prompt is
declined, we just return `result.Bail()` after printing a message
without the `error: ` prefix.

Fixes #2070
2019-03-26 15:17:18 -07:00
Sean Gillespie f34aef2f4d
Prepare CHANGELOG.md for release (#2591) 2019-03-26 14:45:09 -07:00
Sean Gillespie 4d227f7ed2
Load default providers deterministically (#2590)
* Load default providers deterministically

This commit adds a new algorithm for deriving a list of default
providers from the set of plugins reported from the language host and
from the snapshot. If the language host reports a set of plugins,
default providers are sourced directly from that set, otherwise default
providers are sourced from the full set of plugins, including ones from
the snapshot.

When multiple versions of the same provider are requested, the newest
version of that provider is always select as the default provider.

* Add CHANGELOG.md entry

* Skip the language host's plugins if it reports no resource plugins

* CR feedback

* CR: Log when skipping non resource plugin
2019-03-26 13:29:34 -07:00
Matt Ellis 3621c01f4b Fix spurious warnings when using dynamic provider
When using a dynamic provider, the following messages are printed,
because we did not update the dynamic provider when we added these new
RPCs:

```
Method handler checkConfig for /pulumirpc.ResourceProvider/CheckConfig expected but not provided
Method handler diffConfig for /pulumirpc.ResourceProvider/DiffConfig expected but not provided
```

Given that we don't do anything for the `configure` RPC today, we can
also just no-op these RPCs as well.
2019-03-26 12:04:15 -04:00
CyrusNajmabadi 906488c7d0
Fix issue where we could not serialize an arrow-function with a deconstructed parameter. (#2582) 2019-03-22 16:48:38 -07:00
CyrusNajmabadi be1c3eb05b
Add support for serializing bigints (#2581) 2019-03-22 15:33:37 -07:00
Matt Ellis a1bb16407d Add pulumi stack rename
`pulumi stack rename` allows you to change the name of an existing
stack. This operation is non-distructive, however it is possible that
the next update will show additional changes to resources, if the
pulumi program uses the value of `getStack()` as part of a resource
name.
2019-03-19 11:04:33 -07:00