2017-06-26 23:46:34 +02:00
|
|
|
// Copyright 2016-2017, Pulumi Corporation. All rights reserved.
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
|
|
|
package deploy
|
|
|
|
|
|
|
|
import (
|
2017-09-09 16:37:10 +02:00
|
|
|
"fmt"
|
2017-08-30 03:24:12 +02:00
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
"github.com/golang/glog"
|
2017-08-30 03:24:12 +02:00
|
|
|
"github.com/pkg/errors"
|
|
|
|
"golang.org/x/net/context"
|
|
|
|
"google.golang.org/grpc"
|
|
|
|
|
2017-09-22 04:18:21 +02:00
|
|
|
"github.com/pulumi/pulumi/pkg/pack"
|
|
|
|
"github.com/pulumi/pulumi/pkg/resource"
|
|
|
|
"github.com/pulumi/pulumi/pkg/resource/plugin"
|
|
|
|
"github.com/pulumi/pulumi/pkg/tokens"
|
|
|
|
"github.com/pulumi/pulumi/pkg/util/contract"
|
|
|
|
"github.com/pulumi/pulumi/pkg/util/rpcutil"
|
|
|
|
lumirpc "github.com/pulumi/pulumi/sdk/proto/go"
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
)
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// EvalRunInfo provides information required to execute and deploy resources within a package.
|
|
|
|
type EvalRunInfo struct {
|
2017-08-31 23:31:33 +02:00
|
|
|
Pkg *pack.Package `json:"pkg"` // the package metadata.
|
|
|
|
Pwd string `json:"pwd"` // the package's working directory.
|
|
|
|
Program string `json:"program"` // the path to the program we are executing.
|
|
|
|
Args []string `json:"args,omitempty"` // any arguments to pass to the package.
|
|
|
|
Config map[tokens.ModuleMember]string `json:"config,omitempty"` // any configuration to pass to the program.
|
2017-08-30 03:24:12 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// NewEvalSource returns a planning source that fetches resources by evaluating a package with a set of args and
|
|
|
|
// a confgiuration map. This evaluation is performed using the given plugin context and may optionally use the
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
// given plugin host (or the default, if this is nil). Note that closing the eval source also closes the host.
|
2017-06-21 19:31:06 +02:00
|
|
|
//
|
|
|
|
// If destroy is true, then all of the usual initialization will take place, but the state will be presented to the
|
|
|
|
// planning engine as if no new resources exist. This will cause it to forcibly remove them.
|
2017-08-30 03:24:12 +02:00
|
|
|
func NewEvalSource(plugctx *plugin.Context, runinfo *EvalRunInfo, destroy bool, dryRun bool) Source {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
return &evalSource{
|
2017-06-21 19:31:06 +02:00
|
|
|
plugctx: plugctx,
|
2017-08-30 03:24:12 +02:00
|
|
|
runinfo: runinfo,
|
2017-06-21 19:31:06 +02:00
|
|
|
destroy: destroy,
|
2017-08-30 03:24:12 +02:00
|
|
|
dryRun: dryRun,
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
type evalSource struct {
|
2017-08-30 03:24:12 +02:00
|
|
|
plugctx *plugin.Context // the plugin context.
|
|
|
|
runinfo *EvalRunInfo // the directives to use when running the program.
|
|
|
|
destroy bool // true if this source will trigger total destruction.
|
|
|
|
dryRun bool // true if this is a dry-run operation only.
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
func (src *evalSource) Close() error {
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
func (src *evalSource) Pkg() tokens.PackageName {
|
|
|
|
return src.runinfo.Pkg.Name
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
func (src *evalSource) Info() interface{} {
|
|
|
|
return src.runinfo
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// Iterate will spawn an evaluator coroutine and prepare to interact with it on subsequent calls to Next.
|
2017-09-16 01:38:52 +02:00
|
|
|
func (src *evalSource) Iterate(opts Options) (SourceIterator, error) {
|
2017-08-30 03:24:12 +02:00
|
|
|
// First, fire up a resource monitor that will watch for and record resource creation.
|
2017-09-04 18:25:31 +02:00
|
|
|
reschan := make(chan *evalSourceGoal)
|
2017-09-20 02:23:10 +02:00
|
|
|
mon, err := newResourceMonitor(src, reschan)
|
2017-08-30 03:24:12 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, errors.Wrap(err, "failed to start resource monitor")
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Create a new iterator with appropriate channels, and gear up to go!
|
|
|
|
iter := &evalSourceIterator{
|
2017-09-09 00:11:09 +02:00
|
|
|
mon: mon,
|
|
|
|
src: src,
|
|
|
|
finchan: make(chan error),
|
|
|
|
reschan: reschan,
|
2017-06-22 07:02:57 +02:00
|
|
|
}
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Now invoke Run in a goroutine. All subsequent resource creation events will come in over the gRPC channel,
|
|
|
|
// and we will pump them through the channel. If the Run call ultimately fails, we need to propagate the error.
|
2017-09-16 01:38:52 +02:00
|
|
|
iter.forkRun(opts)
|
2017-08-30 03:24:12 +02:00
|
|
|
|
|
|
|
// Finally, return the fresh iterator that the caller can use to take things from here.
|
|
|
|
return iter, nil
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
type evalSourceIterator struct {
|
2017-09-09 00:11:09 +02:00
|
|
|
mon *resmon // the resource monitor, per iterator.
|
|
|
|
src *evalSource // the owning eval source object.
|
|
|
|
finchan chan error // the channel that communicates completion.
|
|
|
|
reschan chan *evalSourceGoal // the channel that contains resource elements.
|
|
|
|
done bool // set to true when the evaluation is done.
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
func (iter *evalSourceIterator) Close() error {
|
2017-09-09 00:11:09 +02:00
|
|
|
// Cancel the monitor and reclaim any associated resources.
|
|
|
|
return iter.mon.Cancel()
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
func (iter *evalSourceIterator) Next() (SourceGoal, error) {
|
|
|
|
// If we are done, quit.
|
|
|
|
if iter.done {
|
|
|
|
return nil, nil
|
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
2017-06-21 19:31:06 +02:00
|
|
|
// If we are destroying, we simply return nothing.
|
|
|
|
if iter.src.destroy {
|
2017-08-30 03:24:12 +02:00
|
|
|
return nil, nil
|
2017-06-21 19:31:06 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Await the program to compute some more state and then inspect what it has to say.
|
|
|
|
select {
|
|
|
|
case err := <-iter.finchan:
|
|
|
|
// If we are finished, we can safely exit. The contract with the language provider is that this implies
|
|
|
|
// that the language runtime has exited and so calling Close on the plugin is fine.
|
|
|
|
iter.done = true
|
|
|
|
if err != nil {
|
|
|
|
glog.V(5).Infof("EvalSourceIterator ended with an error: %v", err)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
2017-08-30 03:24:12 +02:00
|
|
|
return nil, err
|
|
|
|
case res := <-iter.reschan:
|
|
|
|
contract.Assert(res != nil)
|
|
|
|
goal := res.Resource()
|
|
|
|
glog.V(5).Infof("EvalSourceIterator produced a new object: t=%v,name=%v,#props=%v",
|
|
|
|
goal.Type, goal.Name, len(goal.Properties))
|
|
|
|
return res, nil
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// forkRun performs the evaluation from a distinct goroutine. This function blocks until it's our turn to go.
|
2017-09-16 01:38:52 +02:00
|
|
|
func (iter *evalSourceIterator) forkRun(opts Options) {
|
2017-09-09 00:11:09 +02:00
|
|
|
// If we are destroying, no need to perform any evaluation beyond the config initialization.
|
|
|
|
if !iter.src.destroy {
|
2017-08-30 03:24:12 +02:00
|
|
|
// Fire up the goroutine to make the RPC invocation against the language runtime. As this executes, calls
|
|
|
|
// to queue things up in the resource channel will occur, and we will serve them concurrently.
|
|
|
|
// FIXME: we need to ensure that out of order calls won't deadlock us. In particular, we need to ensure: 1)
|
|
|
|
// gRPC won't block the dispatching of calls, and 2) that the channel's fixed size won't cause troubles.
|
|
|
|
go func() {
|
2017-09-09 00:11:09 +02:00
|
|
|
// Next, launch the language plugin.
|
|
|
|
// IDEA: cache these so we reuse the same language plugin instance; if we do this, monitors must be per-run.
|
|
|
|
rt := iter.src.runinfo.Pkg.Runtime
|
|
|
|
langhost, err := iter.src.plugctx.Host.LanguageRuntime(rt, iter.mon.Address())
|
|
|
|
if err != nil {
|
|
|
|
err = errors.Wrapf(err, "failed to launch language host for '%v'", rt)
|
|
|
|
} else if langhost == nil {
|
|
|
|
err = errors.Errorf("could not load language plugin for '%v' from $PATH", rt)
|
2017-06-17 00:35:39 +02:00
|
|
|
} else {
|
2017-09-09 00:11:09 +02:00
|
|
|
// Make sure to clean up before exiting.
|
|
|
|
defer contract.IgnoreClose(langhost)
|
|
|
|
|
|
|
|
// Now run the actual program.
|
|
|
|
var progerr string
|
|
|
|
progerr, err = langhost.Run(plugin.RunInfo{
|
2017-09-17 17:10:46 +02:00
|
|
|
Pwd: iter.src.runinfo.Pwd,
|
|
|
|
Program: iter.src.runinfo.Program,
|
|
|
|
Args: iter.src.runinfo.Args,
|
|
|
|
Config: iter.src.runinfo.Config,
|
|
|
|
DryRun: iter.src.dryRun,
|
|
|
|
Parallel: opts.Parallel,
|
2017-09-09 00:11:09 +02:00
|
|
|
})
|
|
|
|
if err == nil && progerr != "" {
|
|
|
|
// If the program had an unhandled error; propagate it to the caller.
|
|
|
|
err = errors.Errorf("an unhandled error occurred: %v", progerr)
|
|
|
|
}
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
2017-09-09 00:11:09 +02:00
|
|
|
|
|
|
|
// Communicate the error, if it exists, or nil if the program exited cleanly.
|
|
|
|
iter.finchan <- err
|
2017-08-30 03:24:12 +02:00
|
|
|
}()
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
2017-08-30 03:24:12 +02:00
|
|
|
}
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// resmon implements the lumirpc.ResourceMonitor interface and acts as the gateway between a language runtime's
|
|
|
|
// evaluation of a program and the internal resource planning and deployment logic.
|
|
|
|
type resmon struct {
|
2017-09-20 02:23:10 +02:00
|
|
|
src *evalSource // the evaluation source.
|
2017-09-04 18:25:31 +02:00
|
|
|
reschan chan *evalSourceGoal // the channel to send resources to.
|
2017-09-09 16:37:10 +02:00
|
|
|
addr string // the address the host is listening on.
|
2017-09-04 18:25:31 +02:00
|
|
|
cancel chan bool // a channel that can cancel the server.
|
|
|
|
done chan error // a channel that resolves when the server completes.
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// newResourceMonitor creates a new resource monitor RPC server.
|
2017-09-20 02:23:10 +02:00
|
|
|
func newResourceMonitor(src *evalSource, reschan chan *evalSourceGoal) (*resmon, error) {
|
2017-08-30 03:24:12 +02:00
|
|
|
// New up an engine RPC server.
|
|
|
|
resmon := &resmon{
|
2017-09-20 02:23:10 +02:00
|
|
|
src: src,
|
2017-09-04 18:25:31 +02:00
|
|
|
reschan: reschan,
|
|
|
|
cancel: make(chan bool),
|
2017-08-30 03:24:12 +02:00
|
|
|
}
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Fire up a gRPC server and start listening for incomings.
|
|
|
|
port, done, err := rpcutil.Serve(0, resmon.cancel, []func(*grpc.Server) error{
|
|
|
|
func(srv *grpc.Server) error {
|
|
|
|
lumirpc.RegisterResourceMonitorServer(srv, resmon)
|
|
|
|
return nil
|
|
|
|
},
|
|
|
|
})
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
2017-06-21 19:31:06 +02:00
|
|
|
|
2017-09-21 19:56:45 +02:00
|
|
|
resmon.addr = fmt.Sprintf("127.0.0.1:%d", port)
|
2017-08-30 03:24:12 +02:00
|
|
|
resmon.done = done
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
return resmon, nil
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Address returns the address at which the monitor's RPC server may be reached.
|
|
|
|
func (rm *resmon) Address() string {
|
2017-09-09 16:37:10 +02:00
|
|
|
return rm.addr
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Cancel signals that the engine should be terminated, awaits its termination, and returns any errors that result.
|
|
|
|
func (rm *resmon) Cancel() error {
|
|
|
|
rm.cancel <- true
|
|
|
|
return <-rm.done
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-09-20 02:23:10 +02:00
|
|
|
// Invoke performs an invocation of a member located in a resource provider.
|
|
|
|
func (rm *resmon) Invoke(ctx context.Context, req *lumirpc.InvokeRequest) (*lumirpc.InvokeResponse, error) {
|
|
|
|
// Fetch the token and load up the resource provider.
|
|
|
|
tok := tokens.ModuleMember(req.GetTok())
|
|
|
|
prov, err := rm.src.plugctx.Host.Provider(tok.Package())
|
|
|
|
if err != nil {
|
|
|
|
return nil, errors.Wrapf(err, "failed to load resource provider for %v", tok)
|
|
|
|
}
|
|
|
|
|
|
|
|
// Now unpack all of the arguments and prepare to perform the invocation.
|
|
|
|
args, err := plugin.UnmarshalProperties(
|
|
|
|
req.GetArgs(), plugin.MarshalOptions{AllowUnknowns: true})
|
|
|
|
if err != nil {
|
|
|
|
return nil, errors.Wrapf(err, "failed to unmarshal %v args", tok)
|
|
|
|
}
|
|
|
|
|
|
|
|
// Do the invoke and then return the arguments.
|
|
|
|
glog.V(5).Info("ResourceMonitor.Invoke received: tok=%v #args=%v", tok, len(args))
|
|
|
|
ret, failures, err := prov.Invoke(tok, args)
|
|
|
|
if err != nil {
|
|
|
|
return nil, errors.Wrapf(err, "invocation of %v returned an error", tok)
|
|
|
|
}
|
|
|
|
mret, err := plugin.MarshalProperties(ret, plugin.MarshalOptions{AllowUnknowns: true})
|
|
|
|
if err != nil {
|
|
|
|
return nil, errors.Wrapf(err, "failed to marshal %v return", tok)
|
|
|
|
}
|
|
|
|
var chkfails []*lumirpc.CheckFailure
|
|
|
|
for _, failure := range failures {
|
|
|
|
chkfails = append(chkfails, &lumirpc.CheckFailure{
|
|
|
|
Property: string(failure.Property),
|
|
|
|
Reason: failure.Reason,
|
|
|
|
})
|
|
|
|
}
|
|
|
|
return &lumirpc.InvokeResponse{Return: mret, Failures: chkfails}, nil
|
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// NewResource is invoked by a language process when a new resource has been allocated.
|
|
|
|
func (rm *resmon) NewResource(ctx context.Context,
|
|
|
|
req *lumirpc.NewResourceRequest) (*lumirpc.NewResourceResponse, error) {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Communicate the type, name, and object information to the iterator that is awaiting us.
|
2017-09-15 01:40:44 +02:00
|
|
|
props, err := plugin.UnmarshalProperties(
|
|
|
|
req.GetObject(), plugin.MarshalOptions{AllowUnknowns: true})
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2017-08-30 03:24:12 +02:00
|
|
|
goal := &evalSourceGoal{
|
|
|
|
goal: resource.NewGoal(
|
|
|
|
tokens.Type(req.GetType()),
|
|
|
|
tokens.QName(req.GetName()),
|
2017-09-15 01:40:44 +02:00
|
|
|
props,
|
2017-08-30 03:24:12 +02:00
|
|
|
),
|
2017-09-05 19:01:00 +02:00
|
|
|
done: make(chan *evalState),
|
2017-06-15 00:52:36 +02:00
|
|
|
}
|
2017-09-04 18:25:31 +02:00
|
|
|
glog.V(5).Infof("ResourceMonitor.NewResource received: t=%v, name=%v, #props=%v",
|
|
|
|
goal.goal.Type, goal.goal.Name, len(goal.goal.Properties))
|
|
|
|
rm.reschan <- goal
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Now block waiting for the operation to finish.
|
|
|
|
// FIXME: we probably need some way to cancel this in case of catastrophe.
|
2017-09-05 19:01:00 +02:00
|
|
|
done := <-goal.done
|
|
|
|
state := done.State
|
|
|
|
outprops := state.Synthesized()
|
|
|
|
stable := done.Stable
|
Add a notion of stable properties
This change adds the capability for a resource provider to indicate
that, where an action carried out in response to a diff, a certain set
of properties would be "stable"; that is to say, they are guaranteed
not to change. As a result, properties may be resolved to their final
values during previewing, avoiding erroneous cascading impacts.
This avoids the ever-annoying situation I keep running into when demoing:
when adding or removing an ingress rule to a security group, we ripple
the impact through the instance, and claim it must be replaced, because
that instance depends on the security group via its name. Well, the name
is a great example of a stable property, in that it will never change, and
so this is truly unfortunate and always adds uncertainty into the demos.
Particularly since the actual update doesn't need to perform replacements.
This resolves pulumi/pulumi#330.
2017-10-04 14:22:21 +02:00
|
|
|
var stables []string
|
|
|
|
for _, sta := range done.Stables {
|
|
|
|
stables = append(stables, string(sta))
|
|
|
|
}
|
|
|
|
glog.V(5).Infof(
|
|
|
|
"ResourceMonitor.NewResource operation finished: t=%v, urn=%v (name=%v), stable=%v, #stables=%v #outs=%v",
|
|
|
|
state.Type, state.URN, goal.goal.Name, stable, len(stables), len(outprops))
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Finally, unpack the response into properties that we can return to the language runtime. This mostly includes
|
|
|
|
// an ID, URN, and defaults and output properties that will all be blitted back onto the runtime object.
|
2017-09-15 01:40:44 +02:00
|
|
|
outs, err := plugin.MarshalProperties(outprops, plugin.MarshalOptions{AllowUnknowns: true})
|
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2017-08-30 03:24:12 +02:00
|
|
|
return &lumirpc.NewResourceResponse{
|
Add a notion of stable properties
This change adds the capability for a resource provider to indicate
that, where an action carried out in response to a diff, a certain set
of properties would be "stable"; that is to say, they are guaranteed
not to change. As a result, properties may be resolved to their final
values during previewing, avoiding erroneous cascading impacts.
This avoids the ever-annoying situation I keep running into when demoing:
when adding or removing an ingress rule to a security group, we ripple
the impact through the instance, and claim it must be replaced, because
that instance depends on the security group via its name. Well, the name
is a great example of a stable property, in that it will never change, and
so this is truly unfortunate and always adds uncertainty into the demos.
Particularly since the actual update doesn't need to perform replacements.
This resolves pulumi/pulumi#330.
2017-10-04 14:22:21 +02:00
|
|
|
Id: string(state.ID),
|
|
|
|
Urn: string(state.URN),
|
|
|
|
Object: outs,
|
|
|
|
Stable: stable,
|
|
|
|
Stables: stables,
|
2017-08-30 03:24:12 +02:00
|
|
|
}, nil
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-09-05 19:01:00 +02:00
|
|
|
type evalState struct {
|
Add a notion of stable properties
This change adds the capability for a resource provider to indicate
that, where an action carried out in response to a diff, a certain set
of properties would be "stable"; that is to say, they are guaranteed
not to change. As a result, properties may be resolved to their final
values during previewing, avoiding erroneous cascading impacts.
This avoids the ever-annoying situation I keep running into when demoing:
when adding or removing an ingress rule to a security group, we ripple
the impact through the instance, and claim it must be replaced, because
that instance depends on the security group via its name. Well, the name
is a great example of a stable property, in that it will never change, and
so this is truly unfortunate and always adds uncertainty into the demos.
Particularly since the actual update doesn't need to perform replacements.
This resolves pulumi/pulumi#330.
2017-10-04 14:22:21 +02:00
|
|
|
State *resource.State // the resource state.
|
|
|
|
Stable bool // if true, the resource state is stable and may be trusted.
|
|
|
|
Stables []resource.PropertyKey // an optional list of specific resource properties that are stable.
|
2017-09-05 19:01:00 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
type evalSourceGoal struct {
|
2017-09-05 19:01:00 +02:00
|
|
|
goal *resource.Goal // the resource goal state produced by the iterator.
|
|
|
|
done chan *evalState // the channel to communicate with after the resource state is available.
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
func (g *evalSourceGoal) Resource() *resource.Goal {
|
|
|
|
return g.goal
|
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
Add a notion of stable properties
This change adds the capability for a resource provider to indicate
that, where an action carried out in response to a diff, a certain set
of properties would be "stable"; that is to say, they are guaranteed
not to change. As a result, properties may be resolved to their final
values during previewing, avoiding erroneous cascading impacts.
This avoids the ever-annoying situation I keep running into when demoing:
when adding or removing an ingress rule to a security group, we ripple
the impact through the instance, and claim it must be replaced, because
that instance depends on the security group via its name. Well, the name
is a great example of a stable property, in that it will never change, and
so this is truly unfortunate and always adds uncertainty into the demos.
Particularly since the actual update doesn't need to perform replacements.
This resolves pulumi/pulumi#330.
2017-10-04 14:22:21 +02:00
|
|
|
func (g *evalSourceGoal) Done(state *resource.State, stable bool, stables []resource.PropertyKey) {
|
2017-08-30 03:24:12 +02:00
|
|
|
// Communicate the resulting state back to the RPC thread, which is parked awaiting our reply.
|
Add a notion of stable properties
This change adds the capability for a resource provider to indicate
that, where an action carried out in response to a diff, a certain set
of properties would be "stable"; that is to say, they are guaranteed
not to change. As a result, properties may be resolved to their final
values during previewing, avoiding erroneous cascading impacts.
This avoids the ever-annoying situation I keep running into when demoing:
when adding or removing an ingress rule to a security group, we ripple
the impact through the instance, and claim it must be replaced, because
that instance depends on the security group via its name. Well, the name
is a great example of a stable property, in that it will never change, and
so this is truly unfortunate and always adds uncertainty into the demos.
Particularly since the actual update doesn't need to perform replacements.
This resolves pulumi/pulumi#330.
2017-10-04 14:22:21 +02:00
|
|
|
g.done <- &evalState{
|
|
|
|
State: state,
|
|
|
|
Stable: stable,
|
|
|
|
Stables: stables,
|
|
|
|
}
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}
|