2017-06-26 23:46:34 +02:00
|
|
|
// Copyright 2016-2017, Pulumi Corporation. All rights reserved.
|
2017-06-10 03:34:37 +02:00
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
package deploy
|
2017-06-10 03:34:37 +02:00
|
|
|
|
|
|
|
import (
|
|
|
|
"github.com/golang/glog"
|
|
|
|
goerr "github.com/pkg/errors"
|
|
|
|
|
2017-08-02 18:25:22 +02:00
|
|
|
"github.com/pulumi/pulumi-fabric/pkg/compiler/errors"
|
|
|
|
"github.com/pulumi/pulumi-fabric/pkg/resource"
|
|
|
|
"github.com/pulumi/pulumi-fabric/pkg/resource/plugin"
|
2017-08-30 03:24:12 +02:00
|
|
|
"github.com/pulumi/pulumi-fabric/pkg/tokens"
|
2017-08-02 18:25:22 +02:00
|
|
|
"github.com/pulumi/pulumi-fabric/pkg/util/contract"
|
2017-06-10 03:34:37 +02:00
|
|
|
)
|
|
|
|
|
2017-08-02 18:25:22 +02:00
|
|
|
// TODO[pulumi/pulumi-fabric#106]: parallelism.
|
2017-06-10 03:34:37 +02:00
|
|
|
|
|
|
|
// Apply performs all steps in the plan, calling out to the progress reporting functions as desired. It returns four
|
|
|
|
// things: the resulting Snapshot, no matter whether an error occurs or not; an error, if something went wrong; the step
|
|
|
|
// that failed, if the error is non-nil; and finally the state of the resource modified in the failing step.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
func (p *Plan) Apply(prog Progress) (PlanSummary, Step, resource.Status, error) {
|
2017-06-10 03:34:37 +02:00
|
|
|
// Fetch a plan iterator and keep walking it until we are done.
|
|
|
|
iter, err := p.Iterate()
|
|
|
|
if err != nil {
|
|
|
|
return nil, nil, resource.StatusOK, err
|
|
|
|
}
|
2017-06-21 19:31:06 +02:00
|
|
|
|
2017-06-10 03:34:37 +02:00
|
|
|
n := 1
|
|
|
|
step, err := iter.Next()
|
|
|
|
if err != nil {
|
2017-06-21 19:31:06 +02:00
|
|
|
_ = iter.Close() // ignore close errors; the Next error trumps
|
2017-06-10 03:34:37 +02:00
|
|
|
return nil, nil, resource.StatusOK, err
|
|
|
|
}
|
2017-06-21 19:31:06 +02:00
|
|
|
|
2017-06-10 03:34:37 +02:00
|
|
|
for step != nil {
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
// Do the pre-step.
|
|
|
|
rst := resource.StatusOK
|
|
|
|
err := step.Pre()
|
|
|
|
|
2017-06-10 03:34:37 +02:00
|
|
|
// Perform pre-application progress reporting.
|
|
|
|
if prog != nil {
|
|
|
|
prog.Before(step)
|
|
|
|
}
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
if err == nil {
|
|
|
|
rst, err = step.Apply()
|
|
|
|
}
|
2017-06-10 03:34:37 +02:00
|
|
|
|
|
|
|
// Perform post-application progress reporting.
|
|
|
|
if prog != nil {
|
|
|
|
prog.After(step, rst, err)
|
|
|
|
}
|
|
|
|
|
|
|
|
// If an error occurred, exit early.
|
|
|
|
if err != nil {
|
|
|
|
glog.V(7).Infof("Plan step #%v failed [%v]: %v", n, step.Op(), err)
|
2017-06-21 19:31:06 +02:00
|
|
|
_ = iter.Close() // ignore close errors; the Apply error trumps
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
return iter, step, rst, err
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
glog.V(7).Infof("Plan step #%v succeeded [%v]", n, step.Op())
|
|
|
|
step, err = iter.Next()
|
2017-06-21 22:24:35 +02:00
|
|
|
if err != nil {
|
|
|
|
glog.V(7).Infof("Advancing to plan step #%v failed: %v", n+1, err)
|
|
|
|
_ = iter.Close() // ignore close errors; the Apply error trumps
|
|
|
|
return iter, step, resource.StatusOK, err
|
|
|
|
}
|
2017-06-10 03:34:37 +02:00
|
|
|
n++
|
|
|
|
}
|
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
// Finally, return a summary and the resulting plan information.
|
2017-06-21 19:31:06 +02:00
|
|
|
return iter, nil, resource.StatusOK, iter.Close()
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// Iterate initializes and returns an iterator that can be used to step through a plan's individual steps.
|
|
|
|
func (p *Plan) Iterate() (*PlanIterator, error) {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
// Ask the source for its iterator.
|
2017-08-30 03:24:12 +02:00
|
|
|
src, err := p.source.Iterate()
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
|
|
|
// Create an iterator that can be used to perform the planning process.
|
|
|
|
return &PlanIterator{
|
2017-06-11 17:09:20 +02:00
|
|
|
p: p,
|
|
|
|
src: src,
|
2017-06-27 22:04:06 +02:00
|
|
|
urns: make(map[resource.URN]bool),
|
2017-06-11 17:09:20 +02:00
|
|
|
creates: make(map[resource.URN]bool),
|
|
|
|
updates: make(map[resource.URN]bool),
|
|
|
|
replaces: make(map[resource.URN]bool),
|
|
|
|
deletes: make(map[resource.URN]bool),
|
|
|
|
sames: make(map[resource.URN]bool),
|
2017-06-12 16:16:08 +02:00
|
|
|
dones: make(map[*resource.State]bool),
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
}, nil
|
|
|
|
}
|
2017-06-10 03:34:37 +02:00
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
// PlanSummary is an interface for summarizing the progress of a plan.
|
|
|
|
type PlanSummary interface {
|
|
|
|
Steps() int
|
|
|
|
Creates() map[resource.URN]bool
|
|
|
|
Updates() map[resource.URN]bool
|
|
|
|
Replaces() map[resource.URN]bool
|
|
|
|
Deletes() map[resource.URN]bool
|
|
|
|
Sames() map[resource.URN]bool
|
|
|
|
Resources() []*resource.State
|
|
|
|
Snap() *Snapshot
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// PlanIterator can be used to step through and/or execute a plan's proposed actions.
|
|
|
|
type PlanIterator struct {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
p *Plan // the plan to which this iterator belongs.
|
|
|
|
src SourceIterator // the iterator that fetches source resources.
|
2017-06-10 03:34:37 +02:00
|
|
|
|
2017-06-27 22:04:06 +02:00
|
|
|
urns map[resource.URN]bool // URNs discovered.
|
2017-06-10 03:34:37 +02:00
|
|
|
creates map[resource.URN]bool // URNs discovered to be created.
|
|
|
|
updates map[resource.URN]bool // URNs discovered to be updated.
|
|
|
|
replaces map[resource.URN]bool // URNs discovered to be replaced.
|
|
|
|
deletes map[resource.URN]bool // URNs discovered to be deleted.
|
|
|
|
sames map[resource.URN]bool // URNs discovered to be the same.
|
|
|
|
|
2017-08-06 19:05:51 +02:00
|
|
|
stepqueue []Step // a queue of steps to drain.
|
2017-06-12 16:16:08 +02:00
|
|
|
delqueue []*resource.State // a queue of deletes left to perform.
|
|
|
|
resources []*resource.State // the resulting ordered resource states.
|
|
|
|
dones map[*resource.State]bool // true for each old state we're done with.
|
2017-06-10 03:34:37 +02:00
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
srcdone bool // true if the source interpreter has been run to completion.
|
|
|
|
done bool // true if the planning and associated iteration has finished.
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
func (iter *PlanIterator) Plan() *Plan { return iter.p }
|
|
|
|
func (iter *PlanIterator) Steps() int {
|
|
|
|
return len(iter.creates) + len(iter.updates) + len(iter.replaces) + len(iter.deletes)
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
func (iter *PlanIterator) Creates() map[resource.URN]bool { return iter.creates }
|
|
|
|
func (iter *PlanIterator) Updates() map[resource.URN]bool { return iter.updates }
|
|
|
|
func (iter *PlanIterator) Replaces() map[resource.URN]bool { return iter.replaces }
|
|
|
|
func (iter *PlanIterator) Deletes() map[resource.URN]bool { return iter.deletes }
|
|
|
|
func (iter *PlanIterator) Sames() map[resource.URN]bool { return iter.sames }
|
|
|
|
func (iter *PlanIterator) Resources() []*resource.State { return iter.resources }
|
2017-06-12 16:16:08 +02:00
|
|
|
func (iter *PlanIterator) Dones() map[*resource.State]bool { return iter.dones }
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
func (iter *PlanIterator) Done() bool { return iter.done }
|
2017-06-10 03:34:37 +02:00
|
|
|
|
2017-06-21 19:31:06 +02:00
|
|
|
// Close terminates the iteration of this plan.
|
|
|
|
func (iter *PlanIterator) Close() error {
|
|
|
|
return iter.src.Close()
|
|
|
|
}
|
|
|
|
|
2017-06-10 03:34:37 +02:00
|
|
|
// Next advances the plan by a single step, and returns the next step to be performed. In doing so, it will perform
|
|
|
|
// evaluation of the program as much as necessary to determine the next step. If there is no further action to be
|
|
|
|
// taken, Next will return a nil step pointer.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
func (iter *PlanIterator) Next() (Step, error) {
|
2017-06-10 03:34:37 +02:00
|
|
|
for !iter.done {
|
2017-08-06 19:05:51 +02:00
|
|
|
if len(iter.stepqueue) > 0 {
|
|
|
|
step := iter.stepqueue[0]
|
|
|
|
iter.stepqueue = iter.stepqueue[1:]
|
|
|
|
return step, nil
|
2017-08-30 03:24:12 +02:00
|
|
|
} else if !iter.srcdone {
|
|
|
|
goal, err := iter.src.Next()
|
2017-06-10 03:34:37 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
2017-08-30 03:24:12 +02:00
|
|
|
} else if goal != nil {
|
|
|
|
steps, err := iter.nextResourceSteps(goal)
|
2017-06-10 03:34:37 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2017-08-06 19:05:51 +02:00
|
|
|
contract.Assert(len(steps) > 0)
|
|
|
|
if len(steps) > 1 {
|
|
|
|
iter.stepqueue = steps[1:]
|
|
|
|
}
|
|
|
|
return steps[0], nil
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
|
|
|
|
// If all returns are nil, the source is done, note it, and don't go back for more. Add any deletions to be
|
|
|
|
// performed, and then keep going 'round the next iteration of the loop so we can wrap up the planning.
|
|
|
|
iter.srcdone = true
|
|
|
|
iter.delqueue = iter.calculateDeletes()
|
2017-06-10 03:34:37 +02:00
|
|
|
} else {
|
|
|
|
// The interpreter has finished, so we need to now drain any deletions that piled up.
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
if step := iter.nextDeleteStep(); step != nil {
|
2017-06-10 03:34:37 +02:00
|
|
|
return step, nil
|
|
|
|
}
|
|
|
|
|
|
|
|
// Otherwise, if the deletes have quiesced, there is nothing remaining in this plan; leave.
|
|
|
|
iter.done = true
|
|
|
|
break
|
|
|
|
}
|
|
|
|
}
|
|
|
|
return nil, nil
|
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// nextResourceSteps produces one or more steps required to achieve the desired resource goal state, or nil if there
|
|
|
|
// aren't any steps to perform (in other words, the actual known state is equivalent to the goal state). It is
|
|
|
|
// possible to return multiple steps if the current resource state necessitates it (e.g., replacements).
|
|
|
|
func (iter *PlanIterator) nextResourceSteps(goal SourceGoal) ([]Step, error) {
|
2017-06-10 03:34:37 +02:00
|
|
|
// Fetch the provider for this resource type.
|
2017-08-30 03:24:12 +02:00
|
|
|
res := goal.Resource()
|
|
|
|
prov, err := iter.Provider(res.Type)
|
2017-06-10 03:34:37 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
|
2017-08-01 03:26:15 +02:00
|
|
|
var invalid bool // will be set to true if this object fails validation.
|
2017-06-27 22:04:06 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Use the resource goal state name to produce a globally unique URN.
|
2017-08-31 18:05:10 +02:00
|
|
|
urn := resource.NewURN(iter.p.Target().Name, iter.p.source.Pkg(), res.Type, res.Name)
|
2017-06-27 22:04:06 +02:00
|
|
|
if iter.urns[urn] {
|
|
|
|
invalid = true
|
2017-08-30 03:24:12 +02:00
|
|
|
// TODO[pulumi/pulumi-framework#19]: improve this error message!
|
|
|
|
iter.p.Diag().Errorf(errors.ErrorDuplicateResourceURN, urn)
|
2017-06-27 22:04:06 +02:00
|
|
|
}
|
|
|
|
iter.urns[urn] = true
|
2017-06-10 03:34:37 +02:00
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Produce a new state object that we'll build up as operations are performed. It begins with empty outputs.
|
|
|
|
// Ultimately, this is what will get serialized into the checkpoint file.
|
|
|
|
new := resource.NewState(res.Type, urn, "", res.Properties, nil, nil)
|
|
|
|
|
2017-08-01 03:26:15 +02:00
|
|
|
// If there is an old resource, apply its default properties before going any further.
|
|
|
|
old, hasold := iter.p.Olds()[urn]
|
|
|
|
if hasold {
|
|
|
|
new.Defaults = new.Defaults.Merge(old.Defaults)
|
|
|
|
if glog.V(9) {
|
|
|
|
for k, v := range old.Defaults {
|
|
|
|
glog.V(9).Infof("Applying old default %v=%v", k, v)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// Ensure the provider is okay with this resource and see if it has any new defaults to contribute.
|
|
|
|
inputs := new.AllInputs()
|
2017-08-31 22:10:55 +02:00
|
|
|
defaults, failures, err := prov.Check(urn, inputs)
|
2017-06-10 03:34:37 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
2017-08-01 03:26:15 +02:00
|
|
|
} else if iter.issueCheckErrors(new, urn, failures) {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
invalid = true
|
2017-08-01 03:26:15 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// If there are any new defaults, apply them, and use the combined view for various operations below.
|
|
|
|
if defaults != nil {
|
|
|
|
new.Defaults = new.Defaults.Merge(defaults)
|
|
|
|
inputs = new.AllInputs() // refresh our snapshot.
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// Next, give each analyzer -- if any -- a chance to inspect the resource too.
|
|
|
|
for _, a := range iter.p.analyzers {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
analyzer, err := iter.p.ctx.Host.Analyzer(a)
|
2017-06-10 03:34:37 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
2017-08-30 03:24:12 +02:00
|
|
|
failures, err := analyzer.Analyze(new.Type, inputs)
|
2017-06-10 03:34:37 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
for _, failure := range failures {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
invalid = true
|
2017-06-15 02:00:13 +02:00
|
|
|
iter.p.Diag().Errorf(errors.ErrorAnalyzeResourceFailure, a, urn, failure.Property, failure.Reason)
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// If the resource isn't valid, don't proceed any further.
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
if invalid {
|
2017-06-10 03:34:37 +02:00
|
|
|
return nil, goerr.New("One or more resource validation errors occurred; refusing to proceed")
|
|
|
|
}
|
|
|
|
|
|
|
|
// Now decide what to do, step-wise:
|
|
|
|
//
|
|
|
|
// * If the URN exists in the old snapshot, and it has been updated,
|
|
|
|
// - Check whether the update requires replacement.
|
|
|
|
// - If yes, create a new copy, and mark it as having been replaced.
|
|
|
|
// - If no, simply update the existing resource in place.
|
|
|
|
//
|
|
|
|
// * If the URN does not exist in the old snapshot, create the resource anew.
|
|
|
|
//
|
2017-08-01 03:26:15 +02:00
|
|
|
if hasold {
|
2017-08-30 03:24:12 +02:00
|
|
|
contract.Assert(old != nil && old.Type == new.Type)
|
2017-06-11 17:09:20 +02:00
|
|
|
|
2017-06-10 03:34:37 +02:00
|
|
|
// The resource exists in both new and old; it could be an update. This constitutes an update if the old
|
|
|
|
// and new properties don't match exactly. It is also possible we'll need to replace the resource if the
|
|
|
|
// update impact assessment says so. In this case, the resource's ID will change, which might have a
|
|
|
|
// cascading impact on subsequent updates too, since those IDs must trigger recreations, etc.
|
2017-08-01 03:26:15 +02:00
|
|
|
oldinputs := old.AllInputs()
|
|
|
|
if !oldinputs.DeepEquals(inputs) {
|
|
|
|
// The properties changed; we need to figure out whether to do an update or replacement.
|
2017-08-31 22:10:55 +02:00
|
|
|
diff, err := prov.Diff(urn, old.ID, oldinputs, inputs)
|
2017-08-01 03:26:15 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
}
|
|
|
|
// This is either an update or a replacement; check for the latter first, and handle it specially.
|
|
|
|
if diff.Replace() {
|
|
|
|
iter.replaces[urn] = true
|
|
|
|
// If we are going to perform a replacement, we need to recompute the default values. The above logic
|
|
|
|
// had assumed that we were going to carry them over from the old resource, which is no longer true.
|
2017-08-31 22:10:55 +02:00
|
|
|
defaults, failures, err := prov.Check(urn, new.Inputs)
|
2017-08-01 03:26:15 +02:00
|
|
|
if err != nil {
|
|
|
|
return nil, err
|
|
|
|
} else if iter.issueCheckErrors(new, urn, failures) {
|
|
|
|
return nil, goerr.New("One or more resource validation errors occurred; refusing to proceed")
|
|
|
|
}
|
|
|
|
new.Defaults = defaults
|
|
|
|
if glog.V(7) {
|
|
|
|
glog.V(7).Infof("Planner decided to replace '%v' (oldprops=%v inputs=%v)",
|
|
|
|
urn, oldinputs, new.AllInputs())
|
|
|
|
}
|
2017-08-06 19:05:51 +02:00
|
|
|
return []Step{
|
2017-08-30 03:24:12 +02:00
|
|
|
NewCreateReplacementStep(iter, goal, old, new, diff.ReplaceKeys),
|
|
|
|
NewReplaceStep(iter, old, new, diff.ReplaceKeys),
|
2017-08-06 19:05:51 +02:00
|
|
|
}, nil
|
2017-08-01 03:26:15 +02:00
|
|
|
}
|
|
|
|
iter.updates[urn] = true
|
|
|
|
if glog.V(7) {
|
|
|
|
glog.V(7).Infof("Planner decided to update '%v' (oldprops=%v inputs=%v",
|
|
|
|
urn, oldinputs, new.AllInputs())
|
|
|
|
}
|
2017-08-30 03:24:12 +02:00
|
|
|
return []Step{NewUpdateStep(iter, goal, old, new)}, nil
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
2017-08-01 03:26:15 +02:00
|
|
|
// No need to update anything, the properties didn't change.
|
|
|
|
iter.sames[urn] = true
|
|
|
|
if glog.V(7) {
|
|
|
|
glog.V(7).Infof("Planner decided not to update '%v' (same) (inputs=%v)", urn, new.AllInputs())
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
2017-08-30 03:24:12 +02:00
|
|
|
return []Step{NewSameStep(iter, goal, old, new)}, nil
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// Otherwise, the resource isn't in the old map, so it must be a resource creation.
|
|
|
|
iter.creates[urn] = true
|
2017-08-01 03:26:15 +02:00
|
|
|
glog.V(7).Infof("Planner decided to create '%v' (inputs=%v)", urn, new.AllInputs())
|
2017-08-30 03:24:12 +02:00
|
|
|
return []Step{NewCreateStep(iter, goal, new)}, nil
|
2017-08-01 03:26:15 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// issueCheckErrors prints any check errors to the diagnostics sink.
|
|
|
|
func (iter *PlanIterator) issueCheckErrors(new *resource.State, urn resource.URN,
|
|
|
|
failures []plugin.CheckFailure) bool {
|
|
|
|
if len(failures) == 0 {
|
|
|
|
return false
|
|
|
|
}
|
|
|
|
inputs := new.AllInputs()
|
|
|
|
for _, failure := range failures {
|
|
|
|
if failure.Property != "" {
|
|
|
|
iter.p.Diag().Errorf(errors.ErrorResourcePropertyInvalidValue,
|
2017-08-30 03:24:12 +02:00
|
|
|
new.Type, urn.Name(), failure.Property, inputs[failure.Property], failure.Reason)
|
2017-08-01 03:26:15 +02:00
|
|
|
} else {
|
2017-08-30 03:24:12 +02:00
|
|
|
iter.p.Diag().Errorf(errors.ErrorResourceInvalid, new.Type, urn.Name(), failure.Reason)
|
2017-08-01 03:26:15 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
return true
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
// nextDeleteStep produces a new step that deletes a resource if necessary.
|
|
|
|
func (iter *PlanIterator) nextDeleteStep() Step {
|
2017-06-10 03:34:37 +02:00
|
|
|
if len(iter.delqueue) > 0 {
|
|
|
|
del := iter.delqueue[0]
|
|
|
|
iter.delqueue = iter.delqueue[1:]
|
2017-08-30 03:24:12 +02:00
|
|
|
urn := del.URN
|
2017-06-10 03:34:37 +02:00
|
|
|
iter.deletes[urn] = true
|
2017-06-11 02:03:58 +02:00
|
|
|
if iter.replaces[urn] {
|
|
|
|
glog.V(7).Infof("Planner decided to delete '%v' due to replacement", urn)
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
} else {
|
|
|
|
glog.V(7).Infof("Planner decided to delete '%v'", urn)
|
2017-06-11 02:03:58 +02:00
|
|
|
}
|
Implement `get` functions on all resources
This change implements the `get` function for resources. Per pulumi/lumi#83,
this allows Lumi scripts to actually read from the target environment.
For example, we can now look up a SecurityGroup from its ARN:
let group = aws.ec2.SecurityGroup.get(
"arn:aws:ec2:us-west-2:153052954103:security-group:sg-02150d79");
The returned object is a fully functional resource object. So, we can then
link it up with an EC2 instance, for example, in the usual ways:
let instance = new aws.ec2.Instance(..., {
securityGroups: [ group ],
});
This didn't require any changes to the RPC or provider model, since we
already implement the Get function.
There are a few loose ends; two are short term:
1) URNs are not rehydrated.
2) Query is not yet implemented.
One is mid-term:
3) We probably want a URN-based lookup function. But we will likely
wait until we tackle pulumi/lumi#109 before adding this.
And one is long term (and subtle):
4) These amount to I/O and are not repeatable! A change in the target
environment may cause a script to generate a different plan
intermittently. Most likely we want to apply a different kind of
deployment "policy" for such scripts. These are inching towards the
scripting model of pulumi/lumi#121, which is an entirely different
beast than the repeatable immutable infrastructure deployments.
Finally, it is worth noting that with this, we have some of the fundamental
underpinnings required to finally tackle "inference" (pulumi/lumi#142).
2017-06-20 02:24:00 +02:00
|
|
|
return NewDeleteStep(iter, del, iter.replaces[urn])
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
return nil
|
|
|
|
}
|
|
|
|
|
|
|
|
// calculateDeletes creates a list of deletes to perform. This will include any resources in the snapshot that were
|
|
|
|
// not encountered in the input, along with any resources that were replaced.
|
|
|
|
func (iter *PlanIterator) calculateDeletes() []*resource.State {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
// To compute the deletion list, we must walk the list of old resources *backwards*. This is because the list is
|
|
|
|
// stored in dependency order, and earlier elements are possibly leaf nodes for later elements. We must not delete
|
|
|
|
// dependencies prior to their dependent nodes.
|
2017-06-10 03:34:37 +02:00
|
|
|
var dels []*resource.State
|
2017-06-11 15:52:56 +02:00
|
|
|
if prev := iter.p.prev; prev != nil {
|
|
|
|
for i := len(prev.Resources) - 1; i >= 0; i-- {
|
|
|
|
res := prev.Resources[i]
|
2017-08-30 03:24:12 +02:00
|
|
|
urn := res.URN
|
2017-06-11 15:52:56 +02:00
|
|
|
contract.Assert(!iter.creates[urn])
|
2017-06-11 17:09:20 +02:00
|
|
|
if (!iter.sames[urn] && !iter.updates[urn]) || iter.replaces[urn] {
|
2017-06-11 15:52:56 +02:00
|
|
|
dels = append(dels, res)
|
|
|
|
}
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
}
|
|
|
|
return dels
|
|
|
|
}
|
|
|
|
|
Tidy up more lint
This change fixes a few things:
* Most importantly, we need to place a leading "." in the paths
to Gometalinter, otherwise some sub-linters just silently skip
the directory altogether. errcheck is one such linter, which
is a very important one!
* Use an explicit Gometalinter.json file to configure the various
settings. This flips on a few additional linters that aren't
on by default (line line length checking). Sadly, a few that
I'd like to enable take waaaay too much time, so in the future
we may consider a nightly job (this includes code similarity,
unused parameters, unused functions, and others that generally
require global analysis).
* Now that we're running more, however, linting takes a while!
The core Lumi project now takes 26 seconds to lint on my laptop.
That's not terrible, but it's long enough that we don't want to
do the silly "run them twice" thing our Makefiles were previously
doing. Instead, we shall deploy some $$($${PIPESTATUS[1]}-1))-fu
to rely on the fact that grep returns 1 on "zero lines".
* Finally, fix the many issues that this turned up.
I think(?) we are done, except, of course, for needing to drive
down some of the cyclomatic complexity issues (which I'm possibly
going to punt on; see pulumi/lumi#259 for more details).
2017-06-22 21:09:46 +02:00
|
|
|
// Snap returns a fresh snapshot that takes into account everything that has happened up till this point. Namely, if a
|
2017-06-10 03:34:37 +02:00
|
|
|
// failure happens partway through, the untouched snapshot elements will be retained, while any updates will be
|
|
|
|
// preserved. If no failure happens, the snapshot naturally reflects the final state of all resources.
|
|
|
|
func (iter *PlanIterator) Snap() *Snapshot {
|
2017-06-12 16:16:08 +02:00
|
|
|
var resources []*resource.State
|
|
|
|
|
|
|
|
// If we didn't finish the execution, we must produce a partial snapshot of old plus new states.
|
|
|
|
if !iter.done {
|
|
|
|
if prev := iter.p.prev; prev != nil {
|
|
|
|
for _, res := range prev.Resources {
|
|
|
|
if !iter.dones[res] {
|
|
|
|
resources = append(resources, res)
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// Always add the new resoures afterwards that got produced during the evaluation of the current plan.
|
|
|
|
resources = append(resources, iter.resources...)
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
return NewSnapshot(iter.p.Target().Name, resources, iter.p.source.Info())
|
2017-06-12 16:16:08 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// MarkStateSnapshot marks an old state snapshot as being processed. This is done to recover from failures partway
|
|
|
|
// through the application of a deployment plan. Any old state that has not yet been recovered needs to be kept.
|
|
|
|
func (iter *PlanIterator) MarkStateSnapshot(state *resource.State) {
|
|
|
|
iter.dones[state] = true
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
// AppendStateSnapshot appends a resource's state to the current snapshot.
|
|
|
|
func (iter *PlanIterator) AppendStateSnapshot(state *resource.State) {
|
2017-06-12 16:16:08 +02:00
|
|
|
iter.resources = append(iter.resources, state)
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|
|
|
|
|
2017-08-30 03:24:12 +02:00
|
|
|
// Provider fetches the provider for a given resource type, possibly lazily allocating the plugins for it. If a
|
|
|
|
// provider could not be found, or an error occurred while creating it, a non-nil error is returned.
|
|
|
|
func (iter *PlanIterator) Provider(t tokens.Type) (plugin.Provider, error) {
|
Make more progress on the new deployment model
This change restructures a lot more pertaining to deployments, snapshots,
environments, and the like.
The most notable change is that the notion of a deploy.Source is introduced,
which splits the responsibility between the deploy.Plan -- which simply
understands how to compute and carry out deployment plans -- and the idea
of something that can produce new objects on-demand during deployment.
The primary such implementation is evalSource, which encapsulates an
interpreter and takes a package, args, and config map, and proceeds to run
the interpreter in a distinct goroutine. It synchronizes as needed to
poke and prod the interpreter along its path to create new resource objects.
There are two other sources, however. First, a nullSource, which simply
refuses to create new objects. This can be handy when writing isolated
tests but is also used to simulate the "empty" environment as necessary to
do a complete teardown of the target environment. Second, a fixedSource,
which takes a pre-computed array of objects, and hands those, in order, to
the planning engine; this is mostly useful as a testing technique.
Boatloads of code is now changed and updated in the various CLI commands.
This further chugs along towards pulumi/lumi#90. The end is in sight.
2017-06-10 20:50:47 +02:00
|
|
|
pkg := t.Package()
|
|
|
|
return iter.p.ctx.Host.Provider(pkg)
|
2017-06-10 03:34:37 +02:00
|
|
|
}
|