For some tools we used, like `dep` and `gometalinter` we were just
calling `go install` which caused us to pick up whatever was in `HEAD`
at that time. Now, we move to a model where we install fixed versions,
which will change per milestone.
While doing this, I changed the way our .travis.yml file runs
everything to move as much as possible out into scripts and do so in a
way that allows us to share as much common logic across our
repositories.
Uploading a bunch of tiny files is inefficient. Worse, it results in a ton of
paths printed to `stdout` -- enough that we could hit Travis' 4MB limit and
kill the job before we'd finished uploading.
Now we create a `.tar.gz` and upload that one, compressed file.
We also noticed that copied files might not be accessible from the `dev`
account, even though that account owns the `eng.pulumi.com` bucket. When
files are uploaded by one AWS account to a bucket owned by another, the
objects are, by default, only readable by the first account (the writer).
Change the ACL so the account that owns the bucket also has full access.
Our use of parens around the individual clauses of our if condition
seemed to cause Travis to ignore the condition and build
regardless. I've asked Travis about this, but for now, drop them (with
a comment) so we stop building pushes for topic branchs.
In order to begin publishing our core SDK package to NPM, we will
need it to be underneath the @pulumi scope so that it may remain
private. Eventually, we can alias pulumi back to it.
This is part of pulumi/pulumi#915.
Previously, we used safelisting to ensure we only build the master
branch on push jobs. Because Travis treated branches and tags as the
same thing when it came to safelisting, it meant that pushes of tags
did not cause builds.
Our versioning workflow requires that we rebuild on tag pushes,
because the tag plays into the version number we embed in our programs
and packages.
Instead, we move to the new Conditional Builds feature of Travis to
also allow us to build tags.
Part of pulumi/sdk#11
In travis, we've seen cases where writes to our standard streams
results in an error like: `/dev/stderr: resource temporarily
unavailable` which causes the tests to panic.
Now, in a perfect world, writes to /dev/stderr would not fail in this
way, but we do not live in a perfect world. Other processes on the
machine may make stderr/stdout non-blocking. We've are now seeing this
failure in Travis more often and it is masking real Pulumi failures
we want to debug.
In Travis, Node sees that `stdout` is a pipe and sets it to nonblocking.
Go's standard library isn't prepared for `stdout` to be nonblocking, so
sometimes long print statements can fail.
Unlike go binaries (where we can cross compile) the node module that
we publish needs to be built on the platform we publish for. Update
our `.travis.yml` file to also build on macOS and fix the publishing
script so we don't don't cross publish Darwin from Linux. Once we have
CI working for Windows, we'll remove the loop over PUBLISH_GOOS and
each build will publish just the artifacts for the host OS.
Defer build and publish logic into the makefiles instead of trying to
use Travis to do so. This let's us sidestep Travis's not great error
handling (see pulumi/home#21 for more details)
This change adds a `make configure` target, which handles preparing
the environment for building the project. This includes existing
steps, like dep ensure and yarn installing the Node.js SDK NPM
dependencies, and also includes downloading the right Node.js/V8
includes, putting them in the right place, and then generating the
appropriate node-gyp project files that reference those includes.
We are now on our fourth vendoring tool: Govendor. This appears to
be about 10X faster than Dep, fails less frequently, and has a rich,
well documented set of commands (that make far more intuitive sense
to me, particularly when compared to Dep). I could never really get
Godep to work with vendor/ correctly, whereas Govendor did the trick
right away. Lastly, Terraform uses it, so we'll probably have fewer
headaches in that department also.
To workaround the fact that our repos end in a dirty state after
the CI job finishes, I'm adding PUBFORCE=true to the CI script.
pulumi/lumi#300 tracks figuring out the root cause and removing it.
This adds a new make target, publish, that will create a release
package and upload it to an S3 bucket. This package simply contains
the built CLI plus the core library packages, lumi, lumirt, and lumijs.
This target is invoked automatically at the end of a successful
Travis run against a push to master.
Adds a make task to generate code coverage for all Go sources.
That make task re-runs the tests, and can be fairly expensive,
so it is enabled only in the nightly tests for now.
Part of #206.
This change enables VM-based builds in Travis, versus the default of
container-based builds. This will give us more memory (7.5GB rather
than 4GB max), and more compute (~2, bursted rather than 2). This
may help to fix some of the build/test speed time issues we are seeing.
