Commit graph

432 commits

Author SHA1 Message Date
joeduffy 6a23300d21 Reformat example; use "new()" in place of "resources" 2016-12-16 11:15:33 -08:00
joeduffy 284cec204d Experiment with intersection types
This change experiments with using intersection types to cut down
on some of the boilerplate -- and make reuse easier -- in the
service definitions.
2016-12-16 10:04:46 -08:00
joeduffy ed1eebe7e5 Clean up imports 2016-12-15 19:42:48 -08:00
joeduffy 2e941bbc57 Make an initial attempt at a better factoring
This splits the overall example rack service into many sub-services.
This leads to a much cleaner factoring of the code.  Note that there are
some missing properties -- it's hard to eyeball this without a real compiler.
But the essence of the example is pretty spot on.
2016-12-15 19:40:34 -08:00
joeduffy 68a3d27a73 Use "prop: value" instead of "prop = value"
I personally prefer this syntax.  It is more "declarative" and, particularly
because we support true assignments, I feel like it's more representative of
what's going on.  It's easier to scan for real imperative assignments.  This
approach also eliminates a single horizontal character per assignment.
2016-12-15 17:57:36 -08:00
joeduffy 14471c0ac7 Experiment with different syntax for resource creation
In this change, ":= new" declares a new resource.

In addition, we change arrays to be Go-like, in that zero-initialized
ones can be appended to safely, such that declaring a new instance is
done merely by stating "var arr: T[]".
2016-12-15 17:50:10 -08:00
joeduffy 47e1125f3b Use arrays to cut down on boilerplate
This is all hand-waving, of course, however it's helping with the language design.
2016-12-15 16:58:21 -08:00
joeduffy 5e652c3dd3 Check in the Xovnoc conversion example 2016-12-15 16:40:49 -08:00
joeduffy 5549d3a2e8 Make a few minor language doc edits 2016-12-15 16:40:19 -08:00
joeduffy d972ab9b6b Talk a little about runtime bindings 2016-12-14 16:55:24 -08:00
joeduffy 9d9c57335f Rename macro to func; rename mu/func to mu/lambda 2016-12-14 16:44:31 -08:00
joeduffy 540d058cf2 Use a separate service for AWS cluster
This change encapsulates what's required for an AWS cluster in new service,
mu/clouds/aws/Cluster, so that mu/Cluster can remain fairly simplistic (it
just switches on the names and dispatches to the appropriate sub-modules).
2016-12-14 15:15:29 -08:00
joeduffy 46e89084bf Add a hypothetical Echo example 2016-12-14 15:13:17 -08:00
joeduffy 646bb624db Rename rpcs to interface 2016-12-14 15:11:57 -08:00
joeduffy 8ba1bf6f22 Commit more progress on the metadata language design
Mostly a bunch of TODOs, however, a bit more about RPCs, services, streams,
and the output graph state.
2016-12-14 15:04:27 -08:00
joeduffy 87d2a6e06e Create a new design doc for the metadata language
This checkin includes a new design doc for the Mull metadata language.

This is very much a work-in-progress.

This will eventually supersede the language.md document.
2016-12-14 13:08:08 -08:00
joeduffy 4bc7737545 Eliminate quotes from module/import statements
There's no need for quotes here and having them makes it feel somehow
"dynamically typed" (I was subconsciously influenced by Go, I think).
Removing them...
2016-12-13 16:47:27 -08:00
joeduffy 468d7f8f3e A few renames
This change renames a few things; more to come, but this is at least
a self-consistent checkpoint:

* Use the "new" keyword to create service objects and not "resource".

* Use the "func" keyword to indicate functions and not "macro".

* Use "bool" instead of "boolean" for boolean types (more Go-like).
2016-12-13 16:39:37 -08:00
joeduffy e1c3494ba3 Fix two minor issues in the *.mu files
First, I meant to rename "func" to "macro".  This is perhaps contentious,
however my thinking was to differentiate between functions which have runtime
representations -- like lambdas, API gateway handlers, and the like -- and
macros which are purely a metadata/compile-time construct.

Second, there was a resource naming typo.
2016-12-13 15:44:12 -08:00
joeduffy 2d003a2df3 Sketching out a minimal configuration language
This sketches out a minimal configuration language, inspired by a combination
of the Mull document, Hashicorp's various language efforts, Protobufs, and a
mixture of TypeScript.  I'm attempting to whittle down the concepts to the bare
minimum necessary for a universal "intermediate language" for our runtime that
is (a) complete enough to serve our needs, (b) advanced enough to deliver some
of the improvements in componentization and reuse that we desire, (c) appealing
enough that humans are able to write code in the language while possibly even
enjoying it, and, yet, (d) sufficient for machine interoperability needs.

Of course, I will capture all of this in a proper specification (overwriting
the existing Mull one), however I wanted to land these as a proof of concept and
for feedback purposes.  Obviously, none of this code actually compiles or works!
2016-12-13 15:33:13 -08:00
joeduffy 1a3bfc182c Use strongly typed mu.Stacks, not strings, for CloudFormation's dependsOn 2016-12-12 18:56:37 -08:00
joeduffy 832bb80b47 Checkin an initial Mu container that targets AWS ECS
This is a work-in-progress.  In fact, it's going to radically change
given the new approach to representing resource construction, however
I wanted the snapshot in source control as we evolve things.
2016-12-12 17:56:54 -08:00
joeduffy e731229901 Make the Mu library a Node package; get it compiling
This change morphs the Mu library into a Node package and gets it
compiling against the latest AWS library and SDK changes.
2016-12-12 17:56:13 -08:00
joeduffy 47c67e7d5c Turn the AWS library into a Node package 2016-12-12 17:55:30 -08:00
joeduffy c7f0465c41 Make an aws/cloudformation module
This includes the original CloudFormation resource type in addition
to the various tag helpers.
2016-12-12 17:54:50 -08:00
joeduffy d10f2e6bff Get the AWS stacks compiling
This actually compiles... and does absolutely nothing (yet).
2016-12-12 17:25:27 -08:00
joeduffy 6160d1a4c8 Export all submodules 2016-12-12 16:28:39 -08:00
joeduffy 25c8211703 Add a JSON-like suite of types 2016-12-12 16:26:56 -08:00
joeduffy 35694229ca Add a minimal JavaScript (TypeScript) SDK
This change introduces a basic JavaScript SDK (actually in TypeScript,
but consumable either way).  This is just scaffolding but provides the
minimal set of abstractions necessary to start writing real stacks.
2016-12-12 16:07:39 -08:00
joeduffy 3c082ce20e Modularize the AWS proof of concept stacks
And get them building.  Mostly.
2016-12-12 15:35:56 -08:00
joeduffy 5f647b6cc0 Create proof of concept TypeScript resources
This demonstrates what infra-as-code might look like.  Given the difficulties
in expressing everything purely as metadata, I've decided to embark upon a little
spike to see what this would look like.  At first glance, I like it.

To see the difference, compare these to the Mu.yaml files alongside them.
2016-12-12 14:35:00 -08:00
joeduffy 4195c5be9b Add two follow-ups we need to think about, typing-wise 2016-12-12 09:26:06 -08:00
joeduffy d12f249248 Fix references to each
In an older version of this doc, `if` was called `when`, and `for`
was called `each`, to encourage a more declarative feel.  I renamed
them back to their familiar forms.  I missed a few spots.  But I also
thought at least documenting this heritage would be useful, hence
this descriptive checkin comment.
2016-12-12 09:21:30 -08:00
joeduffy dc76cbb989 Add thoughts on a little language
This change includes a miniature spec for what we'd want out of a little
markup language that extends YAML/JSON with typing and minimal templating.

We've begun to reach the limits of what Go's templates give us; the usability
is quite poor: the order of template expansion is "confused" (as it must
happen before verification of stack properties); it is dumb textual copy-and-
paste, and thus knows nothing about the lexical and semantic rules; evaluation
of expressions that should produce actual objects inserted into the metadata
stream as-is must actually be serialized into text (problematic for the above
reasons); and, finally, as a result of all of this, failure modes are terrible.

But worse than this, we simply can't do what we need in many places.  For
instance, mapping a stack's properties onto the services that it creates works
in simple cases -- like strings, booleans, and ints -- but quickly breaks down
when referencing complex objects (for the same above reasons).  This is why
we've needed to special case property mapping in the aws/x/cf provider, but
clearly this won't generalize to all the compositional situations that arise.

It's worth nothing Hashicorp's HCL/HIL is closest to what we want.  (The
language used for Terraform.)  It isn't exactly what we want, however, for two
reasons.  First, it lacks conditionals and iteration.  This is likely to appear
at some point (see https://github.com/hashicorp/terraform/issues/1604), and
indeed in this past week alone, a new C-like conditional operator (which I
actually don't love) got added to HIL:
5fe4b10b43.
Second, and perhaps more importantly, its approach is to create a new language.
The design I list here is a natural extension that adds typechecking and
minimal templating to the existing YAML/JSON formats.  As a stand-alone
project, this whould have a much broader appeal.  And whether or not we use it
for Mu depends on whether we really want an entirely new markup language or not.

To cut to the chase, I'm shelving this for now.  I'm going to keep hacking my
way through the current Go templates plus special-casing for now.  My eye is
on the initial end-to-end prototype.  But, no doubt, we'll need to revisit this
immediately afterwards, make a decision, and make it happen.
2016-12-12 09:11:27 -08:00
joeduffy d22e6b44b5 Add orElse/orEmpty template functions
This change adds the ability to substitute a default value if one is
missing from a map (orElse), and/or to substitute an empty string (orEmpty).
2016-12-09 17:22:09 -08:00
joeduffy af059c8121 Require cloud=="aws" for aws/x/cf 2016-12-09 13:22:35 -08:00
joeduffy db80229899 Fix a few type binding mistakes
* Persue the default/optional checking if a property value == nil.

* Use the Interface() function to convert a reflect.Type to its underlying
  interface{} value.  This is required for typechecking to check out.

* Also, unrelated to the above, change type assertions to use nil rather than
  allocating real objects.  Although minimal, this incurs less GC pressure.
2016-12-09 13:12:57 -08:00
joeduffy b408c3ce2a Pass compiler options to template evaluation
In some cases, we want to specialize template generation based on
the options passed to the compiler.  This change flows them through
so that they can be accessed as

        {{if .Options.SomeSetting}}
        ...
        {{end}}
2016-12-09 12:42:28 -08:00
joeduffy 3c5ca84d89 Switch back to the official YAML repo
Sam merged the pull request, so we can go back to the official repo.
This closes https://github.com/marapongo/mu/issues/28.
2016-12-09 11:59:05 -08:00
joeduffy e137837455 Revert back to T[], instead of []T, for array syntax
This change reverts the syntax for arrays back to T[] from []T.  The main
reason is that YAML doesn't permit unquoted strings beginning with [], meaning
any array type needs to be quoted as in "[]T", which is annoying compared to all
other primitive types which don't require quotes.  And, anyway, this syntax is
more familiar too.

I've also added a number of tests.
2016-12-07 13:24:05 -08:00
joeduffy 6a8df6126a Reject Refs with missing names 2016-12-07 13:07:54 -08:00
joeduffy eb6ef5a1b8 Fix control paths within bindValue
Any of the bindXValue routines can fail if there was no way to convert
the interface{} to an ast.Literal.  In such a case, we need to issue an
error about the wrong type being passed.  Unfortunately, in the most
recent set of changes, we began simply returning nils without issuing
the error.  This change fixes that.
2016-12-07 12:40:26 -08:00
joeduffy 9e005fb6d8 Rename Schemas to Types
This change renames Schemas to Types on Stack.  More interestingly, it
renames the JSON/YAML property used to specify them, from "schemas:" to
"types:"; I feel like this reads more naturally, especially as a sister
to the existing "services:" section.
2016-12-06 20:56:47 -08:00
joeduffy 86219e781b Custom types, round 2
This checkin continues progress on marapongo/mu#9.  It's still not
complete, however we're getting there.  In particular, this includes:

* Rename of ComplexLiteral to SchemaLiteral, as it is used exclusively
  for schematized types.  Also includes a set of changes associated
  with this, like deep value conversion to `map[string]interface{}`.

* Binding of schema types included within a Stack.  This allows names in
  type references to be bound to those schema types during typechecking.
  This also includes binding schema properties, reusing all the existing
  property binding logic for stacks.  In this way, properties between
  stacks and custom schema types are one and the same, which is nice.

* Enforcement for custom schema constraints; this includes Pattern,
  MaxLength, MinLength, Maximum, and Minimum, as per the JSON Schema
  specification.
2016-12-06 20:51:05 -08:00
joeduffy 38ec8d99ed Add a --skip-codegen option to the build command
This capability already exists (mostly for testing purposes);
expose it at the CLI so that we can easily flip it to true.
2016-12-06 20:49:54 -08:00
joeduffy 713fe29fef Custom types, round 1
This change overhauls the core of how types are used by the entire
compiler.  In particular, we now have an ast.Type, and have begun
using its use where appropriate.  An ast.Type is a union representing
precisely one of the possible sources of types in the system:

* Primitive type: any, bool, number, string, or service.

* Stack type: a resolved reference to an actual concrete stack.

* Schema type: a resolved reference to an actual concrete schema.

* Unresolved reference: a textual reference that hasn't yet been
  resolved to a concrete artifact.

* Uninstantiated reference: a reference that has been resolved to
  an uninstantiated stack, but hasn't been bound to a concrete
  result yet.  Right now, this can point to a stack, however
  eventually we would imagine this supporting inter-stack schema
  references also.

* Decorated type: either an array or a map; in the array case, there
  is a single inner element type; in the map case, there are two,
  the keys and values; in all cases, the type recurses to any of the
  possibilities listed here.

All of the relevant AST nodes have been overhauled accordingly.

In addition to this, we now have an ast.Schema type.  It is loosely
modeled on JSON Schema in its capabilities (http://json-schema.org/).
Although we parse and perform some visitation and binding of these,
there are mostly placeholders left in the code for the interesting
aspects, such as registering symbols, resolving dependencies, and
typechecking usage of schema types.

This is part of the ongoing work behind marapongo/mu#9.
2016-12-06 14:49:47 -08:00
joeduffy 20e584122a Update aws/ecs/task schema to new format 2016-12-05 18:33:26 -08:00
joeduffy f88997cc03 Specify more about extensible schema types
As part of marapongo/mu#9, we want to enable extensible schema types
for stronger typechecking at compile-time.  JSON Schema seems like a
decent starting place (http://json-schema.org/), although it's not yet
clear whether we want to use its module/naming schema or our own.  I
suspect we want to use our own so that stack schemas can be managed
using the same discipline as stack management generally.
2016-12-05 18:11:56 -08:00
joeduffy a9fa42a60c Project ECS task resource stack
This projects the basic AWS::ECS::TaskDefinition CloudFormation template
type as a stack.  It also maps a bunch of schema types using fictitious
syntax, since we don't yet support this (see marapongo/mu#9).  On to that next...
2016-12-05 17:14:22 -08:00
joeduffy a28f02ce68 Use intrinsics in place of predefineds
This change leverages intrinsics in place of the predefined types.
It remains to be seen if we can reach 100% on this, however I am hopeful.
It's also nice that the system will be built "out of itself" with this
approach; in other words, each of the types is simply a Mufile that can
use conditional targeting as appropriate for the given cloud providers.
If we find that this isn't enough, we can always bring back the concept.
2016-12-05 16:13:49 -08:00