This change implements simple index-based CocoPy subscripts (and
not the more fully featured slicing ones).
Alongside this, we relax a binder-time check that all dynamic
access types must be strings. The eval code already handles
numeric (array) accesses, so we will permit these to flow through.
This permits non-nil `this` objects for dynamically loaded properties
of classes and modules, provided they are the prototype or module
object for the target function, respectively.
This change emits all CocoJS interfaces as records. This allows us to
safely construct instances of them from Python using anonymous properties,
essentially emulating object literals.
For example, given a CocoJs interface defined as such:
interface Foo {
x;
y?;
z;
}
we can easily construct fresh instances using normal JS literals:
let f = { x: 42, z: "bar" };
But, Python doesn't have the equivalent literal syntax, wedging us.
It's now possible to initialize an instance from CocoPy as follows:
let f = Foo(x=42, z="bar")
This leverages the notion of records and primary properties, as
described in our CocoPack/CocoIL design documents.
* Pushing a class scope should permit subclasses (assertion).
* Returning nothing is legal for voids *and* dynamic (the latter was missing).
* The dynamic load readonly lval check is redundant; we check at the site of
assignment and doing it here as well led to two errors per usage.
Due to Python's ... interesting ... scoping rules, we want to avoid
forcing block scopes in certain places. Instead, we will let arbitrary
statements take their place. Of course, this could be a block, but it
very well could be a multi-statement (essentially a block that doesn't
imply a new lexical scope), or anything, really.
* Move checking the validity of a derived class with no constructor
from evlauation to binding (verification). Furthermore, make it
a verification error instead of an assert, and make the checking
complete (i.e., don't just check for the existence of a base class,
also check for the existence of a ctor, recursively).
* Refactor the constructor accessor out of the evaluator and make it
a method, Ctor, on the Function abstraction.
* Add a recursive Module population case to property initialization.
* Only treat the top-most frame in a module's initializer as belonging
to the module scope. This avoids interpreting block scopes within
that initializer as belonging to the module scope. Ultimately, it's
up to the language compiler to decide where to place scopes, but this
gives it more precise control over module scoping.
* Assert against unsupported named/star/keyword args in CocoPy.
If we encounter a dynamic invocation of a prototype object, we will
interpret it as an object allocation. This corresponds to code like
import ec2 from aws
instance = ec2.Instance(...)
where the second line dynamically loads the prototype object for the
Instance class from the module object for the aws/ec2 module, and
invokes it.
This change adds support for naming imports. At the moment, this simply
makes the names dynamically accessible for languages that do dynamic loads
against module objects, versus strongly typed tokens. The basic scheme
is to keep two objects per module: one that contains the globals and its
prototype parent that contains just the exports. This ensures we can
share the same slots while attaining the desired information hiding
(e.g., when handing out an object for dynamic access, we give out this
parent objects, while when loading globals from within a module, we use
the childmost one that contains all private and exported variables).
The old model for imports was to use top-level declarations on the
enclosing module itself. This was a laudible attempt to simplify
matters, but just doesn't work.
For one, the order of initialization doesn't precisely correspond
to the imports as they appear in the source code. This could incur
some weird module initialization problems that lead to differing
behavior between a language and its Coconut variant.
But more pressing as we work on CocoPy support, it doesn't give
us an opportunity to dynamically bind names in a correct way. For
example, "import aws" now needs to actually translate into a variable
declaration and assignment of sorts. Furthermore, that variable name
should be visible in the environment block in which it occurs.
This change switches imports to act like statements. For the most
part this doesn't change much compared to the old model. The common
pattern of declaring imports at the top of a file will translate to
the imports happening at the top of the module's initializer. This
has the effect of initializing the transitive closure just as it
happened previously. But it enables alternative models, like imports
inside of functions, and -- per the above -- dynamic name binding.
Previously, it was an error to look up a local that didn't exist.
Now it is common, thanks to dynamic lookups. A few code-paths didn't
previously handle this adequately; now they do.
This rearranges the way dynamic loads work a bit. Previously, they¬
required an object, and did a dynamic lookup in the object's property¬
map. For real dynamic loads -- of the kind Python uses, obviously,¬
but also ECMAScript -- we need to search the "environment".
This change searches the environment by looking first in the lexical¬
scope in the current function. If a variable exists, we will use it.¬
If that misses, we then look in the module scope. If a variable exists¬
there, we will use it. Otherwise, if the variable is used in a non-lval
position, an dynamic error will be raised ("name not declared"). If
an lval, however, we will lazily allocate a slot for it.
Note that Python doesn't use block scoping in the same way that most
languages do. This behavior is simply achieved by Python not emitting
any lexically scoped blocks other than at the function level.
This doesn't perfectly achieve the scoping behavior, because we don't
yet bind every name in a way that they can be dynamically discovered.
The two obvious cases are class names and import names. Those will be
covered in a subsequent commit.
Also note that we are getting lucky here that class static/instance
variables aren't accessible in Python or ECMAScript "ambiently" like
they are in some languages (e.g., C#, Java); as a result, we don't need
to introduce a class scope in the dynamic lookup. Some day, when we
want to support such languages, we'll need to think about how to let
languages control the environment probe order; for instance, perhaps
the LoadDynamicExpression node can have an "environment" property.
This changes the CocoPy default load type from static to dynamic,
since we don't have enough information at compile-time to emit
fully qualified tokens. Previously, Coconut only supported dynamic
loads with object targets, however we will need to support the full
scope search (class, module, global, etc).
The translation from Pythonic underscore_casing to camelCasing is
meant to only apply to serialized AST nodes, not arbitrary map keys
inside of dictionaries within them. This change fixes the previous
behavior, which indiscriminately applied mangling throughout.
As of this change -- just a few minor fixes -- CocoPy produces real Cocopacks!
$ cocopy index.py | coco pack info -a -
// Basic example of an AWS web server accessible over HTTP.
package "webserver" {
dependencies [
aws: "*"
]
module "index.py" {
imports [
"coconut/aws"
]
exports [
".init" -> ".init"
".main" -> ".main"
"_Name" -> "__name__"
"main" -> "main"
]
method ".init": ()
method ".main": ()
property "_Name": dynamic
method "main": (): dynamic
}
}
The contents of various tokens aren't quite right yet (they must be
fully qualified), meaning the pack still won't verify and load.
But we're getting close...
This change adds a lot more aggressive AST type assertions throughout,
ensuring that we catch type mismatches as early as possible.
I realize this is borderline "un-Pythonic", but it will keep me sane.
* Refactor the options on encoding into an options object, to make
it more future proof and a little easier to deal with. (For instance,
we hadn't plumbed the individual parameter options all the way
through in all cases, an easy mistake to make when there are N of them.)
* Produce ast.TypeToken objects in a few requisite places.
* Replace the "." delimiter in Python module names with the CocoPack
equivalent, "/".
This change gets us far enough along to fully parse and lower the
Coconut package metadata for the webserver-py example. It doesn't
serialize everything properly yet (since we haven't implemented the
serializer), and so it isn't runnable, but we're getting close(r).
* Fix a few minor issues in the AST layer:
- is_* functions should accept self parameters.
- All ast.Definition nodes require a name.
- A few assertions were wrong (e.g., is_stmt vs is_statement).
* Refactor the op constants to be a little shorter. I found them
to be gratuitously verbose in calling code.
* Add a compiler.Context object for transformation passes. This
holds state specific to a given module's context, since -- as
soon as we do multi-module passes -- it must remain isolated.
* Introduce support for the special "__name__" Python variable.
This entails creating the variable and initializing it. For now,
since we only support single-module cases, it is always "__main__".
* Properly spill declarations and initialization statements in
the module visitation logic. This includes detecting global
module properties while visiting statements at the top-level,
which gives us an approximation to Python's oddly-whacky scoping
rules (emulating this perfectly is going to be a PITA, I think).
* Properly track and emit imports.
* Add some more defensive assertions throughout, but particularly
in ensuring that the right node types are flowing around
(mostly this means statements vs expressions).
* Support the following Python AST types:
- Blocks.
- Assignments.
- Module-scoped function definitions.
- Imports.
- "Attribute" access (a.k.a., property access, i.e., `x.y`).
- Function calls.
- Comparison operators.
- Names and Name IDs.
This change mostly sketches out placeholders for the various Python
AST nodes and their transformations. A smattering of them have been
implemented -- basically, those that were "trivial and obvious" upon
the first pass over this -- however, the goal of this checkin really
is to make the broad brush strokes that we will now fill in.
A few minor changes are included, like better assertions, a few
fixups to required/optional parameters, stringification and equality
checking functions, and so on.
Additionally, a bit of the logic for transforming modules is included.
For example, we shove statements into the module initializer, wire
up the entrypoint routine, etc. For the most part, this is not
functional, however, and will get a bit more robust in the next commit.
The prior code was trying to be too "clever", skipping initialization
of properties that haven't been given non-None values. This was to
tidy up serialization and ensure the default JSON/YAML behavior skipped
undefined fields. Although that basically works, it causes problems
elsewhere in the code, where certain fields would need to be lazily
initialized; this introduces a level of complexity I'd rather not have,
and furthermore, we will most likely need custom serialization anyway.
This change introduces a new CocoPy compiler. It will implement a
subset of Python and translate it into Coconut packages and IL.
The project is underneath tools/cocopy/, alongside the cocojs/ compiler
directory, and is organized into two top-level directories: cmd/ contains
the command line tools and lib/ contains the library package code.
At the moment, the compiler processes a single file at a time. The
Coconut package metadata is loaded identically to how CocoJS will discover
and load package metadata (including searching "upwards" in the filesystem
for package references).
Most of the Coconut package and AST metadata has been projected as Python
classes in the lib/ast and lib/pack modules. The only missing pieces here
are the token manipulation abstractions which, for Python, will be far less
interesting due to the absence of any strong typing. (Most likely we'll
end up with a bunch of runtime assertions instead of nice types.)
I've opted to use the pythonparser package because (a) it seems to be well-
designed, (b) it is meant for tools just like this, and (c) the Google
Grumpy compiler uses it (which at least tells me it's a safe-ish bet).
The transformer is still mostly an empty shell of placeholders.
This change makes node optional in the lookupBasicType function, which is
necessary in cases where diagnostics information isn't available (such as
with configuration application). This eliminates an assert when you fat-
finger a configuration key. The associated message was missing apostrophes.
We already had the ability to manually execute a CocoPack and generate
a DOT from its object graph. However, for demo purposes we also want
to be able to generate one from the plan. This adds a --dot flag to plan.
This change eliminates the need to constantly type in the environment
name when performing major commands like configuration, planning, and
deployment. It's probably due to my age, however, I keep fat-fingering
simple commands in front of investors and I am embarrassed!
In the new model, there is a notion of a "current environment", and
I have modeled it kinda sorta just like Git's notion of "current branch."
By default, the current environment is set when you `init` something.
Otherwise, there is the `coco env select <env>` command to change it.
(Running this command w/out a new <env> will show you the current one.)
The major commands `config`, `plan`, `deploy`, and `destroy` will prefer
to use the current environment, unless it is overridden by using the
--env flag. All of the `coco env <cmd> <env>` commands still require the
explicit passing of an environment which seems reasonable since they are,
after all, about manipulating environments.
As part of this, I've overhauled the aging workspace settings cruft,
which had fallen into disrepair since the initial prototype.
This changes the example security analyzer from acmecorp/security
to just infosec, to reinforce that we will have certain analyzers
"out of the box" (infosec, cost, etc.)
* Rename the sample from ec2instance to webserver.
* Factor out the AMI map stuff into the AWS library, rather than the sample.
* Strongly type the instance type parameter using the aws.ec2.InstanceType union.
* Add a new webserver-comp example that demonstrates a bit of the ability to do
encapsulation, componentization, and multi-instantiation.
This change adds a `coco plan` command which is simply a shortcut
to the more verbose `coco deploy --dry-run`. This will make demos
flow nicer and elevates planning, an important activity, to a more
prominent position. The `--dry-run` (aka `-n`) flag is still there.
This change also renames `coco env destroy` to just `coco destroy`.
This is consistent with deploy and plan being at the top-level. We
now use `coco env` purely for evironment management commands (init,
config, rm, etc).
