In the last part we created a very basic DSL for use with turtle graphics. Today we’re going to extend the language with more capabilites and learn more about how to use Piglet and also incidentally a bit of compiler construction (nothing scary, I promise).
This is the DSL we ended up with last time:
pendown
move 50
rotate 90
move 50
rotate 90
move 50
rotate 90
move 50
penup
move -10
rotate 90
move 10
pendown
move 30
rotate 90
move 30
rotate 90
move 30
rotate 90
move 30
One obvious thing here is that it would be really nice if we could have variables. We could represent the constants with something else and perform calculations on them. An important thing is to consider the syntax, for a simple DSL we want something that is easy to parse. For this reason we’ll use a var keyword to make variable declarations and require all variables to start with a $. Let’s modify the existing program to include our new language features:
var $size = 50
var $innersize = 30
pendown
move $size
rotate 90
move $size
rotate 90
move $size
rotate 90
move $size
penup
move -10
rotate 90
move 10
pendown
move $innersize
rotate 90
move $innersize
rotate 90
move $innersize
rotate 90
move $innersize
Looking at the new DSL, we note that there is now a new legal statement to make – the var declaration. So we need to extend statement with another rule. The statement list is already pretty long, so we separate it out into another rule:
// Runtime stuff
var variables = new Dictionary<string, int>();
var variableIdentifier = configurator.Expression();
variableIdentifier.ThatMatches("$[A-Za-z0-9]+").AndReturns(f => f.Substring(1));
variableDeclaration.IsMadeUp.By("var")
.Followed.By(variableIdentifier).As("Name")
.Followed.By("=")
.Followed.By<int>().As("InitialValue").WhenFound(f =>
{
variables.Add(f.Name, f.InitialValue);
return null;
});
Since we want something that matches not a single constant string, we need to make an expression. Expressions are made using the configuration.Expression() method. You give it a regex and you get back an object. You also need to make a function to return a value for the expression. What we want is the variable name without the $ at the start.
To keep track of what variables we have and what values they have, we add them to a dictionary of strings and integers which we then can reference.
We add this new rule to our choices for statementList by adding this to the end:
.Or.By(variableDeclaration)
Note that our rule has just a single component we do not need to add a WhenFound method. It will automatically return the value of the function returned by the single value. Not that we’re using the values returned yet, but we will.
Continuing on, we want to use the variables for the rest of the commands. In order to do that we need to make another rule. Let’s call it expression. For now, a valid expression is either a constant int value, or a variable name. We then use the new rule instead of plain ints for the move and rotate commands. Here’s the full listing that supports the language that we set out to support so far:
// Runtime stuff
var turtle = new Turtle(canvas1);
var variables = new Dictionary<string, int>();
// Parser configurator
var configurator = ParserFactory.Fluent();
var statementList = configurator.Rule();
var statement = configurator.Rule();
var variableDeclaration = configurator.Rule();
var expression = configurator.Rule();
var variableIdentifier = configurator.Expression();
variableIdentifier.ThatMatches("$[A-Za-z0-9]+").AndReturns(f => f.Substring(1));
variableDeclaration.IsMadeUp.By("var")
.Followed.By(variableIdentifier).As("Name")
.Followed.By("=")
.Followed.By<int>().As("InitialValue").WhenFound(f =>
{
variables.Add(f.Name, f.InitialValue);
return null;
});
expression.IsMadeUp.By<int>()
.Or.By(variableIdentifier).As("Variable").WhenFound(f => variables[f.Variable]);
statementList.IsMadeUp.ByListOf(statement);
statement.IsMadeUp.By("pendown").WhenFound(f =>
{
turtle.PenDown = true;
return null;
})
.Or.By("penup").WhenFound(f =>
{
turtle.PenDown = false;
return null;
})
.Or.By("move").Followed.By(expression).As("Distance").WhenFound(f =>
{
turtle.Move(f.Distance);
return null;
})
.Or.By("rotate").Followed.By(expression).As("Angle").WhenFound(f =>
{
turtle.Rotate(f.Angle);
return null;
})
.Or.By(variableDeclaration);
A few interesting possibilities here. It’s perfectly acceptable to replace the int for the variable declaration. In fact, it’s a pretty good idea. This enables us to write things like this:
var $foo = 42
var $bar = $foo
What we really want however is to be able to write this:
var $size = 50
var $spacing = 10
var $innersize = $size - $spacing * 2
We’re halfway there already. What is needed is to create some more rules for how an expression can be made. We start simple, we allow only a single addition. Separate what is today an expression into a new rule called term, and make a new rule for additionExpressions.
expression.IsMadeUp.By(additionExpression);
additionExpression.IsMadeUp.By(term).As("First").Followed.By("+").Followed.By(term).As("Second").WhenFound(f => f.First + f.Second)
.Or.By(term);
term.IsMadeUp.By<int>()
.Or.By(variableIdentifier).As("Variable").WhenFound(f => variables[f.Variable]);
So, an addition expression is either an add, or a single lone value. This works fine as long as you dont try something like var $a = $b + $c + 100. This is easily fixable by changing the first term into an additionExpression. We are wiring the rule to itself! And like magic we can have as many additions in a row as we’d like. It’s trivial to add a subtraction expression as well, duplicate the rule, and change the operator and the WhenFound function to subtract instead of add (renaming to addSub instead to reflect the new functionality).
addSub.IsMadeUp.By(additionExpression).As("First").Followed.By("+").Followed.By(term).As("Second").WhenFound(f => f.First + f.Second)
.Or.By(additionExpression).As("First").Followed.By("-").Followed.By(term).As("Second").WhenFound(f => f.First - f.Second)
.Or.By(term);
However, if we add a multiplication and division here things will start to get strange. Everything will be evaluated left to right, so when writing var $a = 2 + 3 * 10 $a will have the value 60 instead of the expected 32. In order to solve this we need to redefine term. A term will now be a multiplication or division expression or a single factor that is a constant. We also wire this rule to itself like the addition rule so we can do many multiplications in a row. The expression grammar now looks like this:
expression.IsMadeUp.By(addSub);
addSub.IsMadeUp.By(addSub).As("First").Followed.By("+").Followed.By(mulDiv).As("Second").WhenFound(f => f.First + f.Second)
.Or.By(addSub).As("First").Followed.By("-").Followed.By(mulDiv).As("Second").WhenFound(f => f.First - f.Second)
.Or.By(mulDiv);
mulDiv.IsMadeUp.By(mulDiv).As("First").Followed.By("*").Followed.By(factor).As("Second").WhenFound(f => f.First * f.Second)
.Or.By(mulDiv).As("First").Followed.By("/").Followed.By(factor).As("Second").WhenFound(f => f.First / f.Second)
.Or.By(factor);
factor.IsMadeUp.By<int>()
.Or.By(variableIdentifier).As("Variable").WhenFound(f => variables[f.Variable]);
As the final thing for today, let’s add support for parenthesis. They go in the factor rule. So, a factor can be an entire expression wrapped in parenthesis. It’s almost a thing of magic. We wire the entire last rule back up to the start.
factor.IsMadeUp.By<int>()
.Or.By(variableIdentifier).As("Variable").WhenFound(f => variables[f.Variable])
.Or.By("(").Followed.By(expression).As("Expression").Followed.By(")").WhenFound(f => f.Expression);
This is where we’ll end for this part. We have a fully functional expression parser. You can use expressions for all the commands that previously only took integer values, and you can assign values to variables. A few things for the adventurous to try:
- Add variable assignment. Make
set $foo = 1+3*$bar a legal, working piece of code
- Add support for the modulus operator
- Add support for unary minus.
var $foo = -$barM work. This is a bit tricky, and don’t be afraid if you get some scary exceptions from Piglet
Next time, we’ll make the DSL a bit more fun. We’ll add some basic flow control! Code on github is updated with the latest version
Binary sculpting 