Software Engineering FTW!

Sorry about the delay. After the success in finally getting partial specialization working, I figured that it would be a good time to do a little cleanup. You know, address some of the tech debt I incurred hacking away before moving on to another feature that would be more complicated than I anticipated.

First on the list as you might expect was a lack of unit tests. Unfortunately, the stuff I wanted to test (code generation) doesn't exactly lend itself to unit testing well. For simplicity, the compiler emits directly to files. For even basic cases, the setup is a bit onerous to reproduce a sane intermediate structure. And the super basic cases aren't the ones that have been causing problems - it's been when things start interacting.

So I decided to do the next best thing, automated integration tests. I took all of the little test programs I've written (and posted here) and put them into a csproj along with their expected output. A little bit of code to kick off the compiler and run the resultant temp file executable, and I had a nice little regression suite.

You can probably guess what happened next. Yeah, half the tests failed.

Some were super quick to fix (typos during test creation). Some were kinda quick to fix (functionality I had removed). A few were slow to fix (wait, that should work...). And one took about a month and a half of intermittent spare time to fix. It was the basic Algebraic Data Type code:

int list :> int | (a: int),(b: int list) {
  (this).head => int { a }
  (this).tail => int list { b }
  print (this) => void {
    print this.head;
print this.tail;
  }
}

print (il: int list) => void {}

entrypoint => void {
  print 1,2,3;
}

It first blew up because of ambiguity errors between print(int list) and print(_anonymousProductType1). That ended up just being some overzealous refactoring on my part during the partial specialization work. I had removed the fanning out of implicit conversions for sum types, so the compiler didn't see the two functions as a specialization.

The second problem was more troublesome. Because of partial specialization fixes, the relationship between int list and _anonymousProductType1 was tighter. Under the covers, int list inherits from Variant<int, _anonymousProductType1>, and _anonymousProductType1 has a constructor that takes an int and an int list. That cyclic reference was biting me in the ass. The compiler was written to take an intermediate type representation and compile up a .NET type for it. I couldn't quite get the ordering right to build most of one type and then the other so that reflection didn't choke on stuff that was half-built.

In the end, I ended up having to take a bit of the nuclear option and change how the code generation worked. Now the compiler will populate all of the TypeBuilders and pretty much always work with half-built types until function generation at which point it uses AppDomain events to build the types in the right order. (taken from this SO answer) I think that this may cause me problems when types get more member methods, and it has made stuff a bit messy.

But at least now all my tests pass and things are stable again. Next on the docket is likely local variables.