PHP has a nice feature whereby if you have a variable called $foo (all scalars in php start with the "$" sigil) and you want to output a string containing that value, you can just write:

echo "The current foo value is $foo units.";

In other words, you don't have to close the string to insert a variable into it. If this feature didn't exist you'd have to write:

echo "The current foo value is " . $foo . " units.";

as in C++:

cout << "The current foo value is " << foo << " units.";

which doesn't seem like much more typing, but when you're writing web pages (which are full of these inserts) it really adds up and makes the result much more pleasant to use. I was skeptical at first but after having written some PHP code I want to steal this feature for my own language. Being a statically typed language, Unity also inserts .toString() calls when the inserted variable is not of String type.

Inserting more complex expressions is also possible. Though I think PHP's syntax for this is a bit confusing:

"<div class='logged_in_as'>Logged in as: {$user->link()}</div>"

Much better to have a single insertion character and allow what follows to be either a variable name or a parenthesized expression:

"<div class='logged_in_as'>Logged in as: $(user.link())</div>"

The trouble with including literal strings like this in your program, though, is that sooner or later you're going to want to translate it into other (human) languages. Current GNU C/C++ best practices for internationalizable software suggest enclosing UI strings in a gettext macro (_("foo")) which adds that string to a table and replaces the string itself with an index into that table - then localization is just a matter of replacing the table. At Microsoft we avoided putting UI strings in source code altogether, instead putting them in an .rc file and making up a macro name to refer to the string's index. That was painful, because to add a string we had to change three files - the source file (relating code to macro name), the header file (relating macro name to index) and the resource file (relating macro name to English text). I suppose there might have been economic forces resisting change to this system - Microsoft software is translated into so many languages that adding a UI string is quite expensive - you have to pay to have it translated a lot of times. Similarly with changing a UI string.

Both these methods also constrain how you need to phrase your UI strings. The canonical example is that you can't write:

printf(_("%i file%s copied"), n, n==1 ? "" : "s");

because the rules for localization vary from language to language. Translating would involve changing the code as well as the contents of the string. Thus such pieces of UI usually get phrased as:

printf(_("Files copied: %i"), n);

instead.

I want to include a localization system right in the Unity compiler which solves all these problems. The high level aims:

• It should be easy to add a string with inserts - as easy as it is in PHP. Modifying strings should be similarly easy - no editing multiple files, no _(), no making up names for the strings or finding the next available ID number.
• The system should keep track of strings between compiles so that they can be consistently associated with their translations.
• Translators should be able to permute, duplicate, add, remove and modify inserts as appropriate for the target language.
• Software designers should not have to compromise their designs in the original language to make it possible to translate them into other languages - in other words there is no need to explicitly internationalize strings - it happens automatically (of course, there are non-string internationalization issues which are not in scope here, such as staying far, far away from maps.

To accomplish this, I think the best way is to have a tool (or perhaps a special mode of the compiler) which modifies the source files it processes. This is generally considered a bad thing but in this case I think it's the best solution. The one and only change it makes is to add an identifying number to the start of each string. The insert system gives a handy way to do this, so it might change:

"<div class='logged_in_as'>Logged in as: $(user.link())</div>"

to:

"$(/*12345*/)<div class='logged_in_as'>Logged in as: $(user.link())</div>"

If you're a programmer changing this string, you now have a choice - you can delete the "$(/*12345*/)" part, causing the translators to consider this a brand new string to be translated from scratch, or you can leave it alone, causing the translators to consider it a modification rather than a deletion and addition. The IDs are only ever added by this tool - no programmer should ever have to try to figure out which is the next available number by hand. It might be a good idea for the version control system to be made aware of this system so that if two strings are added with the same number on different branches and the branches are then merged, the merge system knows how to change one of the IDs automatically (everywhere that it appears). Similar empty inserts with comments can be used to leave notes for translators, such as telling them the context in which a particular string is used.

For non-UI strings, the programmer and/or translator can change the ID to a dummy so that it's clear to all concerned that the string does not need to be translated:

"$(/**/)<html>"

The interface for translators should be something like this. The tool creates one file (the translations file) for each generated binary for each language the software is translated into. This file contains one line per string in the original source, and that line contains the original string (including ID and comments), the translated string, and any notes from the translator. The compiler then reads this generated file back in to generate translation files for each language/binary combination. The translator can either edit this file directly or use some kind of tool, but either way it just gets checked back into the version control system.

The tool also checks for differences between the translations file and the source, and tells the translator which strings have been added and changed (including where the string appears in the original source, so the translator can look up the context). Because the translation file contains the inserts as source (the variable name or expression after the $) the translator is free to change this code to make sense for other languages. To handle the plural case, the string might be written as:

print("$(/*120*/)$n file$(n == 1 ? "" : "s") copied");

and the translator can then change anything inside the outermost double-quotes to make the string make sense for the target language. That might even involve calling out to external code to handle complicated cases:

"$(/*120*/)$n file$(n == 1 ? "" : "s") copied", "$n $(locale.plural(n, "plik")) kopiowane"

An important difference here is that the compiled locale file can now contain code as well as data. This may introduce testing difficulties, but it's necessary to provide this "no compromise to end user experience" goal. If necessary, site policy could dictate the software internationalization rules we usually use now, or some middle ground like ensuring all locale methods are pure functions returning in bounded time, so they can't adversely affect other parts of the software.

A type constructor is a generalization of a type. It's either a type itself (which can be used to declare variables) or a template type constructor, a "function" of sorts which, when given another type constructor, yields a third type constructor. C++, C# and Java programmers are familiar with these "functions" - they're called templates or generics. A good example of a template type constructor is Array. You give it a type like String and you get back Array<String> (i.e. an array of strings). You can also have Array<Array<String>> if you're feeling adventurous. A type constructor can have multiple arguments, so you might have Pair<Integer, String> which (through the magic of something called currying) is the same thing as Pair<Integer><String>.

What if you have a template type constructor with a parameter that is not a type but is itself a template type constructor? Well, you can do that too - say you have a type constructor called FooCollection and you pass it Array (note, not Array<Integer> or Array<String>, which are types, just plain Array) and you get back FooCollection<Array> which might be implemented in terms of Array<Foo>. Similarly, FooCollection<List> might be implemented in terms of List<Foo>.

So if the type of the argument of square is Integer, how do we talk about the argument of FooCollection. We can't say "the type of the argument of FooCollection" because what it accepts isn't a type - a type constructor can't have a type, type is a kind of type constructor. And in fact that is exactly the word we use instead: "the kind of the argument of FooCollection is <>." Other examples of kinds are (nothing, i.e. the empty string) which is the kind of types like Integer and String (no angle brackets follow them), <><> aka <,> (which is the kind of Pair - i.e. you need to give it two types to get back a type) and <<>> (which is the kind of FooCollection - i.e. you need to give it a type constructor of kind <> to get back a type). The wikipedia article on kinds uses a different notation further removed from the C++-inspired one I use here, but the two are equivalent and the transformation from one to the other is purely mechanical.

Incidentally, you can refer to kinds in C++ as well - instead of the empty string for the kind of types, C++ uses "class" (a rather confusing overload of the word - distinct from "a structured type with default accessibility private") or "typename" which is a bit better. Instead of <>, C++ uses "template<typename>", instead of <><>, C++ uses "template<typename, typename>" (C++ has no currying) and instead of <<>> C++ uses "template<template<typename>>" (a so-called "template template parameter" - one of the more obscure areas of C++).

So we've gone from values to types to kinds, what comes next in the sequence? ("Sort?" that already has another meaning in computer science though.) Suppose you had a function which accepts a kind and returns another kind. My angle brackets here have that effect - they take a list of kinds and return a kind which is the kind of a type constructor that accepts type constructors of the listed kinds and returns a type. What a confusing sentence! Fortunately nobody to my knowledge has found a non-trivial use for anything one level up from kind. I say "fortunately" not just because it's really confusing but because we've run out of notation! If you start with a lower case letter to signify a value argument and an upper case letter to signify a type constructor argument, how do you signify a kind argument? Also, if we'd need to use different brackets (I almost wrote "a different kind of brackets" which would further confuse things - writing about this stuff is hard!) to distinguish kind functions from value functions and type constructor functions, and all the ASCII brackets already have other meanings in my language.

Anyway, this is the sort of thing that you bump your head against when you write compilers for languages which support templates. Fortunately, if you're just writing normal applications in this language you'll probably never need to be concerned with it. Until you do. At which point just doing the most obvious thing should work fine without any fuss. I think it's fun to think about though.

The "Multiple break" example in the "More keywords" blog post is a piece of code that can't be de-GOTOized in C. I'm wondering if there are any examples of pieces of code that can't be de-GOTOized without multiple break, extra variables or code duplication. If I understand the abstract of Peterson 1973 correctly I think there aren't - that multiple break is enough to make GOTO redundant (but I can't find a non-paywalled copy of that article to check, and I don't care enough to spend $15 to find out).

Another sticking point seems to be there's a lot of stuff written about Haskell in the form of academic papers (with all the academic jargon that entails). That shouldn't be a surprise (since it's a language with origins in academia which seems to be only reluctantly escaping to industry), but it is kind of interesting that there's a sort of language barrier between academia and industry - what the former calls "deforestation hylomorphism" might be called "tree flattening" by the latter, for example.

But I think the real problem is something shared by other functional languages - it's just a different level of abstraction than we're used to thinking about. In imperative programming languages programmers can think "okay, what do I want the machine to do here?" and write that code. In functional programming languages one instead has to think about what the inputs and outputs are, write functions to perform those transformations and trust that the compiler will optimize it well (to a first approximation). It's much more like doing mathematics than imperative programming. Compilers will do all sorts of crazy things to turn those pure functions into high-performance imperative code. So when things are inevitably too slow, it's not obvious why (since the relationship between the generated code and the source code is very complicated) and it's difficult to understand how to make it faster.

Functional programming languages do have some interesting advantages over imperative languages, though - it's much easier to reason about pure functions than about imperative programs with all their attendant state, which means lots more interesting automatic optimizations are possible and (increasingly importantly) the code can be automatically parallelized so it takes maximum advantage of modern multi-core CPUs.

I'm not sure which side to place my money on. I suspect both imperative and functional paradigms will continue to exist for the foreseeable future. On the imperative side: the low-level OS and runtime components must can't be written using functional languages, and optimizations and multi-core abstractions for imperative languages will continue to improve. On the functional side, compilers will require less and less hand-holding to generate good code and will generate better code than imperative compilers for more and more situations.

One possibility might be to give the code-generation library the freedom to compile whenever it likes, so it can try various things and run with what works best.

That this gives an error about redeclaring i is just plain dumb:

  for (var i:int = 0; i < 10; ++i) { foo(i); }
  for (var i:int = 0; i < 10; ++i) { bar(i); }

It means I have to declare all my variables at the top, old-fashioned-C-style, or spend ages fixing up the declarations whenever I move code around.

I hate it when compilers enforce a "one class per file, with the same name" rule. I'd rather have my source files organized for the convenience of humans, not for the convenience of machines, thank you very much - if I have a bunch of small related classes, it's much better to have them in the same file.

No stack objects means that if you want to want to write a function that returns multiple values (e.g. a function that multiplies a matrix by a vector and returns a vector) you have to allocate an object to put the vector components in, which is very slow. For reproject I ended up just doing everything with plain values in a small number of functions - in C++ the code could be much more well-layered.

However, this sort of thing might be more useful as a run-time process - i.e. a JIT compiler. The system can monitor which bits of the code are being run most often (this is very easy to do - just interrupt at a random time and see what code is being run) and concentrate optimization efforts on those parts. The compilation can continue (gradually making the program faster and faster) until the point of diminishing returns is reached. I understand there's a Java Virtual Machine which can do this.