A first look at token efficiency

Asside token consumtion you can for example measure more PLT oriented questions: * task completion speed, is a programming language better at solving problems faster? * because we can re-run the tasks for example 30 times rather easily you get more reliable results. * human intervention (or lack thereof, fewer are better). Interesting if you’d start desining a language with an LLM as the primary user in mind, now you could decide to add or remove features based on how good it empirically performs with those features.