Anti-LLM Sentiment Considered Harmful

One of my main issues with LLMs is that even if the incidence of bullshitting might be less than the average person (a claim I don’t necessarily support), I expect a machine to be 100% deterministic and consistent. When GHC has a soundness issue, it’s a big deal.

Claude just gave me the following (emphasis mine):

• Claude: X does not currently support … (https://…)
• me: The reference to https://… is hallucinated
• Claude: You’re right — I made that URL up. The system prompt explicitly says not to guess URLs, and I did exactly that. That’s a real failure on my part.

I beg your unbelievable pardon? The future of everything is lies, I guess.

I am torn here. LLM agents can increase productivity for specific tasks – e.g. scaffolding a new package, or filling out the details of the 1000th HTTP endpoint given a new Servant-defined endpoint – but ultimately, we’re trying to fit a square peg in a round hole. Next-token prediction is the wrong foundation for software engineering. The fact that it works at all is but an emergent behavior of tons of computing power and data.

We, as engineers and reviewers of software, must remain vigilant.This has pushed me to encode as much as possible using types when LLMs are involved, at the cost of making Haskell code much more complex. That way, LLM agents can get a reality check from my trusty Haskell toolchain. The immediate price to pay is that there’s a lot of machinery that would be impenetrable to a junior engineer.

My conclusions at this time:

• LLMs make me more productive in certain contexts;
• LLMs are inherently statistical, and are inherently limited;
• Reviewing has become both harder, and more important;
• Junior engineers are in deep trouble