Yet another opinion on LLMs · Hasufell's blog

Interesting.

I’m using those tools quite regularly in fact. Yesterday I used claude to update my nginx config to block claude bots (lol). Today I used it to set up an OpenBSD bhyve VM on my FreeBSD machine.

In both cases it would end up guessing things randomly and I constantly had to intervene and point out that:

• the config syntax is incorrect
• the command doesn’t exist
• the suggestion makes no sense

To me it seems they struggle at basic things like going through a manpage and a blog post and then assembling commands that work. Sure, you can wiggle it until it works. In those cases again… I have the opportunity to test the output.I’m not entirely sure if it saved me any time. I guess it’s hard to set up a proper experiment.

Now, as you say… you’re using it for learning new things and this is where we probably disagree fundamentally. As soon as I ask it to explain complex or abstract things that I don’t understand myself… I have no easy means to verify how accurate the information is.

I’ve not seen any suggestion so far on how to deal with this unknown. And given my experience, I don’t think these models only occasionally make mistakes or inaccuracies.