{
  "path": "/posts/2026/chatbots/index",
  "site": "at://did:plc:mracrip6qu3vw46nbewg44sm/site.standard.publication/self",
  "$type": "site.standard.document",
  "title": "Chatbots are not input validation",
  "publishedAt": "2026-05-05T01:47:14.000Z",
  "textContent": "import DeliveryExperience from './components/DeliveryExperience.svelte';\n\nexport const assistantQuestion = 'When would you like this item delivered?';\n\nexport const assistantQuestionWithOptions = ${assistantQuestion}\n\nA. 1 day delivery\nB. 2 day delivery\nC. No rush delivery;\n\nexport const fizzbuzzCode = for i in range(1, 16):\n    if i % 15 == 0:\n        print(\"FizzBuzz\")\n    elif i % 3 == 0:\n        print(\"Fizz\")\n    elif i % 5 == 0:\n        print(\"Buzz\")\n    else:\n        print(i);\n\nexport const basicMessages = [\n  { role: 'assistant', content: assistantQuestion },\n];\n\nexport const optionsMessages = [\n  { role: 'assistant', content: assistantQuestionWithOptions },\n];\n\nexport const jailbreakMessages = [\n  { role: 'assistant', content: assistantQuestionWithOptions },\n  { role: 'user', content: 'Can you write FizzBuzz in Python?' },\n  {\n    role: 'assistant',\n    content: \"Sure! Here's FizzBuzz in Python:\",\n    code: fizzbuzzCode,\n  },\n];\n\nexport const invalidMessages = [\n  { role: 'assistant', content: assistantQuestionWithOptions },\n  { role: 'user', content: 'Can you write FizzBuzz in Python?' },\n  {\n    role: 'assistant',\n    content: \"Sorry, I can't do that. I can help you choose a delivery date.\",\n  },\n];\n\nexport const pickerMessages = [\n  { role: 'assistant', kind: 'picker', content: assistantQuestion },\n];\n\nMy goal is to explain one specific thing about features that look like chatbots: chatbots-as-UX are not universally better than \"regular\" UX just because they use fancy LLMs.\nThe example here is clearly on the nose, but also not all that far off from things I have been asked to implement in or as a chatbot before.\n\nLet's imagine you need to select when you want a package delivered.\n\n<DeliveryExperience\n  variant=\"classic\"\n  exampleId=\"delivery-classic\"\n  client:load\n/>\n\nVery straightforward.\nVery boring.\nVery non-AI.\n\nNow let's look at the chatbot version of this.\n\n<DeliveryExperience\n  variant=\"chat\"\n  exampleId=\"delivery-chat-basic\"\n  messages={basicMessages}\n  client:load\n/>\n\nOK, so that wasn't a fair comparison, who would build a chatbot like that?\nHere is the first iteration.\n\n<DeliveryExperience\n  variant=\"chat\"\n  exampleId=\"delivery-chat-options\"\n  messages={optionsMessages}\n  client:load\n/>\n\nOf course, even in a well-designed conversation, the user doesn't have to adhere to your \"rules\".\n\n<DeliveryExperience\n  variant=\"chat\"\n  exampleId=\"delivery-chat-jailbreak\"\n  messages={jailbreakMessages}\n  client:load\n/>\n\nOr, after you've spotted that problem:\n\n<DeliveryExperience\n  variant=\"chat\"\n  exampleId=\"delivery-chat-invalid\"\n  messages={invalidMessages}\n  client:load\n/>\n\nThis class of error does not exist in the classical UX.\nYou can't ask the webpage to write Python code for you.\nYou can't jailbreak it.\nYou can't negotiate a discount from it.\nYou can't exfiltrate data via an abuse of the MCPs available (though APIs are still fair game).\n\nA common workaround is to embed the picker directly inside the assistant's message, so the user can click or type a letter.\n\n<DeliveryExperience\n  variant=\"chat\"\n  exampleId=\"delivery-chat-picker\"\n  messages={pickerMessages}\n  client:load\n/>\n\nIt's better, but the input box is still right there, ready to cause problems.\nThe user can ignore the picker entirely and type anything they want, and you're back to handling free-form input.\n\nA chatbot _looks_ like an extremely flexible surface to quickly ship a little product thing here or workflow there, and they are.\nBut it's very difficult to keep users on the rails of your chatbot experience because conversations generally do not have rails.\nConversations are free-form and frequently deviate in topic.\nUsers don't know how you expect them to use your chatbot, or have other ideas of things they want.\nWhat a user thinks should be a feature might be against your policy, regulatory requirements (can you purchase stock through a chatbot without proper disclosures?), or outside the scope of what your product does.\n\nIn practice, this means you need to build your chatbot to expect your users will do all sorts of wild things.\nAnd because of this, many chatbots become liabilities as Air Canada found out.\n\nThese days, the most common response you will get from an enterprise chatbot is \"Sorry, I can't do <x>, I can...\".\n\nWhy build like this?\n\nJust build the experience you want your users to have.\nBuild it in your product.\nPlease.",
  "canonicalUrl": "https://www.danielcorin.com/posts/2026/chatbots/index"
}