{
  "$type": "site.standard.document",
  "content": "---\ntitle: \"Hosting a genAI trivia night\"\ndescription: \"Using an LLM to generate trivia questions, then letting teams challenge any\n  answers they think the model hallucinated.\"\ntags:\n  - ai\n---\n\nI was recently tasked with organising a trivia night, and decided to generate\nall the questions (and answers) with a large language model (I used\n[Claude](https://claude.ai/), although obviously this would work with any model).\n\nHere's the initial prompt I used:\n\n> Write a set of questions (10 rounds, 5 questions per round) for a trivia\n> night, including answers. Each round must have a different theme, including\n> rounds on the topics of \"_insert list of rounds here_\". You must provide\n> questions which have a single, unambiguous correct answer. Include a mix of\n> easy and difficult questions, such that a graduate-level audience would get\n> approximately 50% of the answers correct.\n\nLooking over the answers, they looked a little too easy, so I provided a\nfollow-up:\n\n> Those questions are all too easy. Try again, and dial up the difficulty.\n\nwhich gave questions which looked (to my eyes) to be around the right level.\n\nNow, LLMs are notorious for hallucinating/making up facts, and I couldn't be\nbothered to check that all the answers were correct. So I incorporated this \"is\nthe LLM making stuff up?\" dynamic into the rules. As well as the usual trivia\nnight procedure:\n\n- 1 pt per question\n- each question will be read **twice** (no more than that)\n- we'll give answers and tally scores after each round\n- no cheating (internet _or_ AI models, inc. self-hosted ones)\n\nthere was an additional rule: at the end of each round, each team can challenge\nany answer(s) they think the LLM got wrong. For each challenge, the trivia hosts\nwould investigate (using the internet, or whatever) to see what the correct\nanswer is.\n\n- if the LLM's answer was wrong, _all teams_ have that question re-marked with\n  the correct answer\n- if the LLM's answer was ambiguous (i.e. it was correct, but there are other\n  answers that were equally correct) then _all teams_ have that question\n  re-marked, accepting any of the correct answers\n- if the LLM's answer was correct (or if we can't find a reliable answer in an\n  appropriate timeframe) then the question is not re-marked, and the challenging\n  team receives an additional one-point penalty\n\nAs for whether the LLM was correct/ambiguous/wrong, all decisions by the trivia\nhosts were final.\n\nIt went pretty well overall, though in the end the questions were a bit too\ntricky. Turns out it's really hard to eyeball questions _with_ answers to guess\nhow many you'd get correct, so if you're going to do that make sure you do it\nwithout looking at the answers.\n\nThere was one successful challenge on the night, but overall there didn't seem\nto be too many hallucinations. In some ways it would have been more fun if there\nwere.\n\nAnyway, if you need to organise a trivia night and don't want to do any\npainstaking research, give the above prompts a try.\n",
  "createdAt": "2026-05-13T23:14:45.315Z",
  "description": "Using an LLM to generate trivia questions, then letting teams challenge any answers they think the model hallucinated.",
  "path": "/blog/2024/08/12/hosting-a-genai-trivia-night",
  "publishedAt": "2024-08-12T00:00:00.000Z",
  "site": "at://did:plc:tevykrhi4kibtsipzci76d76/site.standard.publication/self",
  "tags": [
    "ai"
  ],
  "textContent": "Using an LLM to generate trivia questions, then letting teams challenge any answers they think the model hallucinated.",
  "title": "Hosting a genAI trivia night"
}