{
  "$type": "site.standard.document",
  "canonicalUrl": "https://unnecessary.tech/posts/no-language-model",
  "path": "/posts/no-language-model",
  "publishedAt": "2023-07-07T13:49:52.000Z",
  "site": "at://did:plc:jx54v4rmscfwzit7fmgz24ba/site.standard.publication/3mnrsqmzz3w2e",
  "tags": [
    "programming",
    "ai",
    "thoughts"
  ],
  "textContent": "The latest tech fad appears to be generative AI and ChatGPT in\nparticular. As with many technologies, this fad has spawned a lot of concern.\nSome of these concerns are justified,\nwhile some are hyperbolic. Unfortunately most people have a very\nlimited understanding of the technology involved, and that leads to fear, panic,\nand misuse. The press,\nwhich should provide unbiased information is instead amplifying panic, making rational discussion\nof the technology difficult.\n\n{{< image src=\"https://imgs.xkcd.com/comics/machine_learning.png\" alt=\"Comic with two people, one is on a pile of data, and the other addresses him: 'This is Your machine learning system?' The other replies, 'Yup! You pour the data into this big pile of linear algebra, then collect the answers on the other side.' 'What if the answers are wrong?' asks the first person. 'Just Stir the pule until they start looking right.' Courtesy xkcd.com\" width=\"400px\" height=\"474px\" href=\"https://xkcd.com/1838/\" >}}\n\nWhen I first used ChatGPT it reminded me of a much simpler algorithm developed\nover 100 years ago, Markov chains.\nA Markov chain is a stochastic way of building a sequence of items using existing\nsequences. The next element in a sequence is chosen by looking at the last N members\nof that sequence. We then look for that same N sequence pattern in the existing\nsequences and randomly choose one. We then add the next item from that existing\nsequence to the sequence we are generating. To generate text we can create a\nsequence of characters and pick a group of source texts as our existing sequences.\nThe surprising thing about this algorithm is it produces text that looks similar\nto the input texts, producing actual words, mimicking punctuation, and at first\nglance looking fairly passable.\n\nChatGPT is much more advanced, but the fundamentals of the process are somewhat\nsimilar. ChatGPT consumes source text and generates new text using a stochastic\nprocess based on the existing text. Neither algorithm has any understanding of\nthe text they ingest or produce. Neither algorithm is creative or purposeful in\nproducing the text. Neither algorithm is a true thinking machine, or general AI.\nThe Markov chain obviously produces meaningless babbel, but ChatGPT is\nadvanced enough to create gramatically correct sentences and paragraphs that do\nnot wander topic to topic. This can make it seem like ChatGPT is producing objectively\ngood content, but this is not true. Just like Markov chains, ChatGPT regurgitates\npieces of the text it ingested, and even if what it ingests is vetted and true,\nit can get twisted into nonsense by the algorithm.\n\nIn order to explore this a bit, I decided to create a site\nwhich uses Markov chains to produce articles based on articles about ChatGPT and AI.\nThe code is available to demystify the process, and\nthe results though unreliable can sometimes be fairly amusing. Generative AI in all\nits forms is random iteration over human works. There is no soul, no creativity, and\nnot thought involved. It is a remarkable tool, and I am sure it will find uses, but\nit is not the existential threat some people make it out to be.",
  "title": "No Language Model"
}