{
  "path": "/posts/2024/structure-and-functions-and-prompting",
  "site": "at://did:plc:mracrip6qu3vw46nbewg44sm/site.standard.publication/self",
  "tags": [
    "openai",
    "language_models",
    "function_calling",
    "structured_data"
  ],
  "$type": "site.standard.document",
  "title": "Structured Output, Functions and Prompting",
  "updatedAt": "2024-08-12T21:15:27.000Z",
  "publishedAt": "2024-08-12T21:15:27.000Z",
  "textContent": "I've been prompting models to output JSON for about as long as I've been using models.\nSince text-davinci-003, getting valid JSON out of OpenAI's models didn't seem like that big of a challenge, but maybe I wasn't seeing the long tails of misbehavior because I hadn't massively scaled up a use case.\nAs adoption has picked up, OpenAI has released features to make it easier to get JSON output from a model.\nHere are three examples using structured outputs, function calling and just prompting respectively.\n\njson\", \"\").replace(\"`\n\nRunning the script (output objects truncated) yields\n\nDon't read too much into the time durations.\nAfter running the script a few times, all approaches seem to take 4-6 seconds, with none clearly faster than the others.\nThe quality of the extraction seems to be around the same for all approaches for this use case as well.\n\nFunction calls are pitched as a solution to allow you to \"connect models like gpt-4 to external tools and systems\"[^1].\nStructured outputs are supposed to be for \"[g]enerating structured data from unstructured inputs\"[^2].\nThe latter is an improvement on \"JSON mode\" apparently introduced in 2023 that I never tried.\nThe bottom line is we need these models to respond with structure, but we don't want the imposition of this structure to detract from the model's performance.\n\nI need to do some testing to see if the quality of the model's response varies depending on which approach is used.\n\n[^1]: https://platform.openai.com/docs/guides/function-calling\n\n[^2]: https://openai.com/index/introducing-structured-outputs-in-the-api",
  "canonicalUrl": "https://www.danielcorin.com/posts/2024/structure-and-functions-and-prompting"
}