{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreid4ff2dmqkticwfvgfwmhf5ymladjtmhiyvcfe7frycwarpcip6ci",
    "uri": "at://did:plc:lk3jfj3zq4k4wxnk474axylu/app.bsky.feed.post/3mp4lj3vexsf2"
  },
  "path": "/t/gpt-4o-returning-malformed-unicode-like-u0000e6-instead-of-ae-encoding-bug/1323897#post_3",
  "publishedAt": "2026-06-25T13:45:07.000Z",
  "site": "https://community.openai.com",
  "tags": [
    "@OpenAI_Support",
    "@OpenAIAPIhelper"
  ],
  "textContent": "We did an investigation and are confident that it is not from the tokenizer/tokenization. Instead, the endpoints seem to ban the generation of escaped Unicode sequence, other than control characters (`0x0000`-`0x001f` and `0x007f`). But this is an undocumented behaviour, we still need confirmation from @OpenAI_Support @OpenAIAPIhelper\n\nThree things to work this around now:\n→ Set `ensure_ascii=False` in your Python `json.dumps()` calls;\n→ Remove `\\uXXXX` examples from your prompts;\n→ Avoid asking to generate unicode escapes in OpenAI or Azure OpenAI’s endpoints. UTF-8 representation in JSON is sufficient for the expressivity.\n\nMore details in our post: `https://research.giskard.ai/blog/structured-output`",
  "title": "GPT-4o returning malformed Unicode like \\u0000e6 instead of æ — encoding bug?"
}