Made a Python failure dataset for DPO/RLHF — how do you source negative examples?

Working on a planner-executor. Planner outputs a code spec as JSON, executor reads it. Kept running into the same failure: model returns a valid JSON object but wraps it in json… backticks and prepends an explanation. Parser dies before it reaches the first field.

Prompt constraints help. “Return only JSON, no fences, no preamble.” Works for a session, then new inputs or slightly different phrasing and the decoration comes back.

Ended up building training data specifically for this. One row from DinoDS Lane 16 (Code JSON Spec Mode):

JSONL Row (public proof):

{"sample_id": "lane_16_spec_example_01", "mode": "quick", "tone": "professional",
 "intent_family": "content_generation", "intent_subtype": "code_json_spec",
 "representation_choice": "plain_text",
 "user_message": "Give me a JSON spec for a function that validates email addresses.",
 "assistant_response": "{"task_type":"simple_function","language":"python",
   "files":[{"name":"email_validator.py","purpose":"Validate email strings",
   "exports":["is_valid_email"]}],
   "constraints":["No external dependencies","Return boolean only"],
   "tests":["valid: a@b.com","invalid: a@@b.com"]}"}

The key is representation_choice: plain_text — explicitly labels the output contract as raw structured data rather than a formatted response. Still running experiments on whether SFT coverage alone locks this in or if DPO pairs are needed on the fence/no-fence contrast.

If you have failure logs in this area, curious what inputs trigger the regression. If need Full Bundle can come discuss it.