External Publication
Visit Post

How to stop Codex from rushing fixes?

OpenAI Developer Community June 7, 2026
Source
I think this is a useful comparison, because it shows two opposite failure modes. In my case, I may be giving Codex goals that are too specific, too constrained, or competing with each other, so it rushes toward a local solution and treats completion as success. In your case, it sounds like you solved that by giving it a higher-level mission like “produce good science”. That may remove the speed/compliance problem, but it may create a different one: it can now spend enormous effort approximating what “good science” looks like from the outside - reading?, hedging?, consulting?, collecting references?, designing more checks? …instead of exposing the actual judgment criteria. That is where I’m still cautious about “use your judgement”. I agree that the model should not treat every input as a command. But unless “judgement” is operationalized, the model may substitute something like: “what would a careful scientific researcher usually do?” … That can be useful, but it is still imitation of a research culture, not necessarily judgment in the stronger sense. And research culture itself can be wrong or incomplete. That does not mean science is bad. It means even good science depends on assumptions that need to be made explicit. The Aumann example is a nice case here. “Agreeing to Disagree” was not simply stupid or worthless - it was important and influential. But when people recently formalized the theorem in LEAN, the interesting part was assumption accounting: making the hidden structure explicit enough that every step actually compiles. That is the kind of thing I mean. A result can be respected, useful, and widely built upon, while still containing assumptions that were not visible enough until someone forced them into a stricter form. So maybe model needs a clear enough definition of what rigor is for a task. “Produce good science” is better than “finish quickly” but it is still underspecified. Good science according to which standard? Conservative academic consensus? Novel insight? Replicability? Formal proof? Predictive power? Usefulness? Elegance? Risk of embarrassment? These can conflict. So I would not want Codex to merely be released from constraints. I would want the constraints to become explicit and testable. Not “use your judgement” but something closer to: here are the standards of judgement, here is how to notice when they conflict, and here is when to stop reading and run the experiment. If that already works for you and survives journal review, fair enough. I’m not dismissing that. I’m just saying it still doesn’t satisfy the thing I’m trying to solve here.

Discussion in the ATmosphere

Loading comments...