Cross-architectural runtime probability dynamics in transformer LLMs — two clusters not explained by parameter count

Thanks for checking back and yes, I definitely kept going down the rabbit hole.

Your previous comments ended up changing the direction of the project in a meaningful way.

Following your suggestion, I treated logit calibration as a potential confound rather than assuming GD_ratio was measuring intrinsic dynamics. I then ran a series of follow-up audits:

• Temperature normalization audit (V22.5): after matching entropy across models, a substantial part of the original structure changed. Temperature is indeed a major driver.

• Vocabulary-invariant audit (V22.6): normalizing for vocabulary size and tokenizer granularity only partially altered the structure. The result was PARTIAL , suggesting vocabulary explains some, but not all, of the signal.

• Bootstrap confidence audit (V22.7): only part of the pairwise differences remained statistically robust, so I downgraded the strength of the original claim rather than strengthening it.

• Residual structure audit (V22.8b): with the pipeline corrected, the remaining signal is still strongly associated with effective temperature, so I’ve revised the interpretation again.

At this point I no longer describe GD_ratio as a direct measure of “internal dynamics.” A more accurate interpretation is that it captures aspects of output distribution geometry , with temperature acting as an important confounding factor.

The interesting part now is the residual after controlling for those obvious mechanisms. Rather than trying to defend the original interpretation, I’m using the negative results to refine the framework.

Your suggestion about per-token / entropy-bin calibration is still on my list. I think that’s probably the next meaningful control because it asks whether any residual survives after local calibration instead of only matching the global mean.

I’m also moving beyond logits toward hidden-state and attention analyses, but with the same philosophy you recommended: identify the boring explanations first (calibration, tokenizer effects, attention sinks, architectural constraints) before making stronger claims.

So yes the rabbit hole got deeper, but also narrower. The framework is becoming less ambitious in its claims and, I hope, more trustworthy because of that. Thanks again for pushing on the weak points rather than just the positive results.