External Publication
Visit Post

I analyzed hidden-state dynamics across 7 open-weight LLMs and found recurring functional patterns. Looking for feedback

Hugging Face Forums [Unofficial] June 29, 2026
Source
This s a really interesting direction. The rotation result especially stands out it suggests the signal is likely in the overall structure of the representation space, not individual neurons. The biggest next step would be causal tests: probing shows information exists, but not whether it drives the model’s behavior. Things like activation patching or targeted interventions would help confirm that. Also worth testing: * untrained models as a baseline * cross-model comparisons * whether state transitions carry more information than individual hidden states The idea of dynamic functional regimes rather than fixed syntax semantic layers seems like a promising way to think about transformer internals.

Discussion in the ATmosphere

Loading comments...