I analyzed hidden-state dynamics across 7 open-weight LLMs and found recurring functional patterns. Looking for feedback

Thanks, I really appreciate the thoughtful feedback.

I completely agree that probing and causality are two different questions. At this stage, my work is about measuring the organization of hidden-state trajectories, not claiming that the decoded properties are themselves causal. Causal interventions (activation patching, steering or targeted perturbations) are definitely where I’d like to go next once the descriptive framework is stable.

Interestingly, some of the experiments you suggested are already becoming part of the roadmap. I recently expanded the work beyond simple probing into what I call functional organization : measuring functional capacity, functional density, functional entropy, and dynamic trajectory metrics such as curvature, commitment and state transitions across several Transformer families (GPT-2, OPT, Qwen, Phi, TinyLlama and Llama).

One result that surprised me is that increasing prompt ambiguity doesn’t simply make models “more unstable.” Instead, different architectures respond differently, and the effect seems to be expressed more through trajectory geometry than through a simple increase in unstable states. That shifted my focus toward understanding the topology of these internal trajectories rather than looking only at individual layers.

I also like your suggestion about transitions carrying more information than isolated hidden states. In fact, that’s becoming one of the central ideas of the framework: instead of asking “where is information stored?” , I’m increasingly asking “how does information move through the network?” The trajectory itself may be the fundamental object, with individual hidden states being snapshots of a larger dynamic process.

The untrained-model baseline is also a great idea. If the same geometric organization appears before training, that would point toward architectural effects; if it only emerges after training, it would suggest that these structures are learned. That’s definitely an experiment I’d like to include.

Thanks again for the excellent suggestions they fit remarkably well with the direction the project is now taking.