I analyzed hidden-state dynamics across 7 open-weight LLMs and found recurring functional patterns. Looking for feedback

Thank you for taking the time to write such a detailed review. This is exactly the kind of feedback I was hoping for.

I agree with your central point: decodability, proxy labeling, and causal function should remain clearly separated. At the moment I only claim that certain properties are linearly decodable from hidden-state trajectories under a specific extraction pipeline. Whether the model causally uses those representations is a different question that requires intervention-based experiments.

Interestingly, several of the directions you mention have become the focus of my current work over the last few days.

I extended the framework from simple probe accuracy toward a more complete characterization of internal dynamics, including:

• Functional Capacity (how many independent properties are simultaneously linearly decodable from the same layer),

• Functional Density (how that capacity is distributed across depth),

• Functional Entropy and Effective Capacity,

• Dynamic trajectory metrics (curvature, commitment, transition entropy, D-state occupancy, etc.),

• Architecture-level comparisons across GPT-2, OPT, Qwen, Phi, TinyLlama and Llama families.

One unexpected result is that prompt ambiguity does not simply increase instability. Instead, it appears to modify the geometry of trajectories in architecture-dependent ways. In several models, ambiguity significantly changes curvature, cosine similarity, transition entropy or commitment without necessarily increasing D-state occupancy. That shifted my attention from “how many unstable states exist?” toward “where do these dynamic states appear along the trajectory and what role do they play?”

Your suggestion about metadata baselines is also well taken. I haven’t completed that part yet, but it is now high on the roadmap. Holding out prompt families and model families, residualizing obvious covariates, and comparing extraction points should help determine whether these dynamic signatures remain after controlling for simpler explanations.

I also agree that causal interventions are the natural next stage. Once the descriptive framework is stable, I would like to test whether perturbing specific trajectory regions or geometric structures actually changes downstream generation. That would allow us to move from observation toward causal evidence.

Thanks again for the thoughtful comments. They helped refine the roadmap considerably, and I appreciate that your suggestions focus on strengthening the measurement framework rather than simply arguing for or against the current interpretation.