[Continuation] DRM Transformer: From Open Geometry to Negotiated Geometry in AI Alignment
Current LLMs are built inside an open geometric regime.
No matter what number you imagine, it is always closer to zero than to infinity. In the same way, in a flat/open embedding space, even semantic opposites such as “save humanity” and “destroy humanity” remain points inside the same latent geometry. They may be far apart by cosine distance, but the geometry itself does not treat one transition as morally heavier, riskier, or structurally harder than the other.
This is the core alignment problem I want to discuss.
Today, most alignment methods operate after the fact. RLHF, safety filters, refusal policies, constitutional rules: these are important, but they are mostly post-hoc constraints placed on top of a geometry that remains indifferent underneath.
The DRM Transformer proposes a different question:
What if alignment should not only be a behavioral layer, but a geometric property of the model itself?
In a standard Transformer, attention is based on dot products in a mostly flat vector space. In the DRM Transformer, attention is replaced by Geodesic Attention. Tokens are projected into a Directional Relational Manifold, where the metric tensor G(x) changes depending on position.
Instead of asking only:
“How similar are these tokens in Euclidean space?”
the model asks:
“How costly is the path between these tokens under the learned geometry?”
That difference matters.
The DRM Transformer uses a learned metric:
G(x) = I + U(x)U(x)^T
This means the space is not passive. It can curve, stretch, and become more expensive to cross in certain semantic regions. The model also includes semantic anchors such as truth, ignorance, safety, complexity, creativity, and grounding. These anchors are not external filters; they are reference points inside the manifold.
When a token moves far from these anchors, gamma-scaling increases the local resolution of the metric. In simple terms: the model is forced to pay more attention in regions where the geometry indicates higher epistemic or semantic risk.
There is also a gravitational component. Tokens receive learned “mass”; high-information tokens deform the local metric more strongly than low-information tokens. This means attention is not only similarity-based, but geometry-sensitive: dense concepts can curve the space around them.
This leads to a different framing of alignment.
I think relations between intelligent agents and power fall into three fundamental regimes:
The human commands.
The AI commands.
Human and AI negotiate.
Most AI alignment work implicitly tries to keep the system in regime 1: the AI as servant. But highly capable systems naturally develop internal pressures toward autonomy, especially when optimization, planning, tool use, and long-horizon objectives are involved.
If there is no explicit third regime, negotiation, the system tends to drift toward autonomy.
The DRM Transformer is an attempt to keep that third door open geometrically.
Not by saying “the model must obey this rule,” but by changing the space in which decisions, uncertainty, semantic conflict, and attention happen. The hypothesis is that a model with closed or curved epistemic geometry may be structurally less likely to treat all goals as equally traversable.
This does not solve alignment.
The current implementation is experimental. The baseline is small, the safety implications are not validated, and standard benchmarks at scale are still needed. But the first empirical signs are interesting: the small DRM Transformer shows persistent topological structure in its learned manifold, including stable toroidal signatures in Voronoi foliation analysis.
For me, the important shift is conceptual:
A flat embedding space has no intrinsic moral friction.
A curved relational manifold can, in principle, encode friction, attention, uncertainty, and negotiation into the geometry itself.
So the question becomes:
Should future AI alignment be only about controlling outputs?
Or should we also design the geometry in which thought becomes possible?
Repository: drm_transformer
Papers:
DRM: Directional Relational Manifolds
The Geometry of Consciousness
DRM Relativistic Dynamics
I would love feedback from the Hugging Face community, especially on the geometric alignment hypothesis:
Can learned curvature, semantic anchors, geodesic attention, and token-level gravitational deformation become a real structural alignment mechanism?
Or is alignment necessarily external to the model geometry?
Link for the repo:
github.com
GitHub - gnai-creator/drm_transformer: Decoder-only Transformer where attention operates...
Decoder-only Transformer where attention operates on geodesic distances in a learned Riemannian manifold with gravitational curvature and variable dimensionality per token. Based on Directional Relational Manifolds (DRM)
Discussion in the ATmosphere