Project Theseus: A Distributed Inference Architecture Based on Semantic Deduplication and Neural State Caching

Author: [Vanderlei Feyth - ia@feyth.com.br] Keywords: Distributed Inference, Semantic Deduplication, Latent Vector Cache, Peer-to-Peer Networks, Edge AI, Efficient Machine Learning.

Abstract

The scalability of generative artificial intelligence faces a critical trilemma: latency, computational cost, and privacy. The current client-server model, centralized around massive GPU clusters, is unsustainable in the long term. This paper proposes Project Theseus , a peer-to-peer (P2P) distributed inference architecture inspired by the BitTorrent protocol, but extending far beyond simple computational power sharing.

We introduce the concept of a Deduplicated Neural State Cache (DNSC). The core idea, inspired by the observation that “a hard drive containing 1 million MP3s likely has thousands of identical songs with different filenames,” is that much of the computation in language models is redundant.

Project Theseus proposes a network in which an orchestrator distributes inference tasks and, before computing, checks a “torrent index” of intermediate neural states (“thought planks”) generated by previous requests. By reusing semantically identical computation blocks, the network achieves a theoretical efficiency approaching infinite computational compression, resolving the Ship of Theseus paradox for AI: a system that continuously rebuilds itself using existing parts to create new responses.

---

1. Introduction: The Cloud Bottleneck and User Intuition

The current AI paradigm operates under a “cloud mainframe” model. A user sends a prompt; a massive data center processes it from scratch. This architecture inherits three fundamental problems:

• Cost: Energy and hardware expenditures are enormous and centralized.

• Latency: Physical distance and processing queues degrade real-time experiences.

• Blind Redundancy: Every prompt is treated as a unique entity. The system does not “remember” that it has already processed something semantically similar.

A simple user observation exposes the flaw:

“I have a hard drive with more than 1,000,000 MP3s; surely thousands of songs are identical but have different names.”

In the context of Large Language Models (LLMs), prompts such as “What is the capital of France?” and “Where is the Eiffel Tower located?” may differ syntactically, but they share a vast latent computational subspace (the concepts of “France,” “Paris,” “geography”). Today, this subspace is recomputed entirely every time.

We propose a paradigm shift: from repetitive computation to intelligent assembly. Inspired by the fictional “middle-out” compression algorithm (Silicon Valley) and the Ship of Theseus paradox, this paper formalizes an architecture that not only partitions computational power (torrent-style), but also deduplicates and reuses the “thoughts” already computed by the network.

---

2. The Project Theseus Architecture

The architecture consists of three logical layers integrated into a P2P network.

2.1. The Semantic Orchestrator (The Intelligent “Tracker”)

Unlike a BitTorrent tracker that merely locates files, the Semantic Orchestrator is the brain of the operation. Upon receiving a prompt, it does not execute it directly. Instead, it performs a triple function:

Query Decomposition

A lightweight on-device model breaks the prompt into a graph of fundamental concepts.

Example:

[Eiffel_Tower] -> [is_a] -> [monument]
[located_in] -> [Paris]
[Paris] -> [is_a] -> [capital]

Semantic Hash Generation

Each concept and its relationships are transformed into a unique low-dimensional vector hash representing an intermediate neural state (a “ship plank”).

Distributed Orchestration

Based on these hashes, the orchestrator queries the network’s Distributed Hash Table (DHT) to locate which nodes have already computed and cached those specific neural states.

---

2.2. The Deduplicated Neural State Cache (DNSC)

This is the core innovation. Inspired by the observation of “songs with different names,” the DNSC is an ephemeral distributed storage system that preserves the “upper layers of thought.”

What is stored?

When a node processes a prompt such as “What is the capital of France?” , it generates activation vectors at each model layer. The DNSC stores the output vector of layer N corresponding to the concept [France/Paris/Capital].

The input layer (raw text) is not stored.

How is it indexed?

By the semantic hash of the concept, not by the prompt itself. This is the key insight.

The prompts “capital of France” and “city of the Louvre” would point to the same (or similar) semantic hash:

FRANCE_PARIS_CITY

which acts as the key within the torrent-like index. Redundancy is eliminated at its root.

Direct Analogy

The MP3 file is the final response. The DNSC stores the “digital audio samples” (latent vectors) composing the music. If another song uses the same sample, it is reused.

---

2.3. The Hybrid Node (Client = Server = Seeder)

Every device in the network operates in hybrid mode:

• Consumer: Submits requests.

• Processor: Executes inference fractions not found in cache, either by processing a prompt segment (data parallelism) or a model layer (model parallelism), receiving intermediate states from another node.

• Thought Seeder: Stores and serves neural states from its local cache, actively participating in the DNSC.

---

3. The “Middle-Out” Computation Algorithm

The workflow for a new prompt P is as follows:

Step 1 — Analysis and Hashing

The user’s local node decomposes P into a set of semantic hashes:

{H1, H2, H3...}

Step 2 — Cache Discovery (“Come”)

A search is performed in the DHT. For each Hn , the network returns the address of node S containing the corresponding neural state.

A “thought route” is established.

Step 3 — Ship Assembly (Distributed Processing)

For hashes found:

Computation is skipped entirely. The raw neural state is transferred from node S (Seeder) to node R (Responsible for the next layer).

This massively saves bandwidth and processing.

For hashes not found:

The orchestrator allocates the task to the node with the lowest latency and available capacity.

Processing starts from the closest intermediate state already found in cache (“middle-out”).

Example:

If the concept Paris exists in cache, but Eiffel Tower does not, processing starts from the Paris state and only the final layers are executed.

Step 4 — Merge and Delivery (“Go”)

The node responsible for the final layer receives the latest states, completes decoding into text, and delivers the response to the user.

Step 5 — Cache Update

Newly computed neural states (for example, Eiffel Tower) are hashed, stored locally, and announced to the DHT, enriching the network for future queries.

---

4. Feasibility Analysis and Challenges

The feasibility of Project Theseus lies in the convergence of current technological trends, although it faces monumental challenges.

4.1. Why It Is Feasible (The Critical Path)

Advancement of Edge AI

Neural chips (NPUs) in smartphones and laptops are becoming powerful enough to execute portions of large models locally.

Inherent Redundancy of LLMs

Research shows that deep neural activations are sparse. Most parameters are reused across recurring reasoning circuits for families of related questions.

Our system exploits this sparsity not only within a model, but across user sessions.

Federated Learning as a Foundation

The concept of sharing “model updates” without exposing raw data is already mature.

Our DNSC extends this idea toward ephemeral sharing of “inference states,” a logical next step.

---

4.2. Challenges and Proposed Countermeasures

Challenge	Proposed Countermeasure
Network Latency: Transferring vectors between nodes may be slower than local computation.	The orchestrator includes a Trade-off Calculator estimating transfer time (Vector Size / Bandwidth) versus local compute time (Required FLOPs / Node Power). Cache is only used if faster. Intermediate vectors may also be heavily compressed.
Security and Poisoning: A malicious node may serve incorrect neural states.	Lightweight ZK-SNARK Validation: Seeder nodes store succinct cryptographic proofs that vector V legitimately results from applying layer N to vector U. Consumer nodes verify proofs in milliseconds without recomputation.
Prompt Privacy: User prompts are exposed when decomposed into hashes.	Decomposition occurs fully on-device. The orchestrator only handles cryptographic hashes of concepts. The network knows someone requested concept X, but not who or why. Transferred neural states are mathematical tensors, not readable text.
Ship of Theseus Paradox (Versioning): Base models evolve, making old cache entries obsolete.	Semantic hashes include a model-version tag. Orchestrators may map old states into newer versions using an “adaptation function,” or invalidate caches when version gaps become too large. The ship rebuilds itself, while the essence of knowledge persists.

---

5. Conclusion: The Ship of Collective Knowledge

Project Theseus is not merely a computational efficiency architecture. It is both a philosophical and practical proposal for building collective intelligence.

Just as the Ship of Theseus paradox questions the identity of an object whose parts have all been replaced, our AI no longer resides in a static model, but within a dynamic network of neural states evolving with every query.

Each generated response rebuilds the ship using both old and new planks, assembling them together. Knowledge ceases to be a static file on a hard drive and instead becomes the living, mutable structure of the distributed cache itself.

The question “Is it feasible?” transforms into “When will it become inevitable?”

The remaining work lies in solving engineering challenges involving latency and cryptography. This architecture, born from a user’s intuition about MP3 redundancy on a hard drive, may very well become the blueprint for the next decade’s internet:

An internet that does not merely transmit data, but shares thought itself.