External Publication
Visit Post

Request for ArXiv endorsement to submit a preprint on long-context LLM failures

Hugging Face Forums [Unofficial] June 2, 2026
Source
Hello, I’m preparing an arXiv submission on a research project that addresses two failure modes in long-context LLM inference: 1. The “lost in the middle” phenomenon (Liu et al., 2023), where LLMs fail to use information buried in the middle of long contexts. I’m developing a system called TIS that aims to improve retrieval and retention of relevant context in long inputs. 2. Attention drift in speculative decoding drafters (Eldenk et al., arXiv:2605.09992, 2026), where EAGLE-3 style drafter models progressively abandon the original prompt during a speculation chain and attend instead to their own recently-generated tokens. I’m extending the same system to fix this problem. I have a working prototype with unit tests and a smoke test that validates the end-to-end pipeline for both problems. I’m an independent researcher (no institutional affiliation) and I’m looking for endorsement to submit this preprint to arXiv in an appropriate category. The project is still pending final training testing using Mistral-7B-v0.3 as the base model. I will try to make the run at JarvisLabs soon, and if all goes as planned, I will release the paper to ArXiv as an INDEPENDENT Researcher. If a potential endorser wants to know more, you can contact me and I will arrange a private tour of the private repository before endorsement.

Discussion in the ATmosphere

Loading comments...