{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiex7a3kjnejicrl4nphhkiwgbmuwfnvppbgsoybpir3lsp3mgcaim",
    "uri": "at://did:plc:25rdn5elo5izoxrmtis34zuk/app.bsky.feed.post/3mpbm6z4toli2"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreiepcdrpkx7zuodda6gwi65p7g7v6qgafenr6ib3gjvwroxpfehl5i"
    },
    "mimeType": "image/webp",
    "size": 576002
  },
  "path": "/skyne/resurrecting-kepler-getting-modern-llms-running-on-a-gtx-770-kernel-7x-4na",
  "publishedAt": "2026-06-27T13:29:09.000Z",
  "site": "https://dev.to",
  "tags": [
    "cuda",
    "linux",
    "llm",
    "gpu",
    "Fedora/Debian packaging",
    "GitHub Gist"
  ],
  "textContent": "> **⚠️ Experimental hack** : Use on non-critical systems. Ensure you have backups. This patches a proprietary binary at the instruction level — no warranty, no support.\n\n##  The Story: Defying Obsolescence\n\nKepler GPUs (2012–2014) are e-waste by NVIDIA's timeline, but they are perfectly capable hardware for inference workloads. The GTX 770 has 1536 CUDA cores and 2 GB GDDR5 — enough for small-to-medium LLMs. This project proves that with a **five-byte fix** and some kernel backports, these GPUs can be kept useful on modern Linux systems, reducing e-waste and teaching real systems engineering along the way.\n\n##  Goal\n\nKeep an **NVIDIA GeForce GTX 770 (GK104, sm_30)** — a Kepler GPU abandoned by NVIDIA's driver stack after driver 470.256.02 and CUDA 10.2 — running CUDA workloads on a modern Linux kernel (6.15 → 7.x, Ubuntu 26.04).\n\nTwo problems made stock software a dead end:\n\n  1. **Kernel module won't compile** — the 470.256.02 driver source doesn't build against kernels ≥6.15 due to dozens of removed/renamed APIs.\n  2. **`cuInit` returns error 802** — even after the module loads and `nvidia-smi` works, every CUDA program fails with `CUDA_ERROR_SYSTEM_NOT_YET_INITIALIZED`.\n\n\n\n##  Technical Deep-Dive\n\n###  1. Kernel Module Patching\n\nThe proprietary 470.256.02 driver source does not build against kernels ≥6.15 due to removed/renamed APIs. I used community-sourced patch sets (primarily from Fedora/Debian packaging by Joan Bruguera Mico and Andreas Beckmann) to resolve issues like:\n\n  * `screen_info` → `sysfb_primary_display.screen`\n  * `del_timer_sync` → `timer_delete_sync`\n  * `follow_pfn` → `unsafe_follow_pfn`\n  * `dma_fence_signal` now returns void\n  * GCC 14 `efi_enabled` cast and UBSAN mismatches\n\n\n\nAfter these backports, `nvidia-smi` reports the GTX 770 correctly. But `cuInit` still fails.\n\n###  2. Resolving the `cuInit` Error 802\n\nAll `rm_ioctl` kernel calls return `NV_OK` — the kernel module is fine. The failure lives in userspace. With `gdb`, I traced `cuInit` calling `rm_ioctl(0x2a)` twice; both calls succeed at the kernel level, yet the library still returns 802.\n\nDisassembly of the RM response handler in `libcuda.so.470.256.02`:\n\n\n\n    3436a0: mov   0xc(%rsp),%eax      ; load status from RM response\n    3436a4: cmp   $0x2,%eax           ; status == 2?\n    3436a7: je    3436f0              ; → return 802\n    3436a9: jbe   3436e0              ; status <= 1?\n    3436e0: cmp   $0x1,%eax\n    3436e3: jne   3436c5              ; status != 1 → return 999\n    3436e5: xor   %eax,%eax           ; return 0 (success)\n    ...\n    3436f0: add   $0x18,%rsp\n    3436f4: mov   $0x322,%eax         ; return 802\n    3436f9: pop; ret\n\n\n**Root cause:** The Resource Manager firmware on Kepler returns status code `2` (`NV_ERR_BUFFER_TOO_SMALL`) for the second initialization `rm_ioctl`. The library treats `1` and `4` as success, but `2` is fatal → 802. Likely a buffer-size negotiation mismatch between the GTX 770's VBIOS firmware and the final 470.x userspace library. NVIDIA never fixed it because Kepler was already on legacy support.\n\n**The fix:** One instruction at offset `0x3436f4`. Instead of `mov $0x322, %eax` (return 802), return 0:\n\n| Bytes | Instruction\n---|---|---\nBefore | `b8 22 03 00 00` | `mov $0x322, %eax`\nAfter | `31 c0 90 90 90` | `xor %eax, %eax; nop; nop; nop`\n\nSubsequent `rm_ioctl` calls succeed — only this specific init ioctl is broken. Patch script:\n\n\n\n    #!/usr/bin/env python3\n    import shutil, os\n\n    libpath = \"/usr/lib/x86_64-linux-gnu/libcuda.so.470.256.02\"\n    backup_path = libpath + \".bak\"\n\n    if not os.path.exists(backup_path):\n        shutil.copy2(libpath, backup_path)\n\n    with open(libpath, \"rb\") as f:\n        data = bytearray(f.read())\n\n    offset = 0x3436f4\n    expected = bytes([0xb8, 0x22, 0x03, 0x00, 0x00])\n    actual = data[offset:offset+5]\n\n    if actual == expected:\n        data[offset:offset+2] = bytes([0x31, 0xc0])\n        data[offset+2:offset+5] = bytes([0x90, 0x90, 0x90])\n        print(f\"Patched: {actual.hex()} -> {data[offset:offset+5].hex()}\")\n    elif actual[:2] == bytes([0x31, 0xc0]):\n        print(\"Already patched!\")\n    else:\n        print(f\"UNEXPECTED at 0x{offset:x}: {actual.hex()}\")\n        exit(1)\n\n    with open(libpath, \"wb\") as f:\n        f.write(data)\n\n\n###  3. Toolchain & Compilation\n\nsm_30 support was dropped in CUDA 11, so we need CUDA 10.2's `ptxas`. But `nvcc` rejects GCC 15 (Ubuntu 26.04 default). **clang++** bridges legacy CUDA 10.2 headers and modern system libraries.\n\nllama.cpp uses `cg::this_grid()` (CUDA 11+). Patched `softmax.cu` for CUDA 10.2:\n\n\n\n    // Before (CUDA >= 11.0):\n    const cg::grid_group g = cg::this_grid();\n\n    // After (CUDA < 11.00):\n    const cg::thread_block g = cg::this_thread_block();\n\n\nBuild flags:\n\n\n\n    cmake .. -DLLAMA_CUDA=ON \\\n      -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \\\n      -DCUDAToolkit_ROOT=/usr/local/cuda-10.2 \\\n      -DCMAKE_CUDA_COMPILER=clang++ \\\n      -DCMAKE_CUDA_ARCHITECTURES=30 \\\n      -DGGML_CUDA_GRAPHS=OFF\n\n\n`-DGGML_CUDA_GRAPHS=OFF` is critical — CUDA graph capture requires sm_35+ and crashes on sm_30.\n\n##  Performance Benchmarks\n\nHardware: **GTX 770 (2 GB VRAM)** , **Ubuntu 26.04** , **kernel 7.0.0-27** , **llama.cpp c16c35b81**.\n\n###  Qwen 2.5 1.5B — fully offloaded (ngl=99)\n\nQuant | Test | t/s\n---|---|---\nQ4_K_M | pp64 | **69.50±0.95**\nQ4_K_M | tg512 | **25.84±0.20**\n\n###  Qwen 2.5 1.5B — CPU only (ngl=0)\n\nQuant | Test | t/s\n---|---|---\nQ4_K_M | pp64 | **39.03±1.09**\n\nGPU offload gives ~1.8× speedup on prompt processing for this model.\n\n###  Qwen 2.5 3B — fully offloaded (ngl=99)\n\nQuant | Test | t/s\n---|---|---\nQ3_K_M | pp64 | **36.18±0.33**\nQ3_K_M | tg256 | **10.11±0.11**\n\nQwen 3B at Q4_K_M (1.95 GiB) exceeds 2 GB VRAM — Q3_K_M (1.60 GiB) is required for full offloading.\n\n##  It Works\n\n\n    $ nvidia-smi -L\n    GPU 0: NVIDIA GeForce GTX 770 (UUID: GPU-3a93c548-...)\n\n    $ /tmp/test_cuinit\n    cuInit=0\n\n    $ llama-bench --list-devices\n    CUDA0: NVIDIA GeForce GTX 770 (1998 MiB, ...)\n\n\nFull working stack: kernel module → patched `libcuda.so` → CUDA 10.2 runtime → llama.cpp CUDA backend — all on Linux 7.x with a 2013 Kepler GPU.\n\n##  Surviving Kernel Upgrades (DKMS)\n\nRegister the patched driver with DKMS so module rebuilds happen automatically:\n\n\n\n    sudo apt install dkms\n    sudo dkms add nvidia/470.256.02\n    sudo dkms build nvidia/470.256.02 -k $(uname -r)\n    sudo dkms install nvidia/470.256.02 -k $(uname -r)\n\n\n##  Full Technical Write-up\n\nFor the complete debugging log, kernel patch table, patch scripts, and build instructions, see the GitHub Gist.",
  "title": "Resurrecting Kepler: Getting Modern LLMs Running on a GTX 770 (Kernel 7.x)"
}