{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreigpzlytgz4uj5qff6aqcubbqfsop5d3g5d2hgi3sem4znpbhagaoe",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mngbyqjc3gd2"
  },
  "path": "/t/groot-on-so101-failing/176500#post_2",
  "publishedAt": "2026-06-03T23:29:12.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "the LeRobot Discord for the most reliable answer",
    "LeRobot GR00T N1.5 Policy docs",
    "TorchCodec README / compatibility table",
    "LeRobot pyproject.toml",
    "LeRobot installation docs",
    "NVIDIA Isaac-GR00T repository",
    "Fine-Tuning NVIDIA GR00T N1.5 on SO-101 Arm",
    "NVIDIA Isaac-GR00T README",
    "Isaac-GR00T issue #675: uv run python invalidates platform-specific venv / TorchCodec missing",
    "LeRobot issue #2217: dependency resolver / torch 2.7.1 / RTX 5070 / SM120 concern",
    "FlashAttention issue #2168: RTX 5090 / SM120 / no kernel image-style failure",
    "FlashAttention issue #2361: SM120 source build trouble",
    "Isaac-GR00T issue #421: torchcodec is not available",
    "Isaac-GR00T issue #502: Thor / GR00T N1.6 / Could not load libtorchcodec",
    "LeRobot issue #964: Could not load libtorchcodec",
    "LeRobot issue #3199: dataset viewer / Could not load libtorchcodec",
    "Isaac-GR00T issue #392: CUDA 11.8 / torch 2.7.1 / flash-attn install trouble",
    "Isaac-GR00T issue #233: GR00T N1.5 / flash-attn undefined symbol",
    "FlashAttention issue #1696: PyTorch 2.7.0 / CUDA 12.6 / prebuilt wheel ABI mismatch",
    "FlashAttention issue #1644: PyTorch 2.7.0 / 2.6.0 binary compatibility issue",
    "Isaac-GR00T issue #342: wrong video_backend / codec mismatch can get stuck at the first frame",
    "Isaac-GR00T issue #119: torchvision_av backend memory leak; decord / TorchCodec comparison",
    "Isaac-GR00T issue #172: torchvision_av timestamp/frame bug",
    "SO-101 GR00T N1.5 blog",
    "Isaac-GR00T data preparation docs",
    "Isaac-GR00T issue #423: SO-101 dataset format mismatch / KeyError: observation.images.front",
    "NVIDIA Isaac GR00T in LeRobot",
    "Isaac-GR00T issue #266: eval_lerobot.py broken by LeRobot API refactor / lerobot.common missing",
    "Isaac-GR00T issue #323: SO-101 guide missing draccus / lerobot; dependency conflict",
    "LeRobot on Hugging Face"
  ],
  "textContent": "Since this is in the Physical AI / LeRobot territory, I’d still recommend taking it to the LeRobot Discord for the most reliable answer. But before that, here is what I can piece together from the public docs and related issues. Looks like this is several layers nastier than ordinary version drift:\n\n* * *\n\n## Short diagnosis\n\nI would not treat this first as an SO-101 robot-control problem.\n\nIt looks more like a dependency/runtime matrix problem involving several moving layers:\n\n  * NVIDIA Isaac GR00T version line: N1.5 / N1.6 / N1.7 / `main`\n  * LeRobot version line: v0.4.x / v0.5.x / current `main`\n  * PyTorch version\n  * TorchCodec version\n  * FlashAttention / FA2 version\n  * CUDA version and GPU architecture\n  * platform architecture: Linux x86_64, Windows, Linux aarch64, Jetson/Thor/Spark, etc.\n  * FFmpeg / `libtorchcodec`\n  * video backend: `torchcodec`, `decord`, `torchvision_av`, `pyav`\n  * dataset flavor: standard LeRobot dataset vs GR00T-flavored LeRobot v2 dataset\n\n\n\nSo the apparent contradiction you saw — “TorchCodec wants a newer PyTorch, but FlashAttention / GR00T wants torch 2.7-ish” — may be real in some environments, but not in all environments.\n\nThe first thing I would clarify is **which platform and which docs line you are actually following**.\n\n* * *\n\n## The key conflict: GR00T N1.5 docs vs current LeRobot/TorchCodec assumptions\n\nThe LeRobot GR00T N1.5 docs currently say that GR00T N1.5 requires Flash Attention internally and that this is still not fully optional:\n\n  * LeRobot GR00T N1.5 Policy docs\n\n\n\nThose docs give an install path around:\n\n\n    pip install \"torch>=2.2.1,<2.8.0\" \"torchvision>=0.21.0,<0.23.0\"\n    pip install \"flash-attn>=2.5.9,<3.0.0\" --no-build-isolation\n\n\nSo your concern about FlashAttention pulling you toward the torch 2.7-ish era is reasonable.\n\nBut TorchCodec itself is not always a `torch>=2.11` dependency. The official TorchCodec compatibility table says, roughly:\n\nTorchCodec version | Compatible PyTorch line\n---|---\n`0.14`, `0.13`, `0.12` | `torch >= 2.11`\n`0.11` | `torch 2.11`\n`0.10` | `torch 2.10`\n`0.9`, `0.8` | `torch 2.9`\n`0.7`, `0.6` | `torch 2.8`\n`0.5`, `0.4`, `0.3` | `torch 2.7`\n\nSource:\n\n  * TorchCodec README / compatibility table\n\n\n\nSo if you are on a Linux x86_64 machine and you want to stay on `torch==2.7.x`, the relevant question is not “does TorchCodec always require torch >= 2.11?” It is more likely:\n\n> Did the resolver install a too-new TorchCodec, such as `torchcodec>=0.11` or `torchcodec>=0.12`, instead of the torch-2.7-compatible `torchcodec 0.3–0.5` line?\n\nThat said, the platform matters a lot.\n\n* * *\n\n## Why the platform matters\n\nCurrent LeRobot metadata appears to treat TorchCodec differently by platform.\n\nIn current LeRobot `main`, the `dataset` extra has platform-specific TorchCodec markers. At the time I checked, the rough shape was:\n\nPlatform | LeRobot TorchCodec range | Practical implication\n---|---|---\nLinux x86_64 / AMD64 | `torchcodec>=0.3.0,<0.12.0` | You may be able to pin TorchCodec to a torch-2.7-compatible line, such as `0.4` or `0.5`.\nWindows | `torchcodec>=0.7.0,<0.12.0` | This is already more torch-2.8-ish or newer. Do not assume the Linux x86_64 fix applies.\nLinux aarch64 / arm64 | `torchcodec>=0.11.0,<0.12.0` | This points toward the torch-2.11-era TorchCodec line. It can conflict with GR00T N1.5’s torch `<2.8` / FlashAttention instructions.\n`lerobot[groot]` | includes `flash-attn>=2.5.9,<3.0.0` | GR00T is not a normal “install every extra and go” case.\n\nSource:\n\n  * LeRobot pyproject.toml\n\n\n\nOne especially relevant detail: current LeRobot metadata appears to keep the `groot` extra out of the `all` extra with a comment along the lines that GR00T needs specific FlashAttention installation instructions. That is a signal that GR00T is not yet a fully ordinary dependency-extra path.\n\nSo I would be careful with commands like:\n\n\n    pip install \"lerobot[all]\"\n    pip install \"lerobot[groot,dataset,training]\"\n    pip install -U lerobot\n\n\nunless you know exactly which LeRobot version/commit and which GR00T line you are targeting.\n\n* * *\n\n## Possible docs-line drift\n\nThere are at least three “current-looking” instruction sets that can easily get mixed:\n\nDocs / repo line | Likely assumptions\n---|---\nGeneral LeRobot installation docs | Newer Python / newer PyTorch assumptions, TorchCodec default video decoding, FFmpeg requirement.\nLeRobot GR00T N1.5 policy docs | FlashAttention required, torch `<2.8`, CUDA-enabled device required.\nCurrent NVIDIA Isaac-GR00T repository / N1.7 README | `uv`, FFmpeg, TorchCodec default video backend, platform-specific CUDA/Python matrix, aarch64 caveats, FlashAttention / TensorRT GPU deps.\nHF/NVIDIA SO-101 GR00T N1.5 blog | Python 3.10, N1.5-era setup, `flash-attn==2.7.1.post4`, `meta/modality.json`, and `--video-backend torchvision_av`.\n\nSources:\n\n  * LeRobot installation docs\n  * LeRobot GR00T N1.5 Policy docs\n  * NVIDIA Isaac-GR00T repository\n  * Fine-Tuning NVIDIA GR00T N1.5 on SO-101 Arm\n\n\n\nI would not mix these blindly. They do not all imply the same Python / torch / TorchCodec / FlashAttention / video-backend stack.\n\n* * *\n\n## Platform-specific interpretation\n\n### Case A: Linux x86_64, non-Blackwell GPU\n\nIf you are on a normal desktop/server Linux x86_64 machine, and not on an RTX 50 / Blackwell / SM120 GPU, then one plausible path is:\n\n\n    # Example only; verify against your exact GR00T/LeRobot line.\n    pip install \"torch==2.7.1\" \"torchvision==0.22.*\" \"torchaudio==2.7.*\" --index-url <your-matching-pytorch-cuda-wheel-index>\n    pip install \"torchcodec>=0.3,<0.6\"\n    pip install \"flash-attn>=2.5.9,<3.0.0\" --no-build-isolation\n\n\nThe key idea is:\n\n  * keep torch in the GR00T/FlashAttention-compatible range;\n  * keep TorchCodec in the torch-2.7-compatible range;\n  * do not allow the resolver to pull `torchcodec 0.11+` or `0.12+`.\n\n\n\nI am not claiming the exact commands above will work for every CUDA/Python/GPU combination. The point is the version _shape_ : torch 2.7.x should be paired with TorchCodec 0.3–0.5, not the newer TorchCodec line.\n\n### Case B: Linux aarch64 / Jetson / Thor / DGX Spark\n\nThis is probably not the same problem.\n\nThe current GR00T README treats aarch64 platforms as special. It says that for Thor / Orin / Spark-style aarch64 platforms, TorchCodec is a required video backend and other video backends such as `decord` / `pyav` are not supported in the same way.\n\nSource:\n\n  * NVIDIA Isaac-GR00T README\n\n\n\nThere is also a related report where `uv run python` from the repository root appears to rebuild or replace a platform-specific venv and lose `torchcodec` and likely `flash-attn` on aarch64 platforms:\n\n  * Isaac-GR00T issue #675: uv run python invalidates platform-specific venv / TorchCodec missing\n\n\n\nIf you are on Jetson / Thor / Spark / aarch64, I would not start with the generic x86_64 pip recipe. I would first verify the platform-specific GR00T deployment docs and make sure you are using the intended venv, Python version, and platform wheels.\n\n### Case C: RTX 50 / Blackwell / SM120\n\nIf your GPU is RTX 50-series / Blackwell / compute capability `sm_120`, the torch-2.7-era path may be a separate problem.\n\nThere are reports around newer RTX 50 / SM120 environments where older PyTorch or FlashAttention paths fail with architecture/kernel support errors. This does not mean every torch 2.7 build is automatically unusable, but it does mean you should verify the actual binary being imported:\n\n\n    python - <<'PY'\n    import torch\n    print(\"torch:\", torch.__version__)\n    print(\"torch cuda:\", torch.version.cuda)\n    print(\"cuda available:\", torch.cuda.is_available())\n    if torch.cuda.is_available():\n        print(\"device:\", torch.cuda.get_device_name(0))\n        print(\"capability:\", torch.cuda.get_device_capability(0))\n    PY\n\n\nRelated examples:\n\n  * LeRobot issue #2217: dependency resolver / torch 2.7.1 / RTX 5070 / SM120 concern\n  * FlashAttention issue #2168: RTX 5090 / SM120 / no kernel image-style failure\n  * FlashAttention issue #2361: SM120 source build trouble\n\n\n\nFor Blackwell/SM120, I would not assume that “torch 2.7.1 + FlashAttention 2.x” is automatically the right target.\n\n### Case D: Windows\n\nI would treat Windows as a separate branch.\n\nLeRobot’s metadata uses a different TorchCodec minimum on Windows than on Linux x86_64. FlashAttention on Windows is also generally a more fragile/unofficial path than Linux CUDA. I would not apply a Linux x86_64 GR00T/FA2 recipe directly to Windows without checking the exact wheels and support status.\n\n* * *\n\n## TorchCodec errors are not always just “package not installed”\n\nIf the error says something like:\n\n\n    ImportError: torchcodec is not available\n\n\nor\n\n\n    RuntimeError: Could not load libtorchcodec\n\n\nthen check more than `pip show torchcodec`.\n\nTorchCodec depends on FFmpeg and binary/runtime compatibility. The error can be caused by:\n\n  * wrong TorchCodec version for your PyTorch version;\n  * missing or incompatible FFmpeg shared libraries;\n  * `LD_LIBRARY_PATH` / runtime loader issues;\n  * CUDA-enabled vs CPU-only wheel differences;\n  * platform wheel availability;\n  * venv mismatch.\n\n\n\nUseful checks:\n\n\n    python - <<'PY'\n    import sys, subprocess\n    print(\"python:\", sys.version)\n    try:\n        import torch\n        print(\"torch:\", torch.__version__)\n        print(\"torch cuda:\", torch.version.cuda)\n    except Exception as e:\n        print(\"torch import failed:\", repr(e))\n\n    try:\n        import torchcodec\n        print(\"torchcodec:\", getattr(torchcodec, \"__version__\", \"unknown\"))\n    except Exception as e:\n        print(\"torchcodec import failed:\", repr(e))\n    PY\n\n    ffmpeg -version\n    which ffmpeg\n    python -m pip show torch torchcodec torchvision torchaudio flash-attn\n\n\nRelated reports:\n\n  * Isaac-GR00T issue #421: torchcodec is not available\n  * Isaac-GR00T issue #502: Thor / GR00T N1.6 / Could not load libtorchcodec\n  * LeRobot issue #964: Could not load libtorchcodec\n  * LeRobot issue #3199: dataset viewer / Could not load libtorchcodec\n\n\n\n* * *\n\n## FlashAttention errors are not always just “wrong torch version”\n\nFlashAttention is a compiled CUDA extension path. Even if `pip install flash-attn` succeeds, it can still fail later due to ABI, CUDA, GPU architecture, or wheel/source-build mismatch.\n\nRelated examples:\n\n  * Isaac-GR00T issue #392: CUDA 11.8 / torch 2.7.1 / flash-attn install trouble\n  * Isaac-GR00T issue #233: GR00T N1.5 / flash-attn undefined symbol\n  * FlashAttention issue #1696: PyTorch 2.7.0 / CUDA 12.6 / prebuilt wheel ABI mismatch\n  * FlashAttention issue #1644: PyTorch 2.7.0 / 2.6.0 binary compatibility issue\n\n\n\nSo I would collect both install-time and runtime evidence:\n\n\n    python - <<'PY'\n    try:\n        import flash_attn\n        print(\"flash_attn module:\", flash_attn)\n        print(\"flash_attn version:\", getattr(flash_attn, \"__version__\", \"unknown\"))\n    except Exception as e:\n        print(\"flash_attn import failed:\", repr(e))\n    PY\n\n\nAnd if it imports, I would still not assume the actual model path is safe. The failure may appear only when GR00T enters the FlashAttention kernel.\n\n* * *\n\n## Video backend and codec also matter\n\nGR00T/LeRobot video data is not just a side dependency. If the video backend is wrong, the dataset may decode incorrectly, read wrong frames, leak memory, or fail during training/evaluation.\n\nRelevant reports:\n\n  * Isaac-GR00T issue #342: wrong video_backend / codec mismatch can get stuck at the first frame\n  * Isaac-GR00T issue #119: torchvision_av backend memory leak; decord / TorchCodec comparison\n  * Isaac-GR00T issue #172: torchvision_av timestamp/frame bug\n\n\n\nSo please include:\n\n\n    video_backend = ?\n    actual video codec = h264 / h265 / av1 / other?\n    ffmpeg -version = ?\n\n\nIf you are following the SO-101 N1.5 blog, note that it used `--video-backend torchvision_av` in the fine-tuning command:\n\n  * SO-101 GR00T N1.5 blog\n\n\n\nThat does not automatically mean `torchvision_av` is the right backend for every dataset/platform.\n\n* * *\n\n## Dataset flavor can be another separate issue\n\nEven after the install succeeds, the dataset can still fail.\n\nThe SO-101 N1.5 blog uses a GR00T-compatible dataset preparation step and copies a `meta/modality.json` file. The current GR00T data preparation docs also describe GR00T’s dataset format as a LeRobot-compatible flavor with additional modality metadata.\n\nSources:\n\n  * SO-101 GR00T N1.5 blog\n  * Isaac-GR00T data preparation docs\n\n\n\nThere is also a related SO-101 dataset mismatch report:\n\n  * Isaac-GR00T issue #423: SO-101 dataset format mismatch / KeyError: observation.images.front\n\n\n\nSo I would not only ask “is this a LeRobot dataset?” I would ask:\n\n\n    Is this standard LeRobot v2, LeRobot v3, or GR00T-flavored LeRobot v2?\n    Does it contain `meta/modality.json`?\n    Which `embodiment-tag` are you using?\n    What are the exact observation/action/video keys?\n\n\n* * *\n\n## LeRobot version line also matters\n\nGR00T integration in LeRobot is relatively recent and has been moving.\n\nThe HF/NVIDIA GR00T-in-LeRobot article describes GR00T integration around LeRobot v0.4.0:\n\n  * NVIDIA Isaac GR00T in LeRobot\n\n\n\nThere are also related reports where older GR00T examples hit newer LeRobot API changes:\n\n  * Isaac-GR00T issue #266: eval_lerobot.py broken by LeRobot API refactor / lerobot.common missing\n\n\n\nAnd a report that the SO-101 guide’s `eval_lerobot.py` path needed dependencies such as `draccus` and `lerobot`, with dependency conflicts when installing LeRobot on top of the GR00T environment:\n\n  * Isaac-GR00T issue #323: SO-101 guide missing draccus / lerobot; dependency conflict\n\n\n\nSo I would include the exact LeRobot version/commit in your report.\n\n* * *\n\n## What I would ask you to post\n\nBefore trying another workaround, I would ask for the exact environment matrix.\n\n\n    Platform\n    - OS:\n    - CPU architecture: x86_64 / Windows / aarch64 / arm64?\n    - Machine: desktop/server GPU, Jetson Orin, Jetson Thor, DGX Spark, etc.?\n    - GPU model:\n    - GPU compute capability:\n    - CUDA driver:\n    - CUDA toolkit:\n    - Python version:\n\n    Python packages\n    - torch:\n    - torchvision:\n    - torchaudio:\n    - torchcodec:\n    - flash-attn:\n    - transformers:\n    - lerobot:\n    - Isaac-GR00T version/tag/commit:\n    - ffmpeg version:\n\n    Install path\n    - Are you following:\n      - LeRobot general installation docs?\n      - LeRobot GR00T N1.5 docs?\n      - HF/NVIDIA SO-101 N1.5 blog?\n      - current Isaac-GR00T README / N1.7?\n      - another guide?\n    - Did you use pip, uv, conda, or the platform-specific GR00T install scripts?\n    - Exact install commands:\n\n    GR00T / LeRobot usage\n    - Are you fine-tuning, evaluating, or deploying to real SO-101?\n    - Are you using GR00T’s own server/client scripts or LeRobot async inference?\n    - Is `policy.type=groot` involved?\n    - Are you using `lerobot[groot]`, `lerobot[dataset]`, `lerobot[training]`, or all extras?\n\n    Video / dataset\n    - video_backend:\n    - actual video codec:\n    - dataset format: LeRobot v2 / LeRobot v3 / GR00T LeRobot flavor?\n    - does the dataset have `meta/modality.json`?\n    - embodiment tag:\n    - full traceback:\n\n\nUseful diagnostic snippet:\n\n\n    python - <<'PY'\n    import sys, subprocess, platform\n    print(\"python:\", sys.version)\n    print(\"platform:\", platform.platform())\n    print(\"machine:\", platform.machine())\n\n    for name in [\"torch\", \"torchvision\", \"torchaudio\", \"torchcodec\", \"flash_attn\", \"transformers\", \"lerobot\"]:\n        try:\n            mod = __import__(name)\n            print(f\"{name}:\", getattr(mod, \"__version__\", \"unknown\"))\n        except Exception as e:\n            print(f\"{name}: IMPORT FAILED:\", repr(e))\n\n    try:\n        import torch\n        print(\"torch cuda:\", torch.version.cuda)\n        print(\"cuda available:\", torch.cuda.is_available())\n        if torch.cuda.is_available():\n            print(\"gpu:\", torch.cuda.get_device_name(0))\n            print(\"capability:\", torch.cuda.get_device_capability(0))\n    except Exception as e:\n        print(\"torch cuda check failed:\", repr(e))\n    PY\n\n    ffmpeg -version || true\n    python -m pip freeze | grep -E 'torch|torchvision|torchaudio|torchcodec|flash|lerobot|transformers|decord|av|ffmpeg'\n\n\n* * *\n\n## My tentative recovery logic\n\nI would branch the recovery attempt like this.\n\nEnvironment | First thing I would suspect | First recovery direction\n---|---|---\nLinux x86_64, non-Blackwell | Resolver pulled too-new TorchCodec; FlashAttention wheel/ABI mismatch | Try torch 2.7.x + TorchCodec 0.3–0.5 + compatible FA2, with exact CUDA/Python matching.\nLinux x86_64, RTX 50 / SM120 | torch 2.7.x / FA2 kernel path may be too old | Verify actual torch CUDA build and GPU capability; do not assume the N1.5 torch `<2.8` path works.\nWindows | Different TorchCodec minimum and fragile FA2 path | Do not reuse Linux recipe directly.\naarch64 / Jetson / Thor / Spark | Platform-specific GR00T stack, TorchCodec required, venv/wheel mismatch | Use platform-specific GR00T install/deployment docs; avoid generic x86_64 pip recipe.\n`torchcodec is not available` | Could be FFmpeg/shared-lib/PyTorch/TorchCodec mismatch, not just missing pip package | Test `import torchcodec`, `ffmpeg -version`, torch/TorchCodec compatibility.\ndataset/video failure after install | Backend/codec/schema mismatch | Check `video_backend`, codec, GR00T `meta/modality.json`, embodiment tag.\n\n* * *\n\n## Where Discord fits\n\nIf this still fails after collecting the matrix above, I would bring that exact report to the LeRobot Discord rather than asking vaguely.\n\nThe entry point is:\n\n  * LeRobot on Hugging Face\n\n\n\nThat should make it much easier for maintainers or users with a known-good SO-101 / GR00T setup to identify whether this is:\n\n  * a simple TorchCodec pin issue,\n  * a GR00T N1.5 vs current LeRobot docs-line mismatch,\n  * an aarch64 platform-stack issue,\n  * an RTX 50 / Blackwell issue,\n  * an FFmpeg / libtorchcodec runtime issue,\n  * or a dataset/video-backend issue.\n\n",
  "title": "Groot on SO101 failing"
}