{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiduyttunqx4vlwvupxpdhxhdcgqbtkz3ggkersckyrf6afzyh7vvq",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mel5cugflky2"
  },
  "path": "/t/what-are-all-the-files-that-are-being-downloaded/173329#post_2",
  "publishedAt": "2026-02-11T08:51:46.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "caching",
    "Hugging Face",
    "Hugging Face",
    "Hugging Face",
    "Hugging Face",
    "Hugging Face",
    "Hugging Face",
    "Hugging Face",
    "Hugging Face",
    "GitHub",
    "Hugging Face"
  ],
  "textContent": "A common reason for extra storage consumption is caching.\n\nHowever, with major models like `openai/gpt-oss-20b`, it’s common for separate files for multiple platforms to be stored within the repository. Downloading the entire repository means all of these get downloaded…\n\n* * *\n\n`hf download openai/gpt-oss-20b` downloads a **full snapshot of the repo** (i.e., every file in the model repo), not “just one set of weights”. For `openai/gpt-oss-20b`, the repo contains **multiple full-weight artifacts** (hence ~41.3GB total).\n\n## Files in `openai/gpt-oss-20b`\n\n### Repo root\n\n  * `.gitattributes`\n  * `LICENSE`\n  * `README.md`\n  * `USAGE_POLICY`\n  * `chat_template.jinja`\n  * `config.json`\n  * `generation_config.json`\n  * `model-00000-of-00002.safetensors`\n  * `model-00001-of-00002.safetensors`\n  * `model-00002-of-00002.safetensors`\n  * `model.safetensors.index.json`\n  * `special_tokens_map.json`\n  * `tokenizer.json`\n  * `tokenizer_config.json` (Hugging Face)\n\n\n\n### `metal/`\n\n  * `metal/model.bin` (13.8GB) (Hugging Face)\n\n\n\n### `original/`\n\n  * `original/config.json`\n  * `original/dtypes.json`\n  * `original/model.safetensors` (13.8GB) (Hugging Face)\n\n\n\n## Why this becomes ~40–50GB\n\nThis repo includes _three_ large “model-weight” payloads:\n\n  * **Sharded safetensors** in the root (`model-0000*-of-*.safetensors`) totaling ~13.8GB (Hugging Face)\n  * A **single-file safetensors** copy under `original/model.safetensors` (~13.8GB) (Hugging Face)\n  * A **precompiled Metal binary** under `metal/model.bin` (~13.8GB) intended for Apple Metal runtimes (Hugging Face)\n\n\n\nThat’s already ~41.4GB _before_ small metadata/tokenizer files, which matches the repo size shown on the “Files” tab (~41.3GB). (Hugging Face)\n\n## If you want to avoid downloading everything\n\nUse `--include/--exclude` patterns. (Hugging Face)\n\nExamples:\n\n  * Download only the “original” weights (minimal set recommended in OpenAI’s gpt-oss repo docs):\n\n        hf download openai/gpt-oss-20b --include \"original/*\" --local-dir gpt-oss-20b/\n\n\n(GitHub)\n\n  * Download everything _except_ the Metal and original copies (keep only the root sharded safetensors + configs/tokenizer):\n\n        hf download openai/gpt-oss-20b --exclude \"metal/*\" --exclude \"original/*\"\n\n\n(Hugging Face)\n\n\n",
  "title": "What are all the files that are being downloaded?"
}