External Publication

Downloading datasets

Hugging Face Forums [Unofficial] June 15, 2026

Yeah. Waiting can definitely be the right answer here, especially if the real transfer has not started yet. But when a download appears to be stuck, there are a few other layers worth separating besides “slow network” or a temporary network / Hugging Face-side issue:

From the screenshot alone, I would not immediately assume this is an authentication problem. It may be, but a stuck-looking progress bar can also come from the Hub transfer layer, Xet-backed file transfer, datasets materialization/cache, local disk/cache state, or simply progress reporting that is too quiet.

For this particular dataset, I would debug it in layers.

garrying/VMD-D is not a tiny text-only dataset. It is an image/mask dataset stored as Parquet / optimized Parquet, with train/test splits and around 15k rows total:

dataset page: garrying/VMD-D
files tab: garrying/VMD-D/tree/main

So the first materialized load can legitimately take some time. Still, if it stays at 0% for a long time with no network or disk activity, I would test these paths separately.

1. First, check whether it is only the visible progress bar

A progress bar that looks frozen does not always mean the transfer is actually dead. There are recent Xet-related reports where large downloads barely updated the tqdm progress bar, making the process look hung even though bytes were still moving:

huggingface_hub issue #4058: Xet downloads barely report progress

So before changing code, I would check:

Is network traffic still active?
Is disk usage increasing?
Is CPU active?
Does the cache directory size keep changing?
Does it eventually jump from 0% to a later percentage?

If yes, waiting may indeed be the right answer.

If there is no visible network/disk activity for several minutes, then I would move to layer-by-layer tests.

2. Check versions and auth state

This is low-cost and helps others understand what path you are using.

hf auth whoami
hf --version

Then check the Python-side versions:

python - <<'PY'
import platform
print("python", platform.python_version())

try:
    import datasets
    print("datasets", datasets.__version__)
except Exception as e:
    print("datasets import failed:", repr(e))

try:
    import huggingface_hub
    print("huggingface_hub", huggingface_hub.__version__)
except Exception as e:
    print("huggingface_hub import failed:", repr(e))

try:
    import hf_xet
    print("hf_xet", getattr(hf_xet, "__version__", "unknown"))
except Exception as e:
    print("hf_xet unavailable:", repr(e))
PY

If hf auth whoami fails, fix authentication first. But if authentication is fine, I would not keep focusing only on the token. The symptom can still be caused by transfer/cache/materialization layers.

3. Test the Hub transfer layer directly

Even if your final code uses load_dataset(), testing the Hub download path directly is useful because it removes some of the datasets preparation/cache logic from the picture.

Start with a dry run:

hf download garrying/VMD-D --repo-type dataset --dry-run

This should tell you which files would be downloaded and how much is already cached.

Then test the actual Hub transfer:

hf download garrying/VMD-D --repo-type dataset

Interpretation:

Result	What it suggests
`hf download --dry-run` works	Repo metadata and file listing are probably reachable.
`hf download` works	The Hub transfer path is probably okay. Then the issue may be more specific to `datasets.load_dataset()` materialization/cache.
`hf download` also hangs	The problem may be in Hub transfer, Xet, cache, disk, route/proxy/VPN, or runtime environment.
`hf auth whoami` fails	Auth/token should be fixed first.
Small files download but large Parquet shards hang	That points more toward transfer/cache/Xet/large-file behavior than a simple token issue.

Official docs for this path:

hf download / file download docs

4. Test `datasets` streaming separately

Then test whether datasets can at least open and iterate the dataset without fully materializing it locally:

from datasets import load_dataset

ds = load_dataset("garrying/VMD-D", split="train", streaming=True)
print(next(iter(ds)))

streaming=True is useful because it lets you work with a dataset without downloading/materializing the whole dataset first:

Datasets streaming docs

Interpretation:

Result	What it suggests
Streaming works	Basic repo access, split resolution, and first-sample reading probably work. If normal `load_dataset()` hangs, the issue may be in download/materialization/cache.
Streaming also hangs	It may still be a Hub/client/runtime issue, not necessarily auth. Streaming itself can have hang modes too.
Streaming returns a sample quickly but normal load hangs	This is a strong signal to inspect materialized loading, cache, Parquet handling, Xet, and disk.

Important caveat: streaming is a diagnostic, not a perfect proof. There are also recent reports where streaming=True itself can hang:

datasets issue #8169: Streaming dataset hangs consistently
huggingface_hub issue #4178: Streaming dataset hangs consistently

So I would use streaming as a useful split, not as a final verdict.

5. Test the normal materialized path with an explicit split

After that, test the normal path, but with an explicit split:

from datasets import load_dataset

ds = load_dataset("garrying/VMD-D", split="train")
print(ds)

Specifying split="train" makes the test easier to reason about than loading the whole DatasetDict first.

The normal load_dataset() path may involve several steps:

resolving the dataset repository
discovering the data files
downloading files through huggingface_hub
reading Parquet files
preparing/cacheing an Arrow-backed Dataset
returning the selected split

Official loading docs:

Datasets loading docs

This is why I would not debug the screenshot as if it represented only one simple HTTP request.

6. Try isolating the Xet path

Because the dataset files are Xet-backed, and because recent Hub transfers now commonly use hf_xet, I would test once with Xet disabled.

For CLI:

HF_HUB_DISABLE_XET=1 hf download garrying/VMD-D --repo-type dataset

For Python, set it before importing datasets or huggingface_hub:

HF_HUB_DISABLE_XET=1 python - <<'PY'
from datasets import load_dataset

ds = load_dataset("garrying/VMD-D", split="train")
print(ds)
PY

Or inside a notebook, restart the runtime/kernel, then run this before importing datasets:

import os
os.environ["HF_HUB_DISABLE_XET"] = "1"

from datasets import load_dataset

ds = load_dataset("garrying/VMD-D", split="train")
print(ds)

I would treat this as a diagnostic switch, not necessarily a permanent fix.

Result	Interpretation
Works with `HF_HUB_DISABLE_XET=1`	The Xet transfer path is probably involved.
Still hangs with `HF_HUB_DISABLE_XET=1`	The issue may be elsewhere: network route, proxy/VPN, cache, disk, timeout, runtime, or another bug.
Works without disabling Xet after upgrading `hf_xet`	The issue may have been fixed in a newer transfer client.
Xet-disabled HTTP fallback fails for very large files	That does not necessarily mean Xet was innocent; HTTP fallback is not always a universal replacement for Xet.

Related docs and issues:

huggingface_hub environment variables
datasets issue #8129: load_dataset() hangs when hf_xet is enabled
xet-core issue #407: Cannot download file from XET hosted repo using CLI
xet-core issue #789: hf_xet downloads stalling
huggingface_hub issue #3868: HTTP download fails for files >50GB

The closest diagnostic pattern I found is datasets #8129: load_dataset() hangs when hf_xet is enabled, while streaming=True and direct Hub access work, and HF_HUB_DISABLE_XET=1 changes the behavior. I am not saying this dataset is definitely hitting the same bug, but the split is close enough that the same diagnostic structure is useful.

7. Update the transfer-related packages

If this is a fresh environment or notebook, I would also try updating the relevant packages:

pip install -U datasets huggingface_hub hf_xet

Then restart the Python process/kernel and retry.

This matters because huggingface_hub, datasets, and hf_xet can interact in the download path. Updating only one of them may leave you with a mixed stack.

If you are in a notebook environment such as Colab or Kaggle, restarting the runtime after upgrade is important. Otherwise, Python may keep using already-imported modules.

8. Check cache and disk state

Also check free disk space and cache locations. datasets can involve more than one cache:

Hub cache: files downloaded from the Hub
Datasets cache: processed Arrow-formatted datasets
Xet cache: Xet chunks/ranges

Useful docs:

Datasets cache management
huggingface_hub environment variables

Useful commands:

df -h
hf cache ls

If you want to test with a separate clean cache, use a temporary location:

mkdir -p /tmp/hf-test-cache
HF_HOME=/tmp/hf-test-cache python - <<'PY'
from datasets import load_dataset

ds = load_dataset("garrying/VMD-D", split="train")
print(ds)
PY

If that works, your previous cache may be involved. If it still hangs, the issue is probably not only a stale cache entry.

You can also separate Hub and Datasets cache variables, but for a quick test HF_HOME is often simpler because it moves the general Hugging Face cache root.

9. Optional: make the test more focused on the first file/split

If the goal is only to verify access, avoid loading more than needed at first.

For example, use streaming and take one example:

from datasets import load_dataset

ds = load_dataset("garrying/VMD-D", split="train", streaming=True)
first = next(iter(ds))
print(first.keys())

Or use a split-level normal load:

from datasets import load_dataset

train = load_dataset("garrying/VMD-D", split="train")
print(train)

Once that works, then move on to the full workflow.

10. A compact decision table

Here is how I would read the outcomes:

Test	If it works	If it hangs/fails
`hf auth whoami`	Token is probably visible to CLI.	Fix login/token first.
`hf download ... --dry-run`	Metadata and file list are reachable.	Metadata/access/client/network issue.
`hf download ...`	Hub transfer path probably works.	Hub transfer / Xet / cache / disk / network route issue.
`load_dataset(..., streaming=True)`	Basic dataset access and first-sample path probably work.	Still may be Hub/client/runtime; not automatically auth.
`load_dataset(..., split="train")`	Normal materialized path works.	Materialization/cache/Parquet/Xet/disk path may be involved.
`HF_HUB_DISABLE_XET=1 ...`	If behavior changes, Xet path is probably involved.	Not only Xet; inspect network/cache/disk/runtime.
Fresh `HF_HOME=/tmp/...` works	Old cache may be involved.	Cache is less likely to be the only cause.
Package upgrade fixes it	Version interaction was likely involved.	Continue with transfer/runtime diagnostics.

11. If you want to report it upstream

If it still hangs after the tests above, I would include this information in a follow-up issue or forum reply:

Dataset: garrying/VMD-D
Environment: local / Colab / Kaggle / server / etc.
OS:
Python:
datasets:
huggingface_hub:
hf_xet:
Command that hangs:
Does `hf download ... --repo-type dataset --dry-run` work?
Does `hf download ... --repo-type dataset` work?
Does `load_dataset(..., streaming=True)` work?
Does `load_dataset(..., split="train")` work?
Does `HF_HUB_DISABLE_XET=1` change the behavior?
Free disk space:
Cache location:
Proxy/VPN/corporate network:
Last visible progress line:
Whether network/disk activity continues while the bar is stuck:

That makes it much easier to distinguish:

slow-but-normal first download
misleading tqdm progress
Xet-backed transfer stall
Datasets materialization/cache issue
stale/partial cache issue
auth/access issue
route/proxy/VPN/runtime-specific issue

Short version

I would try this order:

hf auth whoami
hf download garrying/VMD-D --repo-type dataset --dry-run
hf download garrying/VMD-D --repo-type dataset

Then:

from datasets import load_dataset

ds = load_dataset("garrying/VMD-D", split="train", streaming=True)
print(next(iter(ds)))