{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiaa3fpgjiyysunisevfyt5er6744ayfo2xeh6iam7mkecherxoeyi",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mqdwhmlagvb2"
  },
  "path": "/t/how-many-images-to-parameters/177672#post_2",
  "publishedAt": "2026-07-11T04:26:22.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "Bigger is not Always Better: Scaling Properties of Latent Diffusion Models",
    "(click for more details)"
  ],
  "textContent": "Hmm… for now, I don’t think the number of images per parameter alone gives us enough information to decide:\n\n* * *\n\n**I would not reduce the number of useful images just to make an images-per-parameter ratio look more similar to Stable Diffusion.** I also would not assume that increasing 430M to 860M will automatically improve quality.\n\nThe main reason is that “430M parameters / 2M images” combines several different constraints:\n\n  * what the 430M parameters belong to;\n  * whether this is training from scratch or continued pretraining;\n  * how many times the images are actually presented;\n  * the semantic, caption, resolution, and aspect-ratio distribution of those images;\n  * the training compute available;\n  * and what kind of “quality” is currently missing.\n\n\n\nThere is an unusually close reference point: Bigger is not Always Better: Scaling Properties of Latent Diffusion Models includes an **exact 430M-parameter denoising UNet** in a controlled family ranging from 39M to 5B parameters. That at least shows that 430M is a perfectly plausible model size rather than an obviously undersized configuration.\n\nHowever, that 430M model was trained from scratch under a very different regime—about 600M image-text pairs, 500K updates, and global batch 2048—so I would not transfer its image/model ratio directly to a 2M-image dataset. The useful lesson from that paper is instead that model size, training compute, and inference budget need to be compared separately.\n\nMy default route would be:\n\n  1. **Keep the 2M useful images.**\n  2. **Check whether the current 430M model is still improving on held-out data.**\n  3. **Separate the current failure mode** —underfitting, prompt alignment, rare concepts, composition, high-resolution detail, or overfitting.\n  4. **Compare the current model with a moderately larger model in a small controlled run** , rather than immediately committing to a full larger-model training run.\n  5. Compare at both:\n     * matched sample presentations; and\n     * matched training compute or wall-clock budget.\n\n\n\nIf the larger model wins under both comparisons, scaling it up becomes much easier to justify. If all model sizes fail in the same way, the bottleneck is more likely to be the data, captions, VAE, resolution pipeline, or training objective.\n\n## The most useful distinctions\n\nInformation | Why it changes the answer\n---|---\nTraining from scratch vs. continued pretraining | Scratch training must learn the broad visual/text prior; CPT can reuse it but may forget old capabilities\n430M denoiser vs. 430M total pipeline | Stable Diffusion parameter figures often count only the denoising network\nUpdates and global batch size | These determine total sample presentations; unique image count alone does not\nSource and training resolutions | Two million 512px crops and two million native 1536px samples are not equivalent training regimes\nCaption format and density | Short tags, long natural-language captions, and noisy web captions teach different conditional mappings\nSpecific quality failure | More capacity may help fine detail but not necessarily bad crops, missing concepts, or caption noise\nTraining and inference budget | A larger model costs more per update and per denoising step\n\nWhat the ratio hides—and the closest controlled evidence (click for more details) How the answer changes by training type and failure mode (click for more details) A controlled comparison that could answer the question (click for more details) How I would interpret the results, plus references (click for more details)\n\nSo my tentative answer would be:\n\n  * **Do not shrink a useful 2M-image dataset merely to match an apparent Stable Diffusion ratio.**\n  * **430M is not obviously too small; there is even a controlled 430M LDM reference point.**\n  * **A larger model may raise the quality ceiling, especially for fine detail, but it may be worse within a fixed training or inference budget.**\n  * **The next useful step is a controlled learning-curve comparison, not a universal ratio calculation.**\n\n\n\nThe three details that would narrow the recommendation most are whether this is scratch training or CPT, what exactly the 430M count includes, and which specific quality failures are currently visible.",
  "title": "How many images to parameters?"
}