How many images to parameters?
Hmm… for now, I don’t think the number of images per parameter alone gives us enough information to decide:
I would not reduce the number of useful images just to make an images-per-parameter ratio look more similar to Stable Diffusion. I also would not assume that increasing 430M to 860M will automatically improve quality.
The main reason is that “430M parameters / 2M images” combines several different constraints:
- what the 430M parameters belong to;
- whether this is training from scratch or continued pretraining;
- how many times the images are actually presented;
- the semantic, caption, resolution, and aspect-ratio distribution of those images;
- the training compute available;
- and what kind of “quality” is currently missing.
There is an unusually close reference point: Bigger is not Always Better: Scaling Properties of Latent Diffusion Models includes an exact 430M-parameter denoising UNet in a controlled family ranging from 39M to 5B parameters. That at least shows that 430M is a perfectly plausible model size rather than an obviously undersized configuration.
However, that 430M model was trained from scratch under a very different regime—about 600M image-text pairs, 500K updates, and global batch 2048—so I would not transfer its image/model ratio directly to a 2M-image dataset. The useful lesson from that paper is instead that model size, training compute, and inference budget need to be compared separately.
My default route would be:
- Keep the 2M useful images.
- Check whether the current 430M model is still improving on held-out data.
- Separate the current failure mode —underfitting, prompt alignment, rare concepts, composition, high-resolution detail, or overfitting.
- Compare the current model with a moderately larger model in a small controlled run , rather than immediately committing to a full larger-model training run.
- Compare at both:
- matched sample presentations; and
- matched training compute or wall-clock budget.
If the larger model wins under both comparisons, scaling it up becomes much easier to justify. If all model sizes fail in the same way, the bottleneck is more likely to be the data, captions, VAE, resolution pipeline, or training objective.
The most useful distinctions
| Information | Why it changes the answer |
|---|---|
| Training from scratch vs. continued pretraining | Scratch training must learn the broad visual/text prior; CPT can reuse it but may forget old capabilities |
| 430M denoiser vs. 430M total pipeline | Stable Diffusion parameter figures often count only the denoising network |
| Updates and global batch size | These determine total sample presentations; unique image count alone does not |
| Source and training resolutions | Two million 512px crops and two million native 1536px samples are not equivalent training regimes |
| Caption format and density | Short tags, long natural-language captions, and noisy web captions teach different conditional mappings |
| Specific quality failure | More capacity may help fine detail but not necessarily bad crops, missing concepts, or caption noise |
| Training and inference budget | A larger model costs more per update and per denoising step |
What the ratio hides—and the closest controlled evidence (click for more details) How the answer changes by training type and failure mode (click for more details) A controlled comparison that could answer the question (click for more details) How I would interpret the results, plus references (click for more details)
So my tentative answer would be:
- Do not shrink a useful 2M-image dataset merely to match an apparent Stable Diffusion ratio.
- 430M is not obviously too small; there is even a controlled 430M LDM reference point.
- A larger model may raise the quality ceiling, especially for fine detail, but it may be worse within a fixed training or inference budget.
- The next useful step is a controlled learning-curve comparison, not a universal ratio calculation.
The three details that would narrow the recommendation most are whether this is scratch training or CPT, what exactly the 430M count includes, and which specific quality failures are currently visible.
Discussion in the ATmosphere