Good data to test tensor based soft sparsity and other sparsity models

Oh. Hmm, I see. In that case, how about something like this?:

That Jennrich reference seems to narrow the empirical question in a useful way.

I also found Jennrich’s later paper, Rotation to Simple Loadings Using Component Loss Functions: The Oblique Case. It explicitly cites Katz and Rohlf’s Functionplane paper and extends the component-loss-function treatment to oblique rotation.

My reading is that this shifts the most interesting benchmark away from merely asking whether an ideal concave CLF can recover an exactly perfect simple structure under ideal population conditions. The more informative empirical questions may be:

• what happens with approximate rather than perfect simple structure;
• whether small but genuine components or cross-loadings are preserved;
• sensitivity to finite-sample noise and spectral artifacts;
• local solutions and initialization;
• behavior near ill-conditioned transformations;
• whether the metric and inverse-metric machinery prevents the particular form of basis collapse you have in mind;
• computational behavior at thousands of spectra and thousands of variables.

For the Raman application, I found one dataset that seems unusually close to the scale and comparison setting you described: the experimental Sugar Mixtures data used in Hyperspectral unmixing for Raman spectroscopy via physics-constrained autoencoders, with its reproduction code on GitHub.

It is also now included in the larger RamanBench collection.

The experiment used mixtures of:

• glucose;
• sucrose;
• fructose;
• maltose;
• water.

The mixture concentrations are known, and additional reference spectra were measured from pure solutions. This seems particularly useful because it provides both a relatively favorable case with reference endmembers and a more difficult blind-unmixing case without adding those references to the observed mixture data.

The RamanBench regression version exposes four sugar-concentration targets; its benchmark excludes water as a supervised target because that target did not pass its learnability criterion. For an unmixing experiment, however, the original paper and repository are probably the more relevant references because they retain the five-component physical formulation and the pure-solution spectra.

So this may be a more direct practical candidate than MicroMass for the Raman side:

• it is real experimental Raman data;
• it is approximately the requested thousands-by-thousands size;
• mixture composition is known;
• high- and low-SNR conditions are available;
• pure reference spectra exist;
• there are published comparison methods and open reproduction code.

A possible evaluation using the Sugar Mixtures data (click for more details) A matching synthetic Raman benchmark (click for more details)

There is also a second Hugging Face candidate, SubstrateMixRaman, with:

• 6,960 spectra;
• 1,878 columns;
• independently varied concentrations of eight fermentation-related metabolites;
• additional variation from mineral salt medium and antifoam;
• a CC BY 4.0 license.

That looks useful for a larger concentration-recovery or robustness test, especially because low concentrations were deliberately represented and the nuisance components affect overall signal intensity.

I would place it after the Sugar Mixtures experiment, however. The Sugar data have a clearer published blind-unmixing setup, pure-solution reference spectra, established unmixing baselines, and matched high-/low-SNR conditions. SubstrateMixRaman appears more naturally suited to concentration prediction and practical stress testing unless an appropriate pure-reference library is also available.

So a relatively compact default route could now be:

1. Fully synthetic simple-structure test Known loadings, known transformation, conditioning ladder.

2. RamanSPy synthetic mixtures Known endmembers and abundances, with realistic Raman artifacts.

3. Sugar Mixtures, high SNR Real spectra under the easier acquisition condition.

4. Sugar Mixtures, low SNR Real noise-robustness comparison.

5. Reference-assisted and blind variants Separate discovery of endmembers from fitting known endmembers.

6. SubstrateMixRaman or another RamanBench dataset Larger practical stress test.

This also gives a clean way to separate several claims:

Regarding publication on Hugging Face: if the artifact is primarily source code, mathematics, and a reusable executable component, a Model Card can certainly serve as an overview and entry point. For the generated tests and result fixtures themselves, a separate Dataset repository and Dataset Card may be easier for others to load and compare.

A practical division could be:

• Zenodo: archival releases, mathematics, DOI;
• source repository: Fortran, Octave wrapper, build and test scripts;
• HF Dataset repo: synthetic inputs, known truths, experimental configurations, outputs and metrics;
• Model Card or project page: explanation, intended uses, limitations, and links to the other artifacts.

The Jennrich result seems to provide a strong theoretical connection. The Sugar Mixtures data may then provide a rather good empirical connection: not merely “does an ideal CLF recover perfect structure?”, but “what happens with weak components, experimental noise, missing pure endmembers, local solutions, and conditioning in a reproducible Raman problem?”