Best MoA with models under 40B
Hugging Face Forums [Unofficial]
July 3, 2026
This combination is doable with total of around 90GB of RAM and it is superior to DeepSeek-v4-pro Please see my other post. Any one can predict the combination: Drafters:
- Gemma-4-31B-QAT-Q4 with MTP (llama.cpp)
- Ornith-1.0-35B - Q4_k_M (llama.cpp) each with a context of 102400 and 8bit kv cache Aggregator: Qwen3.6-27B-4bit context 204800
Call it MoA-Max:
| Task | MoA-MAX |
|---|---|
| Bug Investigation | 89.2 |
| Tool Orchestration | 89.3 |
| Code Architecture | 87.8 |
| Clinical Decision | 80.4 |
| Multi-step Coding (DLQ) | 91.8 |
| Overall | 87.7 |
Compared to single and other MoA:
| Rank | Model / Configuration | Bug | Tool | Arch | Clinical | DLQ | Avg /100 |
|---|---|---|---|---|---|---|---|
| 1 | MoA-MAX † | 89.2 | 89.3 | 87.8 | 80.4 | 91.8 | 87.7 |
| 2 | MoA: Gemma-12B + Ornith-35B → agg Qwen-3.6-27B † | 80.3 | 86.6 | 88.1 | 90.3 | 88.9 | 86.8 |
| 3 | Qwen-3.6-27B (4-bit, vLLM) | 90.8 | 86.2 | 86.2 | 80.7 | 88.9 | 86.6 |
| 4 | MoA: Gemma-12B + Qwen-27B → agg DeepSeek-v4-Pro | 88.1 | 88.1 | 88.1 | 81.1 | 84.2 | 85.9 |
| 5 | Qwen3.6-35B-A3B (Q4_K_M) | 88.1 | 83.1 | 88.1 | 77.6 | 88.9 | 85.2 |
| 6 | MoA: Gemma-12B + Nemotron-2 → agg Qwen-3.6-27B | 80.5 | 87.5 | 88.1 | 80.5 | 88.9 | 85.1 |
| 7 | MoA: Gemma-12B + Qwen-35B-A3B → agg Qwen-3.6-27B | 81.1 | 88.1 | 88.1 | 77.6 | 88.9 | 84.8 |
| 8 | MoA: Gemma-12B + Qwen-27B → agg Qwen3.6-35B-A3B | 88.1 | 88.1 | 86.6 | 81.1 | 77.6 | 84.3 |
| 9 | DeepSeek-v4-Pro (cloud) | 93.2 | 88.1 | 88.1 | 55.2 | 94.0 | 83.7 |
| 10 | DeepSeek-v4-Flash (cloud) | 79.5 | 88.1 | 88.1 | 73.6 | 84.9 | 82.8 |
| 11 | MoA: Gemma-12B + Qwen-27B → agg Ornith-35B | 80.5 | 82.4 | 84.7 | 81.1 | 84.9 | 82.7 |
| 12 | Ornith1.0-35B (Q4_K_M) | 73.0 | 88.1 | 86.2 | 81.1 | 84.9 | 82.7 |
| 13 | MoA: Gemma-12B + Qwen-27B → agg Nemotron-2 | 59.1 | 61.5 | 78.2 | 61.9 | 72.9 | 66.7 |
| 14 | Gemma-4-12B (4-bit, vLLM) | 69.5 | 51.5 | 56.1 | 43.9 | 49.5 | 54.1 |
| 15 | Nemotron-2 Cascade (Q4_K_M) | 5.8 | 5.8 | 5.8 | 5.8 | 5.8 | 5.8 |
See my other post for details
Discussion in the ATmosphere