{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreidxlkggbaif2hkdaxhsn3x3bnynhftpqz5wrot4kkuskgj6et2fwu",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mlsyywxlaw62"
  },
  "path": "/t/practical-match-for-128gb-strix-halo-with-2x3090s-inference-for-coding/175977#post_2",
  "publishedAt": "2026-05-14T12:25:44.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "AMD Ryzen AI Max+ 395 official specs",
    "Strix Halo LLM benchmark discussion/results",
    "NVIDIA RTX 3090 official specs",
    "Gigabyte RTX 3090 Gaming OC specs",
    "Intel Core i7-13700K specifications",
    "Hugging Face Transformers quantization docs",
    "BentoML: GPU memory and LLM inference",
    "ASRock Z690 Taichi specifications",
    "ASRock Z690 Taichi specs",
    "MSI RTX 3090 Gaming Trio datasheet",
    "Puget Systems: NVIDIA NVLink compatibility chart",
    "Puget Systems: how to buy a workstation with NVLink",
    "Ollama GPU support",
    "Ollama: new model scheduling",
    "llama.cpp multi-GPU guide",
    "vLLM parallelism and scaling docs",
    "vLLM GGUF docs",
    "SGLang server arguments"
  ],
  "textContent": "Hmm… For now, **unlike a single powerful GPU, multi-GPU setups are more complex to use. Depending on the LLM backend, they may not deliver the performance you expect.** Additionally, depending on the design of your current PC case, you’ll need to consider heat dissipation and where to install the components:\n\n* * *\n\n# Practical answer: 128GB Strix Halo vs 2×RTX 3090 for local LLM coding inference\n\nMy short answer:\n\n**A 2×RTX 3090 desktop is a very realistic upgrade if your real workload is fast local coding inference with 7B–70B models, plus occasional 80B experiments.**\n\n**It is not a clean replacement for a 128GB Strix Halo box if your real target is dense 100B/120B-class models with useful context length.**\n\nThe reason is simple but important:\n\n> **2×RTX 3090 gives you 48GB of extremely fast but split VRAM. Strix Halo gives you up to 128GB of much slower but much more forgiving shared memory.**\n\nThat distinction matters more than CPU choice, motherboard branding, or theoretical TFLOPS.\n\n* * *\n\n## 1. The core tradeoff\n\nYour proposed desktop would be much faster when the model fits well in GPU memory. But it does **not** behave like a 128GB unified-memory machine.\n\n### Practical comparison\n\nWorkload | 2×RTX 3090 desktop | 128GB Strix Halo\n---|---|---\n7B–14B coding models | Excellent; one 3090 is already enough | Works, but usually slower\n30B–34B coding models | Excellent daily-use zone | Works\n70B Q4 | Strong target for dual 3090s | Works, usually slower\n80B Q4 | Possible, but tight; context/KV cache matter a lot | One of the Halo’s better use cases\nDense 100B/120B Q4 | Not a clean fit in 48GB VRAM | Much more forgiving\nLong-context coding | Good until KV cache eats VRAM | More forgiving due to capacity\nGaming | Far better | Impressive for an APU, but not comparable\nCUDA ecosystem | Strong | Not the same experience\nRepairability | Standard desktop parts | Worse\nPower/heat/noise | Much worse | Much better\n“Just load huge quants and experiment” | More tuning required | Better\n\nSo the decision is not “which machine is faster?” The 3090 desktop is faster when it fits.\n\nThe real decision is:\n\n> **Do you mostly want speed for models that fit in 24–48GB VRAM, or do you want capacity/flexibility for models that do not?**\n\n* * *\n\n## 2. What Strix Halo is good at\n\nAMD lists the Ryzen AI Max+ 395 with:\n\n  * up to **128GB** memory,\n  * **256-bit LPDDR5x** memory,\n  * up to **LPDDR5x-8000** memory speed,\n  * **Radeon 8060S** integrated graphics,\n  * **40** GPU cores,\n  * up to **2900MHz** graphics frequency.\n\n\n\nSource: AMD Ryzen AI Max+ 395 official specs\n\nIndependent Strix Halo LLM testing has reported around **215GB/s observed GPU memory bandwidth** from a **256GB/s theoretical** LPDDR5x-8000 memory subsystem.\n\nSource: Strix Halo LLM benchmark discussion/results\n\nThat bandwidth is much lower than RTX 3090 VRAM bandwidth, but the point of Strix Halo is not peak bandwidth. Its value is that it gives you a relatively large shared memory pool in a compact box.\n\nThat makes it good for:\n\n  * large GGUF experiments,\n  * 80B-class quants,\n  * some 100B/120B-class low-bit experiments,\n  * long-context tests,\n  * “will this load?” experimentation,\n  * running a large local model without building a hot dual-GPU tower.\n\n\n\nThe Halo’s weakness is also clear:\n\n  * it is slower than real high-end VRAM,\n  * ROCm/Vulkan/AMD local-LLM workflows can be more limited than CUDA,\n  * gaming is not comparable to a 3090 desktop,\n  * repairability and upgradeability are worse,\n  * you cannot just add another GPU or change the memory later.\n\n\n\n* * *\n\n## 3. What 2×RTX 3090 is good at\n\nNVIDIA lists RTX 3090 as a 24GB GDDR6X Ampere card with **10,496 CUDA cores**.\n\nSource: NVIDIA RTX 3090 official specs\n\nBoard-partner specs commonly list RTX 3090 with:\n\n  * **24GB GDDR6X** ,\n  * **384-bit** memory bus,\n  * around **936GB/s** memory bandwidth,\n  * high board power,\n  * optional NVLink support on many 3090 cards.\n\n\n\nExample source: Gigabyte RTX 3090 Gaming OC specs\n\nTwo 3090s give you:\n\n  * 24GB VRAM on GPU 0,\n  * 24GB VRAM on GPU 1,\n  * 48GB total installed VRAM,\n  * very high bandwidth per card,\n  * CUDA,\n  * strong gaming performance,\n  * much better software support for many inference stacks.\n\n\n\nBut two 3090s do **not** automatically become one transparent 48GB GPU.\n\nThat means dual 3090s are excellent when:\n\n  * the model can be split cleanly,\n  * the model and KV cache mostly stay in VRAM,\n  * the backend supports multi-GPU well,\n  * PCIe layout is sane,\n  * cooling and power are handled properly.\n\n\n\nThey are less excellent when:\n\n  * the model spills heavily to CPU RAM,\n  * the context is long enough to blow up KV cache,\n  * the frontend hides important split/offload knobs,\n  * one card is thermally throttling,\n  * the second GPU is behind a weak chipset link,\n  * you expect dense 120B Q4 to behave like it is on a 128GB accelerator.\n\n\n\n* * *\n\n## 4. Why 48GB VRAM is not the same as 128GB shared memory\n\nThis is the most important point.\n\nA 2×3090 desktop has two memory tiers:\n\n  1. **Fast tier:** 24GB + 24GB GDDR6X VRAM.\n  2. **Slow tier:** system DDR5 RAM.\n\n\n\nThe slow tier can help you load/offload models, but once inference depends heavily on system RAM, the 3090 advantage drops sharply.\n\nIntel’s i7-13700K spec lists:\n\n  * **2** memory channels,\n  * up to **DDR5-5600** official memory speed,\n  * **89.6GB/s** max memory bandwidth.\n\n\n\nSource: Intel Core i7-13700K specifications\n\nThat is perfectly fine for a gaming/dev desktop, but it is far below RTX 3090 VRAM bandwidth and also below the reported Strix Halo GPU-accessible memory bandwidth.\n\nSo the desktop does not behave like:\n\n> 48GB fast VRAM + 128GB RAM = 176GB medium-speed model memory.\n\nIt behaves more like:\n\n> 48GB fast accelerator memory, then a big performance cliff when you spill.\n\nThat is why 2×3090 is attractive for 70B Q4, borderline for 80B Q4, and not a clean dense-120B solution.\n\n* * *\n\n## 5. Model-size reality\n\nQuantization reduces memory use by storing weights in lower precision. Hugging Face’s Transformers docs describe quantization as a way to reduce memory and compute cost using lower-precision weights/activations, and note support for AWQ, GPTQ, and 8-bit/4-bit bitsandbytes quantization.\n\nSource: Hugging Face Transformers quantization docs\n\nBut inference memory is not only model weights. You also need memory for:\n\n  * KV cache,\n  * activations,\n  * framework/runtime buffers,\n  * temporary prompt-processing memory,\n  * fragmentation/allocator overhead.\n\n\n\nBentoML’s LLM memory guide explicitly calls out model weights, activations, and KV cache as major runtime memory consumers.\n\nSource: BentoML: GPU memory and LLM inference\n\n### Rough planning table\n\nThese are not exact numbers. They are planning estimates.\n\nModel size | FP16/BF16 weights | 8-bit weights | ideal 4-bit weights | Practical Q4-ish estimate | 2×3090 judgment\n---|---|---|---|---|---\n7B | ~14GB | ~7GB | ~3.5GB | ~4–6GB | Easy\n14B | ~28GB | ~14GB | ~7GB | ~8–12GB | Easy\n30B–34B | ~60–68GB | ~30–34GB | ~15–17GB | ~18–24GB | Good\n70B | ~140GB | ~70GB | ~35GB | ~38–45GB | Good target\n80B | ~160GB | ~80GB | ~40GB | ~44–52GB | Borderline\n120B | ~240GB | ~120GB | ~60GB | ~66–80GB+ | Not clean\n\nThe key part is not the weight-only size. The key part is the remaining VRAM after weights.\n\nFor coding, context length often matters a lot because prompts may include:\n\n  * multiple files,\n  * stack traces,\n  * logs,\n  * failing tests,\n  * previous conversation,\n  * dependency details,\n  * generated patches.\n\n\n\nA model that fits at 4k context may become unpleasant at 16k or 32k because the KV cache grows.\n\n* * *\n\n## 6. Model-by-model practical judgment\n\n### 7B–14B\n\nA 3090 desktop wins easily.\n\nOne RTX 3090 is already enough. The second card is mainly useful for:\n\n  * running another model simultaneously,\n  * keeping embeddings/rerankers/vision on another GPU,\n  * running a game or other GPU task separately,\n  * experimenting without evicting the daily model.\n\n\n\nFor this model size, Strix Halo is convenient but not the speed winner.\n\n### 30B–34B\n\nThis may be the best daily local-coding zone.\n\nA strong 30B/32B/34B coding model can be more useful in real work than a slow giant model. On a 3090 desktop, this class can be fast enough for interactive use and high-quality enough for code review, debugging, refactoring, and test generation.\n\nIf your daily work is coding, this size range may matter more than 120B.\n\n### 70B Q4\n\nThis is the best argument for 2×3090.\n\nA 70B Q4 model is large enough to benefit from the second GPU, but usually not so large that 48GB is hopeless. If weights and KV cache stay mostly in VRAM, dual 3090s should beat Strix Halo by a lot.\n\nExpected behavior:\n\n  * 4k context: likely good.\n  * 8k context: likely practical.\n  * 16k context: backend/quant/KV-cache settings matter.\n  * 32k context: be cautious.\n  * GGUF/llama.cpp: practical.\n  * vLLM/SGLang: attractive if using supported model formats.\n  * Ollama: convenient, but less transparent.\n\n\n\nIf your real target is fast 70B Q4 coding inference, selling the Halo and building the desktop is rational.\n\n### 80B Q4\n\nThis is the dangerous middle.\n\nAn 80B Q4 model can be close enough to 48GB that small details decide everything:\n\n  * Q4 format,\n  * KV cache precision,\n  * context length,\n  * backend overhead,\n  * split strategy,\n  * batch size,\n  * whether the frontend hides useful knobs.\n\n\n\nMy practical take:\n\n> **2×3090 can be an 80B Q4 machine, but I would not call it a comfortable 80B Q4 machine until you test your exact models and context lengths.**\n\nThis is where the Halo’s value becomes obvious. The Halo may be slower, but it gives you more room to be sloppy.\n\n### Dense 100B/120B Q4\n\nThis is where dual 3090s stop being a clean replacement.\n\nA dense 120B Q4 model generally wants more than 48GB once you include realistic overhead and KV cache. You can still experiment with:\n\n  * lower-bit quants,\n  * shorter context,\n  * CPU offload,\n  * MoE models,\n  * llama.cpp tuning,\n  * vLLM/SGLang where supported,\n  * more GPUs,\n  * larger-VRAM professional cards.\n\n\n\nBut if your goal is:\n\n> “I want to casually load dense 120B Q4 models and use them with useful coding context.”\n\nThen 2×3090 is the wrong comfort zone.\n\nThe Halo is not necessarily fast here, but it is more forgiving.\n\n### MoE models\n\nMixture-of-Experts models are a separate case.\n\nA huge-looking MoE model may activate only part of the total parameter count per token. That can make some large MoE models much more practical than dense models with the same total parameter count.\n\nBut MoE performance is backend-sensitive. Expert placement, active parameters, CPU offload, and runtime support matter a lot. Do not judge MoE models purely by the headline parameter count.\n\n* * *\n\n## 7. Motherboard: is Z690 Taichi good enough?\n\nYes, electrically it is a reasonable dual-3090 platform.\n\nASRock lists the Z690 Taichi with:\n\n  * support for 12th/13th/14th gen Intel Core processors,\n  * 4 DDR5 DIMM slots,\n  * dual-channel DDR5,\n  * two PCIe 5.0 x16 physical slots,\n  * support for x16 or x8/x8 operation on the primary PCIe slots,\n  * an additional chipset-connected long slot.\n\n\n\nSource: ASRock Z690 Taichi specifications\n\nThe important part is the x8/x8 support for the two main GPU slots.\n\nFor two RTX 3090s, this is much better than a board where the second long slot is only chipset x4.\n\n### PCIe x8/x8 is acceptable\n\nRTX 3090 is PCIe 4.0. PCIe 4.0 x8 is not the same as on-card VRAM bandwidth, but it is a reasonable consumer dual-GPU topology.\n\nThe bigger problems are usually:\n\n  * VRAM capacity,\n  * backend split behavior,\n  * GPU-to-GPU communication,\n  * cooling,\n  * power,\n  * card spacing.\n\n\n\n### Physical spacing matters more than people expect\n\nTwo 3090s can be physically difficult.\n\nCheck:\n\n  * card thickness,\n  * slot spacing,\n  * whether the top card can breathe,\n  * case bottom clearance,\n  * front-panel connector conflicts,\n  * PSU cable routing,\n  * NVLink bridge spacing if you care,\n  * memory junction temperatures under sustained load.\n\n\n\nA dual-GPU build can look valid on paper and still be unpleasant because the top card suffocates.\n\n* * *\n\n## 8. CPU: i7 or i9?\n\nI would not overspend on the CPU before fixing GPU, RAM, PSU, and cooling.\n\nA modern i7/i9 is fine. The i7-13700K class is already strong for:\n\n  * gaming,\n  * compiling,\n  * IDE work,\n  * Docker/WSL,\n  * running local services,\n  * feeding the GPUs,\n  * occasional CPU-offload fallback.\n\n\n\nThe i9 is fine if it is cheap enough or useful for your non-LLM workload. But it does not solve the real problem: insufficient VRAM for 100B/120B dense models.\n\nIntel lists the i7-13700K with 2 memory channels and 89.6GB/s max memory bandwidth, which is the same basic memory-channel class as other LGA1700 desktop parts.\n\nSource: Intel Core i7-13700K specifications\n\n### Why not old Threadripper?\n\nOld Threadripper gives you more memory channels and PCIe lanes, but it is not automatically better for your use case.\n\nIt can be worse for:\n\n  * single-core performance,\n  * gaming,\n  * idle power,\n  * platform age,\n  * motherboard availability,\n  * daily desktop feel.\n\n\n\nMore memory channels help CPU offload, but CPU offload is already the fallback path. If the goal is fast LLM inference, the usual answer is more usable accelerator memory, not simply more CPU memory bandwidth.\n\nI would only choose Threadripper/EPYC/Threadripper Pro if you are building a real multi-GPU workstation/server with 3–4 GPUs, lots of RAM, and less concern for gaming.\n\n* * *\n\n## 9. RAM: 64GB is not enough\n\nFor this machine, I would treat **128GB system RAM as the practical minimum**.\n\nNot because system RAM is fast enough to replace VRAM. It is not.\n\nYou want 128GB because your machine will also run:\n\n  * OS,\n  * browser,\n  * IDE,\n  * Docker,\n  * WSL,\n  * model server,\n  * vector DB / RAG tools,\n  * build tools,\n  * test suites,\n  * CPU offload,\n  * large model loading,\n  * multiple model files,\n  * failed experiments that allocate too much memory.\n\n\n\n### RAM choices\n\nRAM amount | Practical view\n---|---\n32GB | Too low for this project\n64GB | Fine for gaming/dev, weak for large-model experiments\n96GB | Reasonable compromise with 2×48GB kits\n128GB | Best baseline\n192GB | Interesting if the board/BIOS/kit are proven stable\n\nASRock’s Z690 Taichi page lists DDR5 support and modern capacity support, but with any high-capacity DDR5 setup you should verify the board QVL, BIOS version, and real-world stability.\n\nSource: ASRock Z690 Taichi specs\n\nFor your purpose, I would rather have stable 128GB than unstable 192GB.\n\n* * *\n\n## 10. PSU and cooling\n\nThis part is not optional.\n\nA single RTX 3090 is already a high-power card. Two of them plus a high-end CPU can be a serious sustained load.\n\nSome board-partner 3090 cards list around 350–370W board power and recommend a 750W PSU for a single-card system.\n\nExample source: MSI RTX 3090 Gaming Trio datasheet\n\n### PSU recommendation\n\nI would use:\n\n  * **1200W minimum** if power-limiting/undervolting,\n  * **1600W preferred** if using an i9, high sustained load, or less tuning.\n\n\n\nAlso:\n\n  * use separate PCIe power cables where possible,\n  * avoid cheap splitters,\n  * avoid questionable old PSUs,\n  * leave transient headroom,\n  * consider power-limiting both 3090s.\n\n\n\nFor inference, power-limiting is often worthwhile. You may lose less performance than expected while reducing heat, noise, and instability.\n\n### Cooling recommendation\n\nTwo open-air 3090s can be difficult.\n\nPlan for:\n\n  * a large airflow case,\n  * strong front intake,\n  * top/rear exhaust,\n  * enough spacing between cards,\n  * GPU memory junction temperature monitoring,\n  * possible thermal pad maintenance,\n  * sane fan curves,\n  * no “sandwiched with no intake” layout.\n\n\n\nSustained inference is not like a short gaming benchmark. It can keep cards hot for a long time.\n\n* * *\n\n## 11. NVLink: useful, not magic\n\nRTX 3090 is one of the few GeForce cards with NVLink support. Puget Systems notes that GeForce RTX 3090, RTX A6000, and RTX A5000 use the newer NVLink bridge generation, not the old 20-series/Quadro RTX bridge generation.\n\nSources:\n\n  * Puget Systems: NVIDIA NVLink compatibility chart\n  * Puget Systems: how to buy a workstation with NVLink\n\n\n\nBut NVLink does **not** mean:\n\n  * every backend sees one 48GB GPU,\n  * VRAM is universally pooled,\n  * 120B dense Q4 becomes comfortable,\n  * no software tuning is needed,\n  * PCIe layout no longer matters.\n\n\n\nTreat NVLink as optional.\n\nBuy it only if:\n\n  * both 3090s support it,\n  * the bridge spacing matches,\n  * it is not expensive,\n  * your backend can benefit from peer access / NCCL / tensor-parallel communication.\n\n\n\nDo not design the whole system around NVLink. Cooling, power, and VRAM capacity matter more.\n\n* * *\n\n## 12. Backend choice changes the answer\n\nThe same hardware can feel excellent or disappointing depending on backend.\n\n### Ollama\n\nOllama is convenient and has improved multi-GPU behavior. The Ollama model-scheduling update says it improved memory management, reduced OOMs, increased GPU utilization, and improved multi-GPU/mismatched-GPU performance.\n\nSources:\n\n  * Ollama GPU support\n  * Ollama: new model scheduling\n\n\n\nOllama is good for:\n\n  * daily convenience,\n  * local API use,\n  * simple model management,\n  * small/medium models,\n  * 70B Q4 tests.\n\n\n\nBut I would not use only Ollama to decide whether to sell the Halo. It may not expose enough control for borderline 80B/120B testing.\n\n### llama.cpp / GGUF direct\n\nThis is probably the most important stack if you use GGUF.\n\nllama.cpp has explicit multi-GPU controls such as split modes and tensor split options.\n\nSource: llama.cpp multi-GPU guide\n\nUseful things to test:\n\n  * `--n-gpu-layers`\n  * `--split-mode`\n  * `--tensor-split`\n  * `--main-gpu`\n  * `--ctx-size`\n  * KV cache type\n  * Flash attention\n  * prompt processing speed\n  * generation speed\n\n\n\nExample benchmark structure:\n\n\n    CUDA_VISIBLE_DEVICES=0,1 ./llama-bench \\\n      -m /models/<model>.gguf \\\n      -p 512,4096,8192,16384 \\\n      -n 256 \\\n      -ngl 999\n\n\nExample layer split:\n\n\n    CUDA_VISIBLE_DEVICES=0,1 ./llama-cli \\\n      -m /models/<model>.gguf \\\n      --n-gpu-layers 999 \\\n      --split-mode layer \\\n      --tensor-split 24,24 \\\n      --ctx-size 8192 \\\n      --cache-type-k q8_0 \\\n      --cache-type-v q8_0 \\\n      -p \"<real coding prompt>\"\n\n\nExample tensor split:\n\n\n    CUDA_VISIBLE_DEVICES=0,1 ./llama-cli \\\n      -m /models/<model>.gguf \\\n      --n-gpu-layers 999 \\\n      --split-mode tensor \\\n      --tensor-split 24,24 \\\n      --ctx-size 8192 \\\n      -p \"<same real coding prompt>\"\n\n\nFor 2×3090, llama.cpp direct testing is more informative than just trying a desktop frontend.\n\n### vLLM\n\nvLLM is attractive if you are willing to use a server-style inference stack.\n\nIts docs say that for multi-GPU inference, you set `tensor_parallel_size` to the desired GPU count.\n\nSource: vLLM parallelism and scaling docs\n\nExample:\n\n\n    CUDA_VISIBLE_DEVICES=0,1 vllm serve <model-id-or-path> \\\n      --tensor-parallel-size 2 \\\n      --max-model-len 8192\n\n\nvLLM is good if:\n\n  * you use OpenAI-compatible local endpoints,\n  * you want throughput,\n  * you run multiple requests,\n  * you use Hugging Face model formats,\n  * you want explicit tensor parallelism.\n\n\n\nBut vLLM is not primarily a GGUF desktop-app workflow. Its GGUF docs warn that GGUF support is highly experimental and under-optimized.\n\nSource: vLLM GGUF docs\n\n### SGLang\n\nSGLang is another serious local serving option. Its docs say to enable multi-GPU tensor parallelism with `--tp 2`.\n\nSource: SGLang server arguments\n\nExample:\n\n\n    CUDA_VISIBLE_DEVICES=0,1 python -m sglang.launch_server \\\n      --model-path <model-id-or-path> \\\n      --tp 2\n\n\nSGLang is worth testing if you want a real local inference server and not just a desktop frontend.\n\n* * *\n\n## 13. What I would buy/change\n\nFor your stated goal, I would prioritize in this order:\n\n  1. **Second RTX 3090 24GB**\n  2. **Motherboard with real CPU-lane x8/x8**\n  3. **128GB DDR5**\n  4. **High-quality 1200W–1600W PSU**\n  5. **Large airflow case**\n  6. **Cooling/thermal-pad/fan plan**\n  7. **Large NVMe storage for models**\n  8. **Optional NVLink bridge**\n  9. **CPU upgrade only after the above is solved**\n\n\n\n### CPU\n\nI would choose:\n\n  * i7-13700K/KF-class if value matters,\n  * i9-13900K/KF-class only if close in cost or useful outside LLMs,\n  * avoid old Threadripper unless building a dedicated multi-GPU workstation/server.\n\n\n\nThe CPU is not the main local-LLM bottleneck once the model is GPU-resident.\n\n### RAM\n\nI would choose:\n\n  * 128GB DDR5 as baseline,\n  * stability over headline XMP speed,\n  * board-QVL/BIOS-verified kits where possible.\n\n\n\nDo not stop at 64GB for this use case.\n\n### Motherboard\n\nThe Z690 Taichi idea is reasonable because it has the right kind of dual-GPU slot layout. It is not magical, but it is not a bad starting point.\n\nThe main question is physical fit and thermals, not whether x8/x8 is theoretically enough.\n\n### PSU\n\nI would not build dual 3090s on a marginal PSU.\n\nUse:\n\n  * 1200W minimum if tuned,\n  * 1600W if you want comfort.\n\n\n\n### Case\n\nA dual-3090 build can fail because of airflow even when every spec looks correct. Check the exact GPU cooler sizes before buying the case or board.\n\n* * *\n\n## 14. Should you sell the Halo?\n\n### Sell it if your real workload is:\n\n  * 7B–14B fast local assistants,\n  * 30B–34B coding models,\n  * 70B Q4,\n  * occasional 80B testing,\n  * CUDA experiments,\n  * gaming,\n  * wanting one repairable main PC.\n\n\n\nIn that case, the 2×3090 desktop is a better fit.\n\n### Keep it if your real workload is:\n\n  * dense 80B/100B/120B experiments,\n  * long-context coding,\n  * large GGUF models,\n  * “load first, optimize later” testing,\n  * a quiet compact inference box,\n  * a second local endpoint.\n\n\n\nIn that case, the Halo still has a clear role.\n\n### Best hybrid workflow\n\nThe strongest reason to keep both is not that they combine into one giant machine. It is that they can serve different roles:\n\n  * **3090 desktop:** fast small/medium/70B models, CUDA, gaming.\n  * **Strix Halo:** larger slower models, long-context experiments, large-memory fallback.\n\n\n\nThat can be very useful for coding. You can use a fast local model for iteration and keep a larger model available for deeper or slower checks.\n\nBut if two personal machines plus a work laptop is too much, and the Halo resale window is unusually good, selling it is rational.\n\n* * *\n\n## 15. Benchmark before selling\n\nDo not benchmark only one model and one prompt.\n\nTest at least:\n\nTest | Why\n---|---\n14B coding model | Low-latency daily assistant\n30B–34B coding model | Best quality/speed daily zone\n70B Q4 | Main dual-3090 target\n80B Q4 | Borderline decision point\n100B/120B or large MoE | Decides whether Halo still matters\n\nTest these context sizes:\n\nContext | Why\n---|---\n4k | Basic chat/coding\n8k | Realistic coding context\n16k | Multi-file work\n32k | Stress test\n\nRecord:\n\n  * prompt processing tokens/sec,\n  * generation tokens/sec,\n  * time to first token,\n  * VRAM on GPU 0,\n  * VRAM on GPU 1,\n  * system RAM usage,\n  * whether CPU offload occurs,\n  * GPU utilization,\n  * GPU temperature,\n  * memory junction temperature,\n  * power draw if possible.\n\n\n\nThe key question is not:\n\n> “Can it technically run?”\n\nThe key question is:\n\n> “Does it remain pleasant with the prompts I actually use for coding?”\n\nA 70B model that feels responsive is often more useful than a 120B model you avoid using because it is slow and fragile.\n\n* * *\n\n## 16. Final recommendation\n\nI would build the 2×3090 desktop if your goal is:\n\n> fast local coding inference, good gaming, CUDA, repairability, and one main PC.\n\nI would keep the Halo if your goal is:\n\n> frequent dense 80B/100B/120B experiments, long-context tests, and large-memory convenience.\n\nMy concrete answer:\n\n**Z690 Taichi + modern i7/i9 + 128GB DDR5 + 2×RTX 3090 is realistic and sensible for 7B–70B local coding inference.**\n\n**It is a borderline 80B Q4 machine.**\n\n**It is not a comfortable dense 120B Q4 machine.**\n\nIf you sell the Halo, sell it with the correct expectation:\n\n> You are trading large-memory flexibility for much higher speed, better gaming, CUDA, and repairability.\n\nThat is a reasonable trade if 70B and below are your real daily targets.\n\nIt is a bad trade if the Halo’s main value to you is casually loading huge 100B/120B-class quants that do not fit neatly into 48GB VRAM.\n\n* * *\n\n## Short summary\n\n  * **2×RTX 3090 is excellent for 7B–70B.**\n  * **80B Q4 is possible but needs testing.**\n  * **Dense 120B Q4 is not a clean fit in 48GB VRAM.**\n  * **Z690 Taichi-style x8/x8 is acceptable.**\n  * **Use 128GB RAM, not 64GB.**\n  * **Prioritize PSU, cooling, and GPU spacing over i9 bragging rights.**\n  * **Do not buy old Threadripper unless you want a true multi-GPU workstation/server.**\n  * **NVLink is optional, not magic.**\n  * **Ollama is convenient; llama.cpp direct is better for testing; vLLM/SGLang are better if you accept server-style inference.**\n  * **Sell the Halo if your daily use is fast 30B/70B coding plus gaming.**\n  * **Keep the Halo if large-memory 80B/100B/120B experiments are genuinely frequent.**\n\n",
  "title": "Practical match for 128Gb Strix Halo with 2x3090s? (inference for coding)"
}