Layering Local LLM Over Commercial LLM for Privacy?

Mac mini probably wouldn’t be sufficient as the most you can get now is 48GB (and paying a premium for that). You’d probably need a studio and the answer is very expensive.

MacRumors

Apple Cuts More Mac Studio and Mac Mini RAM Options as Memory Shortage Worsens

Apple has removed more desktop Macs from its online store as the global memory shortage continues. Mac mini models with 32GB and 64GB of RAM are no longer available for purchase, nor is the M3 Ultra Mac Studio with 256GB RAM. The M3 Ultra Mac Studio...

MacRumors

Mac Studio 512GB RAM Option Disappears Amid Global DRAM Shortage

Apple quietly updated Mac Studio configuration options this week, removing the 512GB memory upgrade. As of yesterday, there is no option to purchase a Mac Studio with 512GB RAM, with the machine now maxing out at 256GB. The Mac Studio starts with...

Also in terms of stack, a lot of these models/toolsets are on a more mature Linux stack, (that’s whats also going to be running in a datacenter) so if you’re self hosting I wouldn’t be buying a mac for that. Essentially Linux support on macs sucks, and won’t ever be good because of the distinctly different hardware, and you’re at Apple’s mercy for support for however long that is.

The benefits of the unified memory will mostly be a lot more achievable with CAMM2 DDR6 RAM - and i the enterprise space SOCAMM2.

One of the other things to note, is that NVIDIA has a clear advantage with CUDA as a lot of models are balanced around that. AMD’s ROCm is getting there, but still a ways to go. Whatever Apple has is not going to ever get the same kind of density usage as Apple has zero enterprise footprint.