{
  "$type": "site.standard.document",
  "content": {
    "$type": "pub.lemma.blog.entry",
    "content": "# Local generative AI on Linux\r\n\r\n![Alt text](https://cdn.bsky.app/img/feed_fullsize/plain/did:plc:5g2hkj2od4zr3tlpb2su364e/bafkreieearqdmyibndqo7csbjmxaet5sfpl7qnmk75jnfdrvzmskktpxlm@jpeg \"Title\")\r\n\r\n*This process is tested on Kubuntu 26.04 with an NVIDIA RTX GeForce GPU but should be very similar on other distros.*\r\n\r\nKeep your data on your machine and avoid paid subscriptions or usage limits by running generative AI workloads on your own computer.\r\n\r\n## Images\r\n\r\nWe'll use [StabilityMatrix](https://github.com/LykosAI/StabilityMatrix/releases), a package manager for Stable Diffusion. Think of it as the Steam of image generation software. The nice thing about it is that you only download the models once and they are shared between the UIs you install.\r\n\r\n1. Download the .zip file and extract it. It will be an AppImage.  \r\n   Apparently, modern Ubuntu-based systems use a newer version of the \"filesystem\" (FUSE 3), but AppImages still require the older version (FUSE 2) to \"mount\" and run. So we'll install the compatibility layer for AppImages.  \r\n   *Note: In 26.04, the package is specifically named libfuse2t64 rather than just libfuse2*  \r\n   ```sh\r\n   sudo apt install libfuse2t64\r\n   ```\r\n   Remember to make `StabilityMatrix.AppImage` executable and open it.\r\n2. Install the **ComfyUI** package for a simple UI or **Stable Difussion WebUI reForge** for lots of control over the process.\r\n3. Download the checkpoint tha fits your needs e.g. for photorealism or cartoon imagery. For my 12 GB VRAM card, the Juggernaut XL model worked very well.\r\n4. Launch your package. For ComfyUI you can just go straight to the **Inference** tab and it will connect to it with no need to launch a browser window.\r\n5. On that screen (or on the browser window that was opened) select your checkpoint, image dimensions e.g. 1280×720, sampling steps (higher may give you better quality but will be slower), and hit *Generate*.\r\n\r\n## Chatbots\r\n\r\nWe'll use the new [Anaconda Desktop](https://www.anaconda.com/products/desktop).\r\n\r\n1. Download the `.deb` package and install it with Discover. Open the app.\r\n2. Head to AI Models and download the ones you want. For example, my GPU has 12 GB VRAM so it can easily handle Microsoft phi-4 which is a model specialized in mathematical reasoning and coding that has a maximum RAM usage of 9.44 GB.\r\n3. Click the *New* button to start a chat and select your model.\r\n4. Chat!\r\n\r\n*It's important to note that as of Spring 2026 Anaconda Desktop is in Beta so you might run into stability issues.*\r\n\r\nAll done! Of course, depending on the task you may not get the same quality or features you'd get from a paid product. But for simple use cases like a chatbot and quick code or image generation it's a great deal.\r\n"
  },
  "path": "/3mkqdwm4g7s25",
  "publishedAt": "2026-04-30T19:28:46.883Z",
  "site": "at://did:plc:5g2hkj2od4zr3tlpb2su364e/site.standard.publication/3mmhg6ulmlszy",
  "title": "Local generative AI on Linux",
  "updatedAt": "2026-05-01T06:04:35.746Z"
}