{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreihyjw3zv7vook4axxqzno6c7p7su6xjpd67q2gnjomcrszzisayle",
    "uri": "at://did:plc:25rdn5elo5izoxrmtis34zuk/app.bsky.feed.post/3mpszb73qicj2"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreihyzbt5lass6cusgva4uqng5xf6ifuutvg5b3oqx3wxyc3nxcwuwq"
    },
    "mimeType": "image/webp",
    "size": 78436
  },
  "path": "/hiroki-kameyama/taking-rag-to-production-evals-observability-security-and-beyond-introduction-44kb",
  "publishedAt": "2026-07-04T11:45:16.000Z",
  "site": "https://dev.to",
  "tags": [
    "ai",
    "mlops",
    "llm",
    "python",
    "Building a RAG System from Scratch with pgvector and Gemini"
  ],
  "textContent": "##  About This Guide\n\nIn the previous guide, Building a RAG System from Scratch with pgvector and Gemini, we implemented a RAG system from scratch using pgvector and Gemini, then extended it through Tool Use, AI Agents, MCP, and cloud deployment.\n\nThis guide is the sequel. It takes you from _\"building a system that works\"_ to _\"making a system that works in production\"_.\n\n\n\n    [Previous Guide]\n    RAG → Tool Use → AI Agents → MCP → Render × Supabase deployment\n\n    [This Guide]\n    Evals → Observability → Security → MLOps → Fine-tuning\n    → Multi-Agent → Governance\n\n\n##  Why Production Operations Are Hard\n\nAfter implementing RAG or Agents, you'll inevitably hit these problems when trying to go live:\n\n**Quality problems**\nManually checking \"is this answer correct?\" doesn't scale. You need an automated system to measure quality.\n\n**Visibility problems**\nWhen something goes wrong in production, you can't diagnose it if you can't track \"what happened at which step.\"\n\n**Security problems**\nWhen accepting requests from external users, you need to handle prompt injection attacks and prohibited content.\n\n**Continuous improvement problems**\nIf you have no way to verify \"did this actually get better?\" after improving a prompt, your iteration cycle stalls.\n\n**Model problems**\nSometimes a general-purpose model isn't enough — you need a model specialized for your specific domain.\n\n##  Guide Structure\n\nEach chapter can be read independently. The content assumes the previous guide's implementation (pgvector, Gemini, RAG), but you can read for conceptual understanding alone.\n\nChapter | Theme | Problem Solved\n---|---|---\nCh. 2 | Evals | Automated measurement of answer quality\nCh. 3 | Observability | Tracing and cost management\nCh. 4 | Security | Guardrails and attack defense\nCh. 5 | MLOps / LLMOps | CI/CD and prompt management\nCh. 6 | Fine-tuning | Domain-specific model specialization\nCh. 7 | Multi-Agent | Orchestrator × Worker architecture\nCh. 8 | Governance | EU AI Act compliance, audit logs\n\n##  Prerequisites\n\n  * Completed the previous guide's pgvector tutorial\n  * Python 3.11, Docker, pgvector environment set up\n  * `GEMINI_API_KEY` configured in `.env`\n\n\n\n##  Tools Used\n\nTool | Purpose | Free Tier\n---|---|---\nGoogle Gemini API | LLM + Embedding | 1,500 requests/day\npgvector | Vector DB | Unlimited (local)\nLangfuse | Observability | Free cloud tier available\nGitHub Actions | CI/CD pipeline | 2,000 minutes/month (free)\nHugging Face | Fine-tuning models | Free\n\nLet's start with Chapter 2: Evals.",
  "title": "Taking RAG to Production — Evals, Observability, Security, and Beyond (Introduction)"
}