{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreigzcxuipeuvgb4ydahudc2wjuud4arzcp36cbjob3jxpxgorzmv2i",
    "uri": "at://did:plc:64jmmaef3uvbywm67gi4qq76/app.bsky.feed.post/3mh6wl6laj672"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreie4k5e2p5r4pdm4tllqpo6xv2kf65rkdww557oh2kzeiafi6oeydu"
    },
    "mimeType": "image/jpeg",
    "size": 804880
  },
  "description": "Last week the Rogue Scholar science blog archive launched a self-hosted system to systematically collect metrics, logs and errors, with monitoring of service uptime and alert messages. The source code of the observability service is available at https://github.com/front-matter/invenio-observability.",
  "path": "/posts/increasing-operational-transparency/",
  "publishedAt": "2026-03-16T16:58:46.000Z",
  "site": "https://blog.front-matter.de",
  "tags": [
    "Grafana",
    "ViktoriaMetrics",
    "ViktoriaLogs",
    "InvenioRDM",
    "uptime monitor",
    "GlitchTip",
    "Sentry integration",
    "prometheus-flask-exporter",
    "structlog",
    "Vector",
    "https://github.com/front-matter/invenio-observability",
    "inveniosoftware Discord Forum",
    "Slack",
    "Mastodon",
    "Bluesky",
    "donation"
  ],
  "textContent": "Running scholarly infrastructure such as Rogue Scholar beyond the initial launch phase – what is typically called Day 2 operations – requires the following activities:\n\n  * monitoring\n  * incident management\n  * maintenance and updates\n  * scaling\n\n\n\nAs Rogue Scholar is self-hosted infrastructure the natural fit for these activities is self-hosted observability frameworks. One central element is the Grafana dashboard monitoring service health:\n\nThe dashboard collects metrics (ViktoriaMetrics) and logs (ViktoriaLogs) from the InvenioRDM respository platform that powers Rogue Scholar, and monitors the service availability from an external uptime monitor. Errors are sent to the self-hosted GlitchTip service using the Sentry integration built into InvenioRDM.\n\nStandard prometheus metrics were added to InvenioRDM using the prometheus-flask-exporter library, and log files for the InvenioRDM web and worker services are generated in structlog JSON format and reformatted into ViktoriaLogs format using Vector.\n\nThis new infrastructure not only simplifies monitoring service health, but also makes easier specific investigations, for example exploring the bot traffic Rogue Scholar receives each day:\n\nFuture work could include more detailed monitoring of celery workers or a dedicated dashboard for authentication and access monitoring.\n\nThe Rogue Scholar observability service is generic enough to be useful for any InvenioRDM repository instance, uses self-hosted open source software, and can be deployed via the included Docker Compose file. The source code is available at https://github.com/front-matter/invenio-observability, for question send an email or reach out via the inveniosoftware Discord Forum.\n\nPlease use Slack, email, Mastodon, or Bluesky if you have any questions or comments.\n\nRogue Scholar is a scholarly infrastructure that is free for all authors and readers. You can support Rogue Scholar with a one-time or recurring donation or by becoming a sponsor.",
  "title": "Increasing operational transparency in Rogue Scholar",
  "updatedAt": "2026-03-16T16:58:47.362Z"
}