{
"path": "/3mam4f5hzcc2q",
"site": "at://did:plc:7vdlgi2bflelz7mmuxoqjfcr/site.standard.publication/3maigovix422i",
"tags": [
"rust",
"tantivy",
"rockbox"
],
"$type": "site.standard.document",
"title": "Making Search Instant and Forgiving",
"content": {
"$type": "pub.leaflet.content",
"pages": [
{
"id": "019b47ac-7de6-7ddc-9e7f-ebe28755c96e",
"$type": "pub.leaflet.pages.linearDocument",
"blocks": [
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 207,
"byteStart": 177
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
},
{
"index": {
"byteEnd": 309,
"byteStart": 282
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
},
{
"index": {
"byteEnd": 323,
"byteStart": 316
},
"features": [
{
"uri": "https://github.com/quickwit-oss/tantivy",
"$type": "pub.leaflet.richtext.facet#link"
}
]
},
{
"index": {
"byteEnd": 336,
"byteStart": 326
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
}
],
"plaintext": "One of the biggest challenges is handling large local music libraries efficiently. Searching through thousands (or tens of thousands) of tracks, albums, and artists needs to be fast, accurate, and forgiving.\n\nRecently, I leveled up the search experience dramatically by introducing regex-based prefix matching using Tantivy's RegexQuery."
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Let's dive into what changed, why it matters, and how it works under the hood.\n"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.image",
"image": {
"$type": "blob",
"ref": {
"$link": "bafkreiawsgcvn7mwz5gvmhawsbjis27bvmg5ofhf7lhdwwji3mpl3xkidm"
},
"mimeType": "image/png",
"size": 1945598
},
"aspectRatio": {
"width": 2442,
"height": 1704
}
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.header",
"level": 1,
"facets": [],
"plaintext": "\nThe Problem with Basic Search"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Before this update, search relied on Tantivy's standard term or phrase queries. It was decent, but:"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.unorderedList",
"children": [
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Partial matches weren't always prefix-friendly."
},
"children": []
},
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Case sensitivity could trip users up."
},
"children": []
},
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "No easy way to do \"starts with\" across multiple fields without complex query parsing"
},
"children": []
}
]
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "For a music player, search should feel instant and smart—like typing in Spotify or Apple Music, where partial, case-insensitive matches just work.\n"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.image",
"image": {
"$type": "blob",
"ref": {
"$link": "bafkreihnldcf2vsc5o44omqx4jmlakiuenypzwdcejmvabrlnie4pgdftm"
},
"mimeType": "image/png",
"size": 2342444
},
"aspectRatio": {
"width": 2442,
"height": 1704
}
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": ""
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.header",
"level": 1,
"facets": [],
"plaintext": "The Solution: RegexQuery for Prefix Matching"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 115,
"byteStart": 104
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
}
],
"plaintext": "I added a flexible regex layer that transforms the user's query into a case-insensitive prefix pattern: (?i)query.*"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Here's the core snippet:"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.code",
"language": "rust",
"plaintext": "let mut results: Vec<(f32, TantivyDocument)> = vec![];\nlet pattern = format!(\"(?i){}.*\", regex::escape(query_string));\nfor field in fields {\n let query = RegexQuery::from_pattern(&pattern, field)?;\n let regex_results: Vec<(f32, TantivyDocument)> = searcher\n .search(&query, &TopDocs::with_limit(10))?\n .into_iter()\n .map(|(score, doc_address)| (score, searcher.doc(doc_address).unwrap()))\n .collect();\n results.extend(regex_results);\n}",
"syntaxHighlightingTheme": "catppuccin-latte"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": ""
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.unorderedList",
"children": [
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 4,
"byteStart": 0
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
}
],
"plaintext": "(?i) makes it case-insensitive."
},
"children": []
},
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 27,
"byteStart": 0
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
}
],
"plaintext": "regex::escape(query_string) safely escapes special chars."
},
"children": []
},
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 2,
"byteStart": 0
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
}
],
"plaintext": ".* allows matching the rest of the field (prefix + anything)."
},
"children": []
},
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "We run this across all relevant fields (configurable per type)."
},
"children": []
},
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Results are collected, deduped/sorted by score, and returned.\n"
},
"children": []
}
]
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 31,
"byteStart": 11
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
}
],
"plaintext": "This gives true prefix matching with forgiveness on case and exact spelling—type \"dark\" and get \"The Dark Side of the Moon\" instantly."
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Bonus: Tantivy's regex engine is blazing fast for these patterns, so even on huge libraries, results feel instantaneous."
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": ""
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.header",
"level": 1,
"facets": [],
"plaintext": "How the Indexing Works"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 38,
"byteStart": 31
},
"features": [
{
"uri": "https://github.com/quickwit-oss/tantivy",
"$type": "pub.leaflet.richtext.facet#link"
}
]
},
{
"index": {
"byteEnd": 57,
"byteStart": 51
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
},
{
"index": {
"byteEnd": 65,
"byteStart": 59
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
},
{
"index": {
"byteEnd": 78,
"byteStart": 71
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
}
],
"plaintext": "Search is powered by dedicated Tantivy indexes for Tracks, Albums, and Artists."
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 30,
"byteStart": 26
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
},
{
"index": {
"byteEnd": 64,
"byteStart": 58
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
}
],
"plaintext": "Key fields are indexed as TEXT for full-text search, with STORED for quick retrieval."
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "For example, the Track schema indexes essentials like:"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.unorderedList",
"children": [
{
"$type": "pub.leaflet.blocks.unorderedList#listItem",
"content": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 5,
"byteStart": 0
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
},
{
"index": {
"byteEnd": 13,
"byteStart": 7
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
},
{
"index": {
"byteEnd": 20,
"byteStart": 15
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
},
{
"index": {
"byteEnd": 34,
"byteStart": 22
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
},
{
"index": {
"byteEnd": 44,
"byteStart": 36
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
},
{
"index": {
"byteEnd": 51,
"byteStart": 46
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#code"
}
]
}
],
"plaintext": "title, artist, album, album_artist, composer, genre"
},
"children": []
}
]
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "Default searchable fields prioritize the most common ones users query."
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "(Albums focus on title/artist/year; Artists on name/bio.)"
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": "This setup keeps indexes lean while covering 99% of search needs."
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [],
"plaintext": ""
}
},
{
"$type": "pub.leaflet.pages.linearDocument#block",
"block": {
"$type": "pub.leaflet.blocks.text",
"facets": [
{
"index": {
"byteEnd": 12,
"byteStart": 0
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
},
{
"index": {
"byteEnd": 54,
"byteStart": 14
},
"features": [
{
"uri": "https://github.com/tsirysndr/rockbox-zig",
"$type": "pub.leaflet.richtext.facet#link"
}
]
},
{
"index": {
"byteEnd": 68,
"byteStart": 55
},
"features": [
{
"$type": "pub.leaflet.richtext.facet#bold"
}
]
},
{
"index": {
"byteEnd": 124,
"byteStart": 70
},
"features": [
{
"uri": "https://tangled.org/tsiry-sandratraina.com/rockbox-zig",
"$type": "pub.leaflet.richtext.facet#link"
}
]
}
],
"plaintext": "GitHub Repo: https://github.com/tsirysndr/rockbox-zig\nTangled Repo: https://tangled.org/tsiry-sandratraina.com/rockbox-zig"
}
}
]
}
]
},
"description": "Upgraded my Rockbox desktop player's library search with Tantivy's RegexQuery for case-insensitive prefix matching. Now partial queries like 'dark' instantly find 'The Dark Side of the Moon'—fast and forgiving!",
"publishedAt": "2025-12-22T20:51:09.685Z"
}