{
  "$type": "site.standard.document",
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreiavcwqege5rt4o24iaai7j5ymfnc5bg3hukrbzks3wfuy55tv72dq"
    },
    "mimeType": "image/png",
    "size": 30878
  },
  "description": "How I built a local-first app for planning trips, and what I learned about the current state of the local-first ecosystem along the way.",
  "path": "/words/a-local-first-case-study/",
  "publishedAt": "2024-10-01T00:00:00Z",
  "site": "at://did:plc:vrrdgcidwpvn4omvn7uuufoo/site.standard.publication/3mmyfl3pxzi2a",
  "tags": [
    "crdts",
    "localfirst",
    "svelte",
    "yjs"
  ],
  "textContent": "I just got back from a travel sabbatical.\nWhile the trip turned out great, the planning process was decidedly… less so.\nFiguring out six months of travel is a daunting task, and I quickly became dissatisfied with existing tools.\n\nTrue to myself, I yak shaved the problem.\nIntroducing Waypoint: a local-first web app for planning trips!\n\nYou might be thinking \"hey, that looks a lot like that trip planning app Ink & Switch built\", and you'd be right: Embark was the single biggest influence on Waypoint.\nIn fact, Embark is even more ambitious — pulling in data like weather forecasts, embedding arbitrary views like calendars and introducing a new formula language for live calculations.\nI highly recommend reading their writeup!\nBut Ink & Switch didn't make Embark public, and I needed to plan a long trip, so here we are.\n\nI want to talk about three things: the big ideas behind Waypoint, how I actually built it and what I learned.\n\n(Quick disclaimer: Waypoint is not — and probably will never be — production-ready software.\nI built it to fit my exact needs while planning this trip.\nThere are rough edges, missing features and bugs. There's no authentication.\nI'm sharing it because I think it's a useful case study in building an actual local-first app, not because I'm trying to dethrone Google Maps.)\n\nWhy I Built Waypoint\n\nI tried a few existing tools before deciding to build my own.\nApple Notes was too spartan, Notion and Google Maps were too clunky and Wanderlog was much too structured to use for research and exploration.:\nIn every tool, it was either difficult to enter rough, unstructured ideas, or difficult to take those ideas and create a more formal plan.\n\nWaypoint addresses three important shortcomings of other tools:\n\nData entry should be quick\n\nComparisons should be easy\n\nUnstructured data is just as important as structured data\n\nIn short, I wanted an app where I could jot down loose notes about places I was interested in visiting, visualize different routes and gradually narrow it all down into an actual itinerary.\n\nThe interface I landed on has two panels: a text editor on the left and a map on the right.\n\nOne common task when planning a trip is gathering a list of locations you're interested in visiting.\n\nThe simplest solution is using a normal text editor.\nData entry is quick; the only real limiting factor is how fast you can type.\nThe obvious drawback is that locations are displayed textually rather than plotted on a map, obscuring any spatial relationship between them.\n\nThe only dedicated tool for this that I really know of is Google's My Maps (the neglected stepchild of Google Maps).\nIt nails the spatial visualization criterion.\nBut data entry is awkward and slow; tasks like organizing places into groups require a lot of clicking.\n\nIn Waypoint, the main interactive component is a rich text editor.\nYou use it just as you would Google Docs or Microsoft Word — type notes, add some formatting, cut and paste lines to rearrange your thoughts.\nAdding a location is as easy as typing its name, using an @mention-style autocomplete inspired by Embark.\nCharacters show up as quickly as you can type them, and any changes are reflected instantly on the map view beside the document.\n\nEven when apps make data entry easy, that data is often transient, making it difficult to see comparisons.\n\nFor example, if you want to see where two locations are relative to each other in Apple or Google Maps, you're forced to use the navigation feature to create a route between them.\nAnd only one route is visible at a time — to see a different set of locations, you need to clear the route you're currently looking at.\nThis makes it very difficult to, say, determine which of a group of locations are near each other in order to cluster them on different days of an itinerary.\n\nIn Waypoint, every location is plotted on a map, so you always have a bird's eye view of your trip.\nTo show routes, you can create a \"route list\" by beginning a line with ~ (just as you would with - for a bulleted list, or 1. for a numbered list).\nEvery location in the list has a route drawn between its marker on the map and the next one.\nBy default, the routes are the driving directions between the two locations, but you can toggle between that and a straight line by clicking on the location name and unchecking “Navigate”.\n\nIt's easy to add, remove and rearrange locations in the route: just use the text editing commands you already know to edit the list, and the map automatically updates!\nTo compare two routes, you can just copy and paste the whole list and rearrange as you see fit.\n\nA bird's eye view is nice, but sometimes you want to \"zoom in\" on a subset of your work.\nTo accommodate this, Waypoint also includes a focus mode — inspired by iA Writer — which dims all paragraphs other than the one under your text cursor.\nOn the map, Waypoint only shows the locations and routes in that paragraph.\n\nTogether, these features enable a powerful workflow: make a route list, copy and paste it below, alter the second list, enable focus mode and move your cursor between the two to quickly see the difference between them.\nNo other tool I tried made this nearly as quick or as easy.\n\nUnder the Hood\n\nAt a glance, Waypoint isn't too different from your average single-page app:\n\nThe website as a whole is built with SvelteKit.\n\nCustom widgets such as tooltips and dropdowns use the Shoelace web component library.\n\nThe rich text editor is built atop the excellent ProseMirror toolkit.\n\nThe maps and location search are powered by Stadia Maps and the open source MapLibre GL JS library.\n\nData is stored on the client using the Yjs CRDT library.\n\nHold up — that last one seems kinda weird?\n\nIt's actually the key difference between Waypoint and a traditional single-page app.\nRather than storing data on a server using a database like MySQL or Postgres, Waypoint is a local-first app that stores its data on the client using a CRDT.\n\n(Some brief exposition: CRDTs are data structures that can be stored on different computers and are guaranteed to eventually converge upon the same state.\nFor a fuller explanation, check out my article An Interactive Intro to CRDTs, which breaks down the fundamental ideas behind CRDTs and how they work.)\n\nCRDTs are often used to build collaborative experiences like you might see in Google Docs or Figma — except rather than requiring a centralized server to resolve conflicts, the clients can do it themselves.\nThat decentralized sync allows the clients to store the canonical state of the data, rather than a copy fetched from a web server.\n\nThis approach confers some important benefits:\n\nEditing is instantaneous and synchronous.\nThere are no loading spinners, no optimistic updates to roll back if a request fails and no \"go online to save your changes\".\nThe app is faster and more reliable to use, and much easier to develop.\n\nIf I decide to stop hosting Waypoint, you'll still have the file with your data.\nThat file will work in any copy of Waypoint, without the need to set up special infrastructure.\n\nThat's why this kind of app is called local-first.\nIf you have the app and you have your data, you can still work on it — even if you're not connected to the Internet or the developer has gone out of business.\n\nAll of this might seem like overkill for a personal app with a single user.\nBut I was planning this trip with my wife, Sarah, so Waypoint quickly needed realtime collaboration.\nTo address that, Waypoint uses a library called Y-Sweet by Jamsocket.\n\nThere are two parts to Y-Sweet:\n\n@y-sweet/client: an npm package that gets included in the client-side bundle. This package is a Yjs “provider” — a plugin that syncs a Yjs document somewhere.\n\nThe Y-Sweet server: a websocket server that syncs documents between clients and persists them to S3.\n\nArchitecturally, Y-Sweet acts as a bus: clients connect to the Y-Sweet server rather than directly to each other.\nWhenever a client connects or makes changes, it syncs its local document with the Y-Sweet server.\nY-Sweet merges the client's document into its own copy, saves it to S3 and broadcasts updates to other clients.\nSince CRDTs are guaranteed to eventually converge on the same state, at the end of this process all clients have the same document.\n\nThis also makes it easy to share documents.\nEach Waypoint document is identified by a UUID.\nWhen a user opens a link with a given document's UUID, their Waypoint client connects to Y-Sweet and tries to sync their local copy with Y-Sweet's copy.\nIf that user has never opened that document, they have no local copy, and the sync operation results in them just getting Y-Sweet's copy in its entirety.\n\nHere's a diagram of Waypoint's architecture after introducing Y-Sweet:\n\nOne reasonable objection here is that it looks an awful lot like a traditional client-server app — just replace Y-Sweet with a normal application server and S3 with a database.\nDoesn't that defeat the whole purpose of local-first?\n\nInk & Switch addresses this in the case study of their Pushpin software:\n\nThus, in addition to local data storage on each device, the cross-device data synchronisation mechanism should also depend on servers to the least degree possible, and servers should avoid taking unnecessary responsibilities. Where servers are used, we want them to be as simple, generic, and fungible as possible, so that one unavailable server can easily be replaced by another. Further, these servers should ideally not be centralised: any user or organisation should be able to provide servers to serve their needs.\n\nYou can think of Y-Sweet as a \"cloud peer\".\nUnder the hood, it runs plain old stock Yjs — the exact same code that runs on the client.\nIf you connected Waypoint to your own Y-Sweet server, there would be no discernible difference.\nTo borrow Ink & Switch's parlance: it's \"simple, generic, and fungible\".\n\nY-Sweet is one of two Yjs providers that Waypoint uses.\nThe other, called y-indexeddb, takes care of offline editing: it persists the Yjs document to the browser's local IndexedDB storage.\nEven if a user gets disconnected from the Internet, edits a document and then closes their browser, none of their work will be lost.\n\nIs It Local-First?\n\nA popular question lately: what actually counts as local-first?\n\nMy mantra is \"if the client has the canonical copy of the data, it's local-first\".\nBut Ink & Switch formalizes this with seven proposed ideals.\nLet's see how Waypoint stacks up:\n\nNo spinners: your work at your fingertips. While Waypoint's location autocomplete and map are subject to network latency, editing the document itself happens instantly. Verdict: yes.\n\nYour work is not trapped on one device. By simply visiting a link, you can load a Waypoint document written anywhere. Plus, you can download your data and open it in any given Waypoint instance. Verdict: yes.\n\nThe network is optional. Again, other than the autocomplete and map, Waypoint is fully functional offline. Verdict: yes.\n\nSeamless collaboration with your colleagues. Waypoint supports both realtime and asynchronous collaboration, using a CRDT to resolve conflicts. Verdict: yes.\n\nThe long now. Although Y-Sweet is a generic server, autocomplete and maps use a proprietary service called StadiaMaps. However, documents can still be viewed without requiring outside infrastructure. Verdict: sorta.\n\nSecurity and privacy by default. Y-Sweet stores copies of documents unencrypted in an S3 bucket. Verdict: no.\n\nYou retain ultimate ownership and control. The canonical copies of data are stored on the client, with no limitations enforced by the server. Verdict: yes.\n\nFive \"yes\", one \"sorta\" and one \"no\".\nKeep in mind that all the relevant technologies are off-the-shelf; most of these capabilities came for free by choosing Yjs (although any given CRDT library would have worked similarly) and Y-Sweet.\nNot bad!\n\nTakeaways\n\nOkay, so what did I learn?\n\nMost importantly, local-first is not some pie-in-the-sky dream architecture.\nAlthough there are still problems to be worked out, it's very possible to build a useful local-first app, today, with existing tools.\n\nIt helps a lot that various libraries in the ecosystem compose well.\nJust snapping together ProseMirror, Yjs and Y-Sweet gave me a collaborative rich text editor with shared cursors.\nAdding in yjs-indexeddb made it work offline.\nThis was all mostly out of the box, with very little setup; the degree to which everything Just Works is impressive.\n\nThat said, I think this is a best-case scenario — text editors seem to be the most \"plug and play\" genre of local-first app.\nBut in general, the building blocks all fit together nicely.\n\nThe same can't be said of Svelte — or, presumably, frontend JavaScript frameworks in general — which needed some massaging to work with Yjs.\n\nTo determine when to re-render, \"reactive\" frameworks like Svelte and Solid track property access using Proxies, whereas \"immutable\" frameworks like React rely on object identity.\nA Yjs document is a class instance that mutates its internal state, which doesn't play well with either paradigm.\nTo have Svelte re-render when the document changed, I had to trick it into invalidating its state:\n\nEven so, in a lot of ways the developer experience was still much better than in a traditional single-page app.\nHere's (roughly) the code to update the document title:\n\nSure, there's some weird CRDT-related boilerplate, but still: no async function, no try…catch, no worrying about the server.\nI just set the title and move on with my life; Yjs will worry about syncing it in the background.\n\nThat might sound like magic, but I think it's just a natural consequence of a fundamentally better abstraction.\nUsing a local-first architecture rather than client-server promises to dramatically simplify single-page apps.\n\nI was dreading adding offline support, but it turned out to be surprisingly easy.\nSvelteKit supports service workers out of the box, and the documentation even provides some example code as a starting point.\nIt wasn't perfect, but it got me probably 95% of the way there — I could load any document I'd already opened, even without an Internet connection.\nAnd as far as saving edits made offline, integrating y-indexeddb took one single line of code.\n\nDive In\n\nI hope you enjoyed this!\nI had a lot of fun building Waypoint.\nThis was my first hands-on foray into the local-first ecosystem, and it turned out to be a lot smoother than I anticipated.\n\nIf you want to see the code behind this explanation, you can find it on GitHub.",
  "title": "A Local-First Case Study"
}