Sill's new view

Today I'm launching a completely new backend for Sill, one that removes Sill's dependency on the Bluesky AppView and broadens Sill's link aggregation to the rest of the Atmosphere. For the technical details, scroll down to the bottom of this post. First, I'll explain the user-facing improvements that this new backend enables.

Same Sill, just better

For the most part, Sill will look the same to longtime users. It just works better now. The new backend is better at finding links in your network's activity, better at matching slightly different URLs into the same canonical URL, faster to load new activity from your network, and less susceptible to connection errors with your account.

On finding links, Sill is better at finding multiple links in the same post, looking through nested quote posts to find the original link shared, and providing context around where a link came from, whether it was from a custom feed or a reply someone you follow made to someone else.

One of Sill's longstanding pain points is that it wasn't perfect at matching shares that were to the same page, but had differing share URLs (for example, tracking query parameters, gift link codes, URL shorteners). The old backend did its best, but the new one is better at it. As a result, you should see fewer duplicates, and you should see more link cards with rich metadata like publish times and authors.

Activity from across the Atmosphere

Did you know that your Bluesky account can log into a bunch of other services? We call this collection of services the Atmosphere, and the way these products all interoperate is part of a broad vision this developer community has to remake the social internet on human terms.

While Bluesky is still the largest actor on the Atmosphere, there is much more you can do with your account. Sill wants to help you discover what your network is already doing across the Atmosphere through the power of link aggregation.

For example, do you know about Semble? It's a "social knowledge network for learners," letting you bookmark and organize everything you encounter across the web. You can curate collections with others in the network and learn from everyone participating on Semble. It's a great product, and now Sill can show you what your network is bookmarking.

Speaking of bookmarking, Sill also supports the community lexicon for bookmarks. This is what Sill has been using for writing user bookmarks (click the bookmark icon under a link card!), but now it can read them back, and you can see what the rest of your network is bookmarking. These bookmarks also work with Kipclip!

Beyond bookmarking, Sill also finds links from standard.site documents* (like this very blog post). Sill will quote the paragraph the link came from and link you out to the blog post as well.

New ways to browse your links

With better support for rich metadata around links, I can now launch a feature I've had planned for a while. You can now browse your network by domain and by author. To do so, just click on the site name or the author name in a link card. For example, I can see all of the stories from 404 Media that my network has shared over the last two weeks.

If I really like Sam Cole's writing in particular, I can drill down to an author page as well.

Where possible, Sill joins publications and authors to their atproto identity. Right now, Sill is pretty conservative about matching, relying on direct handle matching and Bluesky's Trusted Verification system. Thus, not every publication/author will have fully hydrated profile screens. The worst thing this could do is return false positives, so please let me know if you see any.

What about Mastodon?

From day one, Sill has worked with Mastodon. That's still true today. However, if you only connect a Mastodon account to Sill, you won't see any additional activity beyond Mastodon posts.

The technical details

If you're an atproto dev and you've read this far, you've probably deduced that Sill now runs off of its own AppView. It's a full network-ish AppView optimized for aggregative queries about URLs against a user's social graph. For example: what URLs are most common in a user's network over 24 hours? This query is Sill's default view.

When I talk to atproto devs about Sill, the most common question I get is, "Why not use Constellation?" Constellation is awesome, but it doesn't quite do what Sill needs. Constellation works best when you already know the URL and want every possible reference. Sill's job is to discover what the URLs are in your network. You could probably cobble it together via a bunch of Constellation calls, but it probably wouldn't scale well.

Sill's solution to the problem, back when I originally built it in 2024, was to crib everything off the Bluesky AppView. I looped through every account connected to Sill, called app.bsky.feed.getTimeline, looked for new posts with links in them, and wrote them to Sill's database. It worked for a while, but it didn't scale*. Sill got slower every time a new user signed up. And maintaining all of those OAuth tokens for the worker was a nightmare. But this is why atproto is so cool: I don't have to use the Bluesky AppView to get the data I want. I can also build my own using the firehose (or in my case, jetstream).

The Sill AppView takes a pretty similar approach to Constellation at the start: listen for all records and walk them for anything that resembles a URL. Write down where in the record that URL occurs.

From there, I take a different approach. The Sill AppView special cases some lexicons, namely follow and block lexicons from across the network. With this, it builds social graphs, enabling easy indexing for following timelines. For now, the AppView only builds graphs for Sill users, but I may scale this to the entire network in the future. (Any new user who signs up gets their graph built on-demand, which thankfully only takes a couple seconds.)

I also run a scraper against every URL the walk finds to extract as much metadata as possible. This is what powers the publication/author routes I showed above. Most of the complicated work in this AppView is actually on this side: resolving link shorteners and tracking params to canonical URLs, matching publications and authors to DIDs, adding layers of verification to ensure Sill doesn't pick up impersonations.

In total, the Sill AppView does the following:

It runs on a $60/month OVH box, and honestly, I probably over-provisioned.

Limitations

There are some limitations to this AppView:

What's next

I'll be monitoring the AppView to make sure it holds up to production scale. But once we're stable, this AppView empowers a whole bunch of new features I've wanted to build for Sill for a long time. Also, I'm testing an iOS app. Stay tuned.