Whispers in the Dark: Gleam Edition

A long time ago twitter had some fun APIs, and I made a project called Whispers in the Dark. It took a reduced version of the twitter firehose and put up recent tweets on the screen, one after the other, but eventually the sampled firehose disappeared and API access started to cost money, so I let Whispers in the Dark go. Since the election though, Bluesky has really taken off. It now has over 20 million users and more than 3 million daily active users. Bluesky also is a very open platform with an underlying protocol that includes a firehose which is opened to users. The firehose is fairly heavy duty, imposing a pretty large network and computational load as it aims to fully synchronize remote servers with the state of the Bluesky servers. I don't need such a heavy duty solution for Whispers in the Dark, but Bluesky also offers a lightweight solution to the firehose. The Bluesky Jetstream is a websocket protocol which delivers a stream of JSON encoded events from Bluesky, including posts by users. This is perfect to make a new Whispers in the Dark. It also gave me an opportunity to make small project using a new programming language called Gleam. What is Gleam? > Gleam is a friendly language for building type-safe systems that scale! Gleam is a functional language that runs on the BEAM), but also can compile to javascript. It is a typed, functional language which can be an adjustment for people who have only used procedural or object oriented languages. Gleam data structures are immutable, there are no for loops, and there are no exceptions. The language designer, Louis Pilfold borrowed from a number of languages, including Erlang, Rust, and Go, to create a simple, friendly language that someone can learn in an afternoon, and it lives up to that promise. The language tour is all you need to get up and productive in the language, though knowing details of the BEAM can be helpful. I find that it lives up to this promise of being simple, and I actually enjoy writting Gleam code. The language tools seem very mature and the compiler is very helpful. Examples in the language tour are good, and the code is very readable. I decided that Whispers in the Dark could be a good project to cut my teeth with the language. The code is posted on github The Standard and Other Libraries A language is not syntax alone. A standard library is important, but there are a lot of disagreements over what standard should mean. Gleam has a rather spare standard library, but what is there works well. There is also already a great ecosystem of public libraries which can be found through the awesome gleam repository or through the gleam packages site. Whispers uses a number of public libraries, all of which seem very reliable and easy to use. The libraries I used are described below. - The Stratus websocket client library is what I use to connect to the jetstream and load the posts. I still need to debug what happens when the websocket unexpectedly disconnects, but so far I haven't had the opportunity to do that because the connection is rock solid. - The Wisp web framework runs the frontend. It is easy to understand to anyone who has used a small web framework such as Python's Flask, or Go's internal http server. - The Nakai library can generate HTML in Gleam. I hadn't used a library like this in the past, but I find Nakai is very intuitive and in the context of this app is a simpler way to produce dynamic HTML. Dynamic HTML is particularly important here as I use htmx on the frontend. a result I had to set it to reconnect after a delay. All these libraries, for a language that just hit 1.0 this year, are impressive and made it very easy to develop Whispers in the Dark. Concurrency on the Erlang Open Telecom Platform (OTP) The BEAM was designed to run massively scalable telecommunication platforms. It is where Gleam inherits its ability to work at scale, and central to this is the Open Telecom Platform (OTP). I am just learning about the BEAM, but the concurrency model Gleam encourages is to use actors which can receive and reply to messages, and maintain state. The central component of an actor is a function which given a message and an incoming state produces a new state and potentially a reply to that message. Messages are consumed one by one in a ordered queue. Since there is no shared memory and only immutable data structures are passed around, this makes it easier to reason about how multiple actors can interact. These actors run as lightweight processes within the BEAM. The Stratus library, and the Wisp framework both work using actors (or possibly somewhat more complex processes) to respond to websocket messages, or web events. In order to bridge the gap between the incoming Bluesky posts on the websocket and the outgoing posts provided by the webserver is an actor I wrote called holder which holds the most recent message provided by the websocket and sends that to any receiver that asks for it. The code for the actor is very consise, and I don't have to worry about any locking when I get a new post from the websocket or when I send a post back to the web frontend. This means that the concurrency really is almost effortless. Summary Writing Whispers in the Dark again was a lot of fun. I am still thinking of making some tweaks to the presentation layer, but it's shown that Gleam is very capable. The community is also very welcoming. I look forward to seeing how the language progresses.