Conduit to yield first lines from file without leaking space or file handles?

danidiaz:

my own toy library jet-stream

Nice! Your Jet type is isomorphic to a special case of Bluefin streaming. Let’s see how.

We have

newtype Jet a = Jet
  { runJet :: forall s. (s -> Bool) -> (s -> a -> IO s) -> s -> IO s
  }

which is

StateT s IO Bool -> (a -> StateT s IO ()) -> StateT s IO ()

(modulo the fact that the stop callback can’t modify the state). Now, I’m pretty sure that we could instead use

(a -> IO ()) -> IO ()

1. The StateT s part of this could be equivalently done by having an IORef in the closure of an IO higher order function
2. Then the stop callback functionality could be handled by a useful combinator until :: IO Bool -> Jet a -> Jet a

If I’m right then this is a special case of Bluefin streams which are implemented under the hood as (a -> IO ()) -> IO ().

I’m pretty sure what you call “continuation-based” streaming libraries are the right way to go.