External Publication
Visit Post

Conduit to yield first lines from file without leaking space or file handles?

Haskell Community [Unofficial] May 2, 2026
Source

Would a conduit expert be able to write me a conduit that takes in file names and yields the first n lines from each file without leaking space or file handles? I have the following, but it leaks file handles and crashes on my machine if there are more than about 1000 files.

cabal-script-test27.hs: /tmp/manyfiles/1981: openBinaryFile: resource exhausted (Too many open files)



import Conduit
import qualified Data.Conduit.Combinators as CC
import System.Directory (doesFileExist)
import Data.Text (Text)

main :: IO ()
main = runConduitRes $
    CC.sourceDirectoryDeep False "/tmp/manyfiles"
    .| firstNLines 1
    .| CC.mapM_ (liftIO . print)

firstNLines
  :: Int
  -> ConduitT FilePath Text (ResourceT IO) ()
firstNLines n = awaitForever $ \fp ->
    CC.sourceFile fp
    .| CC.decodeUtf8Lenient
    .| CC.linesUnbounded
    .| CC.take n

Discussion in the ATmosphere

Loading comments...