{
  "$type": "site.standard.document",
  "canonicalUrl": "https://rednafi.com/go/io-reader-signature/",
  "description": "Understand why io.Reader takes a byte slice parameter instead of returning one. Learn about heap allocations and buffer reuse in Go streams.",
  "path": "/go/io-reader-signature/",
  "publishedAt": "2025-02-08T00:00:00.000Z",
  "site": "at://did:plc:fgtm2c26vfcj74rfmeggbyqj/site.standard.publication/3mnl6f7ob462z",
  "tags": [
    "Go",
    "TIL",
    "Performance"
  ],
  "textContent": "I've always found the signature of io.Reader a bit odd:\n\nWhy take a byte slice and write data into it? Wouldn't it be simpler to create the slice\ninside Read, load the data, and return it instead?\n\nThis felt more intuitive to me - you call Read, and it gives you a slice filled with data,\nno need to pass anything.\n\nI found out why it's designed this way while watching this excellent [GopherCon Singapore\ntalk] on understanding allocations by Jacob Walker. It mainly boils down to two reasons.\n\nReducing heap allocations\n\nIf Read created and returned a new slice every time, the memory would always end up on the\nheap.\n\nHeap allocations are slower because they require garbage collection, while stack allocations\nare faster since they are freed automatically when a function returns. By taking a\ncaller-provided slice, Read lets the caller control memory and reuse buffers, keeping them\non the stack whenever possible.\n\nThis matters a lot when reading large amounts of data. If each Read call created a new\nslice, you'd constantly be allocating memory, leading to more work for the garbage\ncollector. Instead, the caller can allocate a buffer once and reuse it across multiple\nreads:\n\nGo's escape analysis tool (go build -gcflags=-m) can confirm this. If Read returned a\nnew slice, the tool would likely show:\n\nmeaning Go has to allocate it dynamically. But by reusing a preallocated slice, we avoid\nunnecessary heap allocations - only if the buffer is small enough to fit in the stack. How\nsmall? Only the compiler knows, and you shouldn't depend on it. Use the escape analysis tool\nto see that. But most of the time, you don't need to worry about this at all.\n\nReusing buffers in streaming\n\nThe second issue is correctness. When reading from a stream, you usually call Read\nmultiple times to get all the data. If Read returned a fresh slice every time, you'd have\nno control over memory usage across calls. Worse, you couldn't efficiently handle partial\nreads, making buffer management unpredictable.\n\nWith the hypothetical version of Read, every call would allocate a new slice. If you\nneeded to read a large stream of data, you'd have to manually piece everything together\nusing append, like this:\n\nThis is a mess. Every time append runs out of space, Go will have to allocate a larger\nslice and copy the existing data over, piling on unnecessary GC pressure.\n\nBy contrast, io.Reader's actual design avoids this problem:\n\nThis avoids unnecessary allocations and produces less garbage for the GC to clean up.\n\n\n\n\n\n[gophercon singapore talk]:\n    https://www.youtube.com/watch?v=ZMZpH4yT7M0",
  "title": "Why does Go's io.Reader have such a weird signature?"
}