Why does Go's io.Reader have such a weird signature?
Redowan Delowar
February 8, 2025
I've always found the signature of io.Reader a bit odd:
Why take a byte slice and write data into it? Wouldn't it be simpler to create the slice
inside Read, load the data, and return it instead?
This felt more intuitive to me - you call Read, and it gives you a slice filled with data,
no need to pass anything.
I found out why it's designed this way while watching this excellent [GopherCon Singapore
talk] on understanding allocations by Jacob Walker. It mainly boils down to two reasons.
Reducing heap allocations
If Read created and returned a new slice every time, the memory would always end up on the
heap.
Heap allocations are slower because they require garbage collection, while stack allocations
are faster since they are freed automatically when a function returns. By taking a
caller-provided slice, Read lets the caller control memory and reuse buffers, keeping them
on the stack whenever possible.
This matters a lot when reading large amounts of data. If each Read call created a new
slice, you'd constantly be allocating memory, leading to more work for the garbage
collector. Instead, the caller can allocate a buffer once and reuse it across multiple
reads:
Go's escape analysis tool (go build -gcflags=-m) can confirm this. If Read returned a
new slice, the tool would likely show:
meaning Go has to allocate it dynamically. But by reusing a preallocated slice, we avoid
unnecessary heap allocations - only if the buffer is small enough to fit in the stack. How
small? Only the compiler knows, and you shouldn't depend on it. Use the escape analysis tool
to see that. But most of the time, you don't need to worry about this at all.
Reusing buffers in streaming
The second issue is correctness. When reading from a stream, you usually call Read
multiple times to get all the data. If Read returned a fresh slice every time, you'd have
no control over memory usage across calls. Worse, you couldn't efficiently handle partial
reads, making buffer management unpredictable.
With the hypothetical version of Read, every call would allocate a new slice. If you
needed to read a large stream of data, you'd have to manually piece everything together
using append, like this:
This is a mess. Every time append runs out of space, Go will have to allocate a larger
slice and copy the existing data over, piling on unnecessary GC pressure.
By contrast, io.Reader's actual design avoids this problem:
This avoids unnecessary allocations and produces less garbage for the GC to clean up.
[gophercon singapore talk]:
https://www.youtube.com/watch?v=ZMZpH4yT7M0
Discussion in the ATmosphere