Retry function in Go
Redowan Delowar
February 4, 2024
I used to reach for reflection whenever I needed a Retry function in Go. It's fun to
write, but gets messy quite quickly.
Here's a rudimentary Retry function that does the following:
- It takes in another function that accepts arbitrary arguments.
- Then tries to execute the wrapped function.
- If the wrapped function returns an error after execution, Retry attempts to run the
underlying function n times with some backoff.
The following implementation leverages the reflect module to achieve the above goals.
We're intentionally avoiding complex retry logic for brevity:
The Retry function uses reflection to call a function passed as any. It takes the
function's arguments as a []any slice, allowing us to run functions with varied
signatures. Using reflect.ValueOf(fn).Call(argVals), it dynamically invokes the target
function after converting arguments from any to reflect.Value.
The retry logic runs up to maxRetry times with exponential backoff. The delay starts at
startBackoff, doubles after each failure, and caps at maxBackoff. If the last return
value is an error and retries remain, it waits and tries again. Otherwise, it gives up.
You can wrap a dummy function that always returns an error to see how Retry works:
Running it will give you the following output:
This isn't too terrible for reflection-heavy code. But now that Go has generics, I wanted to
see if I could avoid the metaprogramming. While reflection is powerful, it's prone to
runtime panics and the compiler can't type-check dynamic code.
Turns out, there's a way to write the same functionality with generics if you don't mind
trading off some flexibility for shorter and more type-safe code. Here's how:
Functionally, the generic implementation works the same way as the previous one. However, it
has a few limitations:
- The generic Retry assumes the wrapped function returns (result, error). This fits Go's
common idiom, but the reflection version could handle varied return patterns.
- The reflection-based Retry wraps any function via the empty interface. The generic
version requires a matching signature, so you need a thin wrapper to adapt it.
Here's how you'd use the generic Retry function:
Running it will give you the same output as before.
Notice how someFunc uses a closure to capture a and b rather than accepting them as
arguments. This adaptation is necessary for type safety. I don't mind it if it means
avoiding reflection - plus the generic version is slightly faster.
After this entry went live, [Anton Zhiyanov pointed out on Twitter] that there's a
closure-based approach that's even simpler and eliminates the need for generics. The
implementation looks like this:
Now calling Retry is easier since the closure signature is static - you don't need to
adapt the call when the wrapped function's signature changes:
The runtime behavior of this version is the same as the ones before.
Fin!
[anton zhiyanov pointed out on twitter]:
https://twitter.com/ohmypy/status/1754105508863393835
Discussion in the ATmosphere