Examples
This section shows 10 common problems and how to solve them with goflow. Each page is a short, complete recipe. Replace the placeholder types and functions (fetch, store, Event, ...) with your own.
| # | Pattern | Main operators |
|---|---|---|
| 1 | Parallel HTTP Requests | FanMapFilter |
| 2 | File Import with Bulk Inserts | FromIter, MapFilter, Split |
| 3 | Background Job Queue | PipeFunc, Process |
| 4 | Event Buffering for a Slow Sink | Pipe, Batch |
| 5 | Paginated API Crawl | FromFunc, Take |
| 6 | Rate-Limited Calls with Deduplication | Distinct, Throttle |
| 7 | Merge Multiple Sources | FanIn |
| 8 | Broadcast to Multiple Sinks | Tee |
| 9 | Moving Average and Anomaly Detection | Window |
| 10 | First Match with Early Exit | FanMapFilter, FindFirst |
Error Handling Rules
Each page shows the happy path first, then an error handling section. These rules apply to all patterns:
- An error in a stage does not reach the terminal. When
Map,MapFilteror the function ofFromFuncreturns an error, the stream closes.CollectandForEachthen end normally. To report the error, carry it as a value in the element, or write it to a buffered channel before the function returns. - Never stop one branch of a fan-out. An error in a
FanMaporFanMapFilterfunction stops one worker. ATeeconsumer that returns stops one branch.FanOutandTeethen block, and the stream stalls until the context ends. Return errors as values, or cancel the context. - Cancel the context when a terminal returns early.
ForEachreturns at the first error, andFindFirstreturns at the first element. The upstream stages keep running until the context ends. - Retry one element, not the stage. Wrap the call in
gofuncy.Do(ctx, fn, gofuncy.WithRetry(n)).WithRetryonWithOptionsrestarts the whole stage goroutine. - Keep long-lived consumers alive. Pass
goflow.WithErrorHandlertoForEachorProcess. The handler gets each error, and the consumer goes on.
See Error Handling for the operator details.
