Skip to content

Examples ​

This section shows 10 common problems and how to solve them with goflow. Each page is a short, complete recipe. Replace the placeholder types and functions (fetch, store, Event, ...) with your own.

#PatternMain operators
1Parallel HTTP RequestsFanMapFilter
2File Import with Bulk InsertsFromIter, MapFilter, Split
3Background Job QueuePipeFunc, Process
4Event Buffering for a Slow SinkPipe, Batch
5Paginated API CrawlFromFunc, Take
6Rate-Limited Calls with DeduplicationDistinct, Throttle
7Merge Multiple SourcesFanIn
8Broadcast to Multiple SinksTee
9Moving Average and Anomaly DetectionWindow
10First Match with Early ExitFanMapFilter, FindFirst

Error Handling Rules ​

Each page shows the happy path first, then an error handling section. These rules apply to all patterns:

  1. An error in a stage does not reach the terminal. When Map, MapFilter or the function of FromFunc returns an error, the stream closes. Collect and ForEach then end normally. To report the error, carry it as a value in the element, or write it to a buffered channel before the function returns.
  2. Never stop one branch of a fan-out. An error in a FanMap or FanMapFilter function stops one worker. A Tee consumer that returns stops one branch. FanOut and Tee then block, and the stream stalls until the context ends. Return errors as values, or cancel the context.
  3. Cancel the context when a terminal returns early. ForEach returns at the first error, and FindFirst returns at the first element. The upstream stages keep running until the context ends.
  4. Retry one element, not the stage. Wrap the call in gofuncy.Do(ctx, fn, gofuncy.WithRetry(n)). WithRetry on WithOptions restarts the whole stage goroutine.
  5. Keep long-lived consumers alive. Pass goflow.WithErrorHandler to ForEach or Process. The handler gets each error, and the consumer goes on.

See Error Handling for the operator details.