Operators
This page lists every goflow operator grouped by category. Each entry shows the Go signature, a description, and a short usage example. Most entries have three tabs: Diagram, Code and Signature.
The diagrams are animations. Blue elements go into the operator box. Green elements come out. Purple elements move between two operators. A red element is an error.
Constructors
Of
Creates a stream from variadic values. The items are buffered into a channel of matching capacity.
s := goflow.Of(ctx, "a", "b", "c")func Of[T any](ctx context.Context, items ...T) Stream[T]From
Wraps an existing receive channel as a stream with the given context.
ch := make(chan int, 10)
s := goflow.From(ctx, ch)func From[T any](ctx context.Context, source <-chan T) Stream[T]FromIter
Creates a stream from a Go 1.23 iter.Seq[T] by pushing elements into a channel.
s := goflow.FromIter(ctx, slices.Values([]int{1, 2, 3}))func FromIter[T any](ctx context.Context, seq iter.Seq[T]) Stream[T]FromFunc
Creates a stream from a blocking function that sends items via the provided send callback. The function should block until it is done producing items and must respect context cancellation. bufSize controls backpressure.
s := goflow.FromFunc(ctx, 16, func(ctx context.Context, send func(int) error) error {
for i := 0; i < 100; i++ {
if err := send(i); err != nil {
return err
}
}
return nil
})func FromFunc[T any](ctx context.Context, bufSize int, fn func(ctx context.Context, send func(T) error) error, opts ...gofuncy.GoOption) Stream[T]Generate
Returns an infinite stream where each element is produced by fn. The stream runs until the context is cancelled.
s := goflow.Generate(ctx, func() int { return rand.Intn(100) })func Generate[T any](ctx context.Context, fn func() T) Stream[T]Iterate
Returns an infinite stream: seed, fn(seed), fn(fn(seed)), and so on. The stream runs until the context is cancelled.
s := goflow.Iterate(ctx, 1, func(n int) int { return n * 2 })
// 1, 2, 4, 8, 16, ...func Iterate[T any](ctx context.Context, seed T, fn func(T) T) Stream[T]OfMap
Returns a stream of KeyValue[K, V] pairs from the given map. Iteration order is non-deterministic, matching Go map semantics.
s := goflow.OfMap(ctx, map[string]int{"x": 1, "y": 2})func OfMap[K comparable, V any](ctx context.Context, m map[K]V) Stream[KeyValue[K, V]]Empty
Returns an immediately closed, empty stream.
s := goflow.Empty[int]()func Empty[T any]() Stream[T]Pipe
Creates a writable stream entry point. Returns a *Sender[T] and the readable stream. Send blocks and returns ErrClosed or the send context's error; Try never blocks and returns ErrFull or ErrClosed. The entry point closes when ctx is done or Close is called. The stream and its stages run on ctx and stop when it ends. To drain a long-lived pipeline, use PipeFunc.
Options: WithBuffer, WithName, WithMeterProvider, WithTracerProvider.
send, s := goflow.Pipe[string](ctx, goflow.WithBuffer(16))
go func() {
send.Send(ctx, "hello")
send.Send(ctx, "world")
send.Close(ctx)
}()func Pipe[T any](ctx context.Context, opts ...Option) (*Sender[T], Stream[T])PipeFunc
Creates a Pipe and launches the consumer fn in a gofuncy.Go goroutine. Returns only the Sender. fn and its stages run on a context that ctx does not cancel; fn must return when its stream ends. Close(ctx) waits until fn returns.
send := goflow.PipeFunc[int](ctx, func(ctx context.Context, s goflow.Stream[int]) error {
return s.ForEach(func(_ context.Context, n int) error {
fmt.Println(n)
return nil
})
})
send.Send(ctx, 42)func PipeFunc[T any](ctx context.Context, fn func(context.Context, Stream[T]) error, opts ...Option) *Sender[T]Transformers
Map
Returns a new stream by applying fn to each element. If fn returns an error, the stream closes and the error is handled by gofuncy.Go.
doubled := goflow.Map(s, func(_ context.Context, n int) (int, error) {
return n * 2, nil
})func Map[T, U any](s Stream[T], fn func(context.Context, T) (U, error)) Stream[U]FlatMap
Applies fn to each element, producing a sub-stream per element, and flattens the results into a single output stream sequentially.
expanded := goflow.FlatMap(s, func(ctx context.Context, n int) goflow.Stream[int] {
return goflow.Of(ctx, n, n*10, n*100)
})func FlatMap[T, U any](s Stream[T], fn func(context.Context, T) Stream[U]) Stream[U]MapFilter
Maps and filters in a single pass. (val, true, nil) emits val; (_, false, nil) skips the item; (_, _, err) stops the stream.
parsed := goflow.MapFilter(s, func(_ context.Context, raw string) (int, bool, error) {
n, err := strconv.Atoi(raw)
if err != nil {
return 0, false, nil // skip unparseable
}
return n, true, nil
})func MapFilter[T, U any](s Stream[T], fn func(context.Context, T) (U, bool, error)) Stream[U]Flatten
Flattens a stream of slices into a stream of individual elements.
flat := goflow.Flatten(goflow.Of(ctx, []int{1, 2}, []int{3, 4}))
// 1, 2, 3, 4func Flatten[T any](s Stream[[]T]) Stream[T]Reverse
Collects all elements and emits them in reverse order.
reversed := goflow.Of(ctx, 1, 2, 3).Reverse()
// 3, 2, 1func (s Stream[T]) Reverse() Stream[T]WARNING
Reverse buffers the entire stream in memory before emitting. Do not use on unbounded streams.
Sort
Collects all elements, sorts them using cmp, and emits in sorted order.
sorted := goflow.Of(ctx, 3, 1, 2).Sort(func(a, b int) int { return a - b })
// 1, 2, 3func (s Stream[T]) Sort(cmp func(T, T) int) Stream[T]WARNING
Sort buffers the entire stream in memory before emitting. Do not use on unbounded streams.
Split
Groups consecutive elements into batches of size n. The last batch may contain fewer than n elements.
batches := goflow.Split(goflow.Of(ctx, 1, 2, 3, 4, 5), 2)
// [1,2], [3,4], [5]func Split[T any](s Stream[T], n int) Stream[[]T]Window
Emits sliding windows of n consecutive elements. If the source has fewer than n elements, no windows are emitted.
windows := goflow.Window(goflow.Of(ctx, 1, 2, 3, 4), 3)
// [1,2,3], [2,3,4]func Window[T any](s Stream[T], n int) Stream[[]T]Batch
Groups consecutive elements into batches. A batch is flushed when it reaches MaxItems(n), when the next element would push it over MaxBytes(n, size), when MaxWait(d) has passed since its first element, or when the input closes (the partial batch is always flushed). Without options, Batch behaves like MaxItems(1). An element larger than MaxBytes forms a batch of its own. The flush reason (count, bytes, interval, close) is recorded in metrics and spans.
Unlike Split (count only) and Window (sliding), Batch also flushes by size and time, which suits long-lived streams.
batches := goflow.Batch(events,
goflow.MaxItems(200),
goflow.MaxBytes(400<<10, func(e Event) int { return len(e.Payload) }),
goflow.MaxWait(5*time.Second),
)func Batch[T any](s Stream[T], opts ...Option) Stream[[]T]Filters
Filter
Returns a stream containing only elements where fn returns true.
evens := goflow.Of(ctx, 1, 2, 3, 4).Filter(func(_ context.Context, n int) bool {
return n%2 == 0
})func (s Stream[T]) Filter(fn func(context.Context, T) bool) Stream[T]Distinct
Deduplicates elements using a key function. First occurrence wins.
unique := s.Distinct(func(item Item) string { return item.ID })func (s Stream[T]) Distinct(key func(T) string) Stream[T]WARNING
Distinct maintains a map of all seen keys in memory. For streams with high cardinality, memory usage grows without bound.
Take
Emits the first n elements then closes the stream.
first3 := goflow.Of(ctx, 1, 2, 3, 4, 5).Take(3)
// 1, 2, 3func (s Stream[T]) Take(n int) Stream[T]Skip
Drops the first n elements and emits the rest.
rest := goflow.Of(ctx, 1, 2, 3, 4, 5).Skip(2)
// 3, 4, 5func (s Stream[T]) Skip(n int) Stream[T]Peek
Calls fn for each element as a side-effect and forwards the element unchanged. Useful for logging or debugging.
s.Peek(func(_ context.Context, n int) {
log.Printf("processing: %d", n)
})func (s Stream[T]) Peek(fn func(context.Context, T)) Stream[T]Throttle
Rate-limits the stream to at most one element per duration d.
limited := s.Throttle(100 * time.Millisecond)func (s Stream[T]) Throttle(d time.Duration) Stream[T]Consumers
Collect
Drains the stream and returns all elements as a slice.
items := goflow.Of(ctx, 1, 2, 3).Collect()func (s Stream[T]) Collect() []TCount
Returns the number of elements in the stream.
n := goflow.Of(ctx, 1, 2, 3).Count()
// 3func (s Stream[T]) Count() intForEach
Consumes the stream, calling fn for each element. Returns the first error from fn or the context, nil when fully consumed. With WithErrorHandler(h), errors from fn go to h and consumption continues; ForEach then only returns context errors or nil.
err := s.ForEach(func(_ context.Context, item Item) error {
return db.Save(item)
})func (s Stream[T]) ForEach(fn func(context.Context, T) error, opts ...Option) errorReduce
Folds all elements into a single value using fn. Returns the accumulated result or the first error from fn.
sum, err := goflow.Reduce(goflow.Of(ctx, 1, 2, 3), 0,
func(_ context.Context, acc, n int) (int, error) {
return acc + n, nil
},
)func Reduce[T, U any](s Stream[T], initial U, fn func(context.Context, U, T) (U, error)) (U, error)ToMap
Collects all stream elements into a map using the key and value functions. If duplicate keys occur, the last value wins.
m := goflow.ToMap(s, func(u User) string { return u.ID }, func(u User) User { return u })func ToMap[T any, K comparable, V any](s Stream[T], key func(T) K, value func(T) V) map[K]VFindFirst
Returns the first element and true, or the zero value and false if the stream is empty.
first, ok := s.FindFirst()func (s Stream[T]) FindFirst() (T, bool)FindFirstMatch
Returns the first element matching the predicate and true, or the zero value and false if no element matches.
admin, ok := s.FindFirstMatch(func(_ context.Context, u User) bool {
return u.Role == "admin"
})func (s Stream[T]) FindFirstMatch(fn func(context.Context, T) bool) (T, bool)AllMatch
Returns true if all elements match the predicate. Short-circuits on the first non-matching element. Returns true for an empty stream.
allPositive := s.AllMatch(func(_ context.Context, n int) bool { return n > 0 })func (s Stream[T]) AllMatch(fn func(context.Context, T) bool) boolAnyMatch
Returns true if any element matches the predicate. Short-circuits on the first matching element. Returns false for an empty stream.
hasNegative := s.AnyMatch(func(_ context.Context, n int) bool { return n < 0 })func (s Stream[T]) AnyMatch(fn func(context.Context, T) bool) boolNoneMatch
Returns true if no elements match the predicate. Short-circuits on the first matching element. Returns true for an empty stream.
noErrors := s.NoneMatch(func(_ context.Context, r Result) bool { return r.Err != nil })func (s Stream[T]) NoneMatch(fn func(context.Context, T) bool) boolMin
Returns the minimum element according to cmp and true, or the zero value and false if the stream is empty.
smallest, ok := s.Min(func(a, b int) int { return a - b })func (s Stream[T]) Min(cmp func(T, T) int) (T, bool)Max
Returns the maximum element according to cmp and true, or the zero value and false if the stream is empty.
largest, ok := s.Max(func(a, b int) int { return a - b })func (s Stream[T]) Max(cmp func(T, T) int) (T, bool)Process
Consumes the stream with n long-lived workers. All errors are collected and returned via errors.Join. With WithErrorHandler(h), errors go to h instead and are not retained. Use WithGroupOptions(...) to pass gofuncy.GroupOption values to the worker group.
err := s.Process(4, func(ctx context.Context, item Item) error {
return upload(ctx, item)
})func (s Stream[T]) Process(n int, fn func(context.Context, T) error, opts ...Option) errorConcurrency
FanOut
Distributes elements round-robin across n output streams.
streams := s.FanOut(3)func (s Stream[T]) FanOut(n int) []Stream[T]WARNING
FanOut blocks on the slowest consumer. If one output stream is not being consumed, the entire pipeline stalls.
FanIn
Combines multiple streams into a single stream. Elements arrive in non-deterministic order as they become available. Uses the context and options from the first stream.
merged := goflow.FanIn(streams)func FanIn[T any](streams []Stream[T]) Stream[T]FanMap
Fans out a stream into n partitions, maps each concurrently, and fans in the results. Output order is non-deterministic. This is a shorthand for FanIn(MapEach(s.FanOut(n), fn)).
results := goflow.FanMap(s, 4, func(ctx context.Context, url string) (Response, error) {
return httpGet(ctx, url)
})func FanMap[T, U any](s Stream[T], n int, fn func(context.Context, T) (U, error)) Stream[U]FanMapFilter
Fans out, applies MapFilter concurrently, and fans in the results. Equivalent to FanIn(MapFilterEach(s.FanOut(n), fn)).
results := goflow.FanMapFilter(s, 4, func(ctx context.Context, id string) (User, bool, error) {
u, err := fetchUser(ctx, id)
if err != nil {
return User{}, false, nil // skip failures
}
return u, true, nil
})func FanMapFilter[T, U any](s Stream[T], n int, fn func(context.Context, T) (U, bool, error)) Stream[U]Tee
Broadcasts every element to n output streams. Unlike FanOut which round-robins, Tee sends each element to all streams.
copies := s.Tee(2)
// copies[0] and copies[1] both receive every elementfunc (s Stream[T]) Tee(n int) []Stream[T]WARNING
Tee blocks on the slowest consumer. If any output stream is not consumed, the pipeline stalls. Consider adding buffered channels via Pipe if consumers have different speeds.
MapEach
Applies Map to each stream in a slice, returning a slice of transformed streams.
transformed := goflow.MapEach(streams, fn)func MapEach[T, U any](streams []Stream[T], fn func(context.Context, T) (U, error)) []Stream[U]MapFilterEach
Applies MapFilter to each stream in a slice.
filtered := goflow.MapFilterEach(streams, fn)func MapFilterEach[T, U any](streams []Stream[T], fn func(context.Context, T) (U, bool, error)) []Stream[U]Combinators
Concat
Returns a stream that emits all elements from each input stream in order: first all elements from streams[0], then streams[1], and so on. Uses the context and options from the first stream.
all := goflow.Concat(s1, s2, s3)func Concat[T any](streams ...Stream[T]) Stream[T]Utility Methods
Context
func (s Stream[T]) Context() context.ContextReturns the stream's bound context.
WithOptions
func (s Stream[T]) WithOptions(opts ...gofuncy.GoOption) Stream[T]Returns a shallow copy of the stream with the given options appended. Options are propagated to all downstream operators.
Chan
func (s Stream[T]) Chan() <-chan TReturns the underlying receive channel.
Iter
func (s Stream[T]) Iter() iter.Seq[T]Returns an iter.Seq[T] that yields each element of the stream. The returned iterator drains the stream and can only be used once.
Iter2
func (s Stream[T]) Iter2() iter.Seq2[int, T]Returns an iter.Seq2[int, T] that yields each element with its zero-based index. The returned iterator drains the stream and can only be used once.
