Skip to content

Operators ​

This page lists every goflow operator grouped by category. Each entry shows the Go signature, a description, and a short usage example. Most entries have three tabs: Diagram, Code and Signature.

The diagrams are animations. Blue elements go into the operator box. Green elements come out. Purple elements move between two operators. A red element is an error.

Constructors ​

Of ​

Creates a stream from variadic values. The items are buffered into a channel of matching capacity.

Of turns the values a, b, c into a stream [a, b, c]
go
s := goflow.Of(ctx, "a", "b", "c")
go
func Of[T any](ctx context.Context, items ...T) Stream[T]

From ​

Wraps an existing receive channel as a stream with the given context.

From turns channel values 1, 2, 3 into a stream [1, 2, 3]
go
ch := make(chan int, 10)
s := goflow.From(ctx, ch)
go
func From[T any](ctx context.Context, source <-chan T) Stream[T]

FromIter ​

Creates a stream from a Go 1.23 iter.Seq[T] by pushing elements into a channel.

FromIter turns an iter.Seq that yields 1, 2, 3 into a stream [1, 2, 3]
go
s := goflow.FromIter(ctx, slices.Values([]int{1, 2, 3}))
go
func FromIter[T any](ctx context.Context, seq iter.Seq[T]) Stream[T]

FromFunc ​

Creates a stream from a blocking function that sends items via the provided send callback. The function should block until it is done producing items and must respect context cancellation. bufSize controls backpressure.

FromFunc turns each send call of fn into a stream element
go
s := goflow.FromFunc(ctx, 16, func(ctx context.Context, send func(int) error) error {
    for i := 0; i < 100; i++ {
        if err := send(i); err != nil {
            return err
        }
    }
    return nil
})
go
func FromFunc[T any](ctx context.Context, bufSize int, fn func(ctx context.Context, send func(T) error) error, opts ...gofuncy.GoOption) Stream[T]

Generate ​

Returns an infinite stream where each element is produced by fn. The stream runs until the context is cancelled.

Generate calls fn for each element: [42, 7, 93, ...]
go
s := goflow.Generate(ctx, func() int { return rand.Intn(100) })
go
func Generate[T any](ctx context.Context, fn func() T) Stream[T]

Iterate ​

Returns an infinite stream: seed, fn(seed), fn(fn(seed)), and so on. The stream runs until the context is cancelled.

Iterate starts at seed 1 and applies n * 2: [1, 2, 4, 8, ...]
go
s := goflow.Iterate(ctx, 1, func(n int) int { return n * 2 })
// 1, 2, 4, 8, 16, ...
go
func Iterate[T any](ctx context.Context, seed T, fn func(T) T) Stream[T]

OfMap ​

Returns a stream of KeyValue[K, V] pairs from the given map. Iteration order is non-deterministic, matching Go map semantics.

OfMap turns map entries into KeyValue elements
go
s := goflow.OfMap(ctx, map[string]int{"x": 1, "y": 2})
go
func OfMap[K comparable, V any](ctx context.Context, m map[K]V) Stream[KeyValue[K, V]]

Empty ​

Returns an immediately closed, empty stream.

Empty returns a closed stream with no elements
go
s := goflow.Empty[int]()
go
func Empty[T any]() Stream[T]

Pipe ​

Creates a writable stream entry point. Returns a *Sender[T] and the readable stream. Send blocks and returns ErrClosed or the send context's error; Try never blocks and returns ErrFull or ErrClosed. The entry point closes when ctx is done or Close is called. The stream and its stages run on ctx and stop when it ends. To drain a long-lived pipeline, use PipeFunc.

Options: WithBuffer, WithName, WithMeterProvider, WithTracerProvider.

Send hello and world into a Pipe; the stream gets [hello, world]
go
send, s := goflow.Pipe[string](ctx, goflow.WithBuffer(16))
go func() {
    send.Send(ctx, "hello")
    send.Send(ctx, "world")
    send.Close(ctx)
}()
go
func Pipe[T any](ctx context.Context, opts ...Option) (*Sender[T], Stream[T])

PipeFunc ​

Creates a Pipe and launches the consumer fn in a gofuncy.Go goroutine. Returns only the Sender. fn and its stages run on a context that ctx does not cancel; fn must return when its stream ends. Close(ctx) waits until fn returns.

Send 42 and 43 into PipeFunc; fn handles each element
go
send := goflow.PipeFunc[int](ctx, func(ctx context.Context, s goflow.Stream[int]) error {
    return s.ForEach(func(_ context.Context, n int) error {
        fmt.Println(n)
        return nil
    })
})
send.Send(ctx, 42)
go
func PipeFunc[T any](ctx context.Context, fn func(context.Context, Stream[T]) error, opts ...Option) *Sender[T]

Transformers ​

Map ​

Returns a new stream by applying fn to each element. If fn returns an error, the stream closes and the error is handled by gofuncy.Go.

Map(n * 2) turns [1, 2, 3] into [2, 4, 6]
go
doubled := goflow.Map(s, func(_ context.Context, n int) (int, error) {
    return n * 2, nil
})
go
func Map[T, U any](s Stream[T], fn func(context.Context, T) (U, error)) Stream[U]

FlatMap ​

Applies fn to each element, producing a sub-stream per element, and flattens the results into a single output stream sequentially.

FlatMap turns [1, 2] into [1, 10, 100, 2, 20, 200]
go
expanded := goflow.FlatMap(s, func(ctx context.Context, n int) goflow.Stream[int] {
    return goflow.Of(ctx, n, n*10, n*100)
})
go
func FlatMap[T, U any](s Stream[T], fn func(context.Context, T) Stream[U]) Stream[U]

MapFilter ​

Maps and filters in a single pass. (val, true, nil) emits val; (_, false, nil) skips the item; (_, _, err) stops the stream.

MapFilter(Atoi) turns ["1", "x", "3"] into [1, 3]
go
parsed := goflow.MapFilter(s, func(_ context.Context, raw string) (int, bool, error) {
    n, err := strconv.Atoi(raw)
    if err != nil {
        return 0, false, nil // skip unparseable
    }
    return n, true, nil
})
go
func MapFilter[T, U any](s Stream[T], fn func(context.Context, T) (U, bool, error)) Stream[U]

Flatten ​

Flattens a stream of slices into a stream of individual elements.

Flatten turns [[1 2], [3 4]] into [1, 2, 3, 4]
go
flat := goflow.Flatten(goflow.Of(ctx, []int{1, 2}, []int{3, 4}))
// 1, 2, 3, 4
go
func Flatten[T any](s Stream[[]T]) Stream[T]

Reverse ​

Collects all elements and emits them in reverse order.

Reverse turns [1, 2, 3] into [3, 2, 1]
go
reversed := goflow.Of(ctx, 1, 2, 3).Reverse()
// 3, 2, 1
go
func (s Stream[T]) Reverse() Stream[T]

WARNING

Reverse buffers the entire stream in memory before emitting. Do not use on unbounded streams.

Sort ​

Collects all elements, sorts them using cmp, and emits in sorted order.

Sort turns [3, 1, 2] into [1, 2, 3]
go
sorted := goflow.Of(ctx, 3, 1, 2).Sort(func(a, b int) int { return a - b })
// 1, 2, 3
go
func (s Stream[T]) Sort(cmp func(T, T) int) Stream[T]

WARNING

Sort buffers the entire stream in memory before emitting. Do not use on unbounded streams.

Split ​

Groups consecutive elements into batches of size n. The last batch may contain fewer than n elements.

Split(2) turns [1, 2, 3, 4, 5] into [[1 2], [3 4], [5]]
go
batches := goflow.Split(goflow.Of(ctx, 1, 2, 3, 4, 5), 2)
// [1,2], [3,4], [5]
go
func Split[T any](s Stream[T], n int) Stream[[]T]

Window ​

Emits sliding windows of n consecutive elements. If the source has fewer than n elements, no windows are emitted.

Window(3) turns [1, 2, 3, 4] into [[1 2 3], [2 3 4]]
go
windows := goflow.Window(goflow.Of(ctx, 1, 2, 3, 4), 3)
// [1,2,3], [2,3,4]
go
func Window[T any](s Stream[T], n int) Stream[[]T]

Batch ​

Groups consecutive elements into batches. A batch is flushed when it reaches MaxItems(n), when the next element would push it over MaxBytes(n, size), when MaxWait(d) has passed since its first element, or when the input closes (the partial batch is always flushed). Without options, Batch behaves like MaxItems(1). An element larger than MaxBytes forms a batch of its own. The flush reason (count, bytes, interval, close) is recorded in metrics and spans.

Unlike Split (count only) and Window (sliding), Batch also flushes by size and time, which suits long-lived streams.

Batch(MaxItems(3)) turns [a, b, c, d, e] into [[a b c], [d e]]
go
batches := goflow.Batch(events,
    goflow.MaxItems(200),
    goflow.MaxBytes(400<<10, func(e Event) int { return len(e.Payload) }),
    goflow.MaxWait(5*time.Second),
)
go
func Batch[T any](s Stream[T], opts ...Option) Stream[[]T]

Filters ​

Filter ​

Returns a stream containing only elements where fn returns true.

Filter(even) turns [1, 2, 3, 4] into [2, 4]
go
evens := goflow.Of(ctx, 1, 2, 3, 4).Filter(func(_ context.Context, n int) bool {
    return n%2 == 0
})
go
func (s Stream[T]) Filter(fn func(context.Context, T) bool) Stream[T]

Distinct ​

Deduplicates elements using a key function. First occurrence wins.

Distinct turns [a, b, a, c, b] into [a, b, c]
go
unique := s.Distinct(func(item Item) string { return item.ID })
go
func (s Stream[T]) Distinct(key func(T) string) Stream[T]

WARNING

Distinct maintains a map of all seen keys in memory. For streams with high cardinality, memory usage grows without bound.

Take ​

Emits the first n elements then closes the stream.

Take(3) turns [1, 2, 3, 4, 5] into [1, 2, 3]
go
first3 := goflow.Of(ctx, 1, 2, 3, 4, 5).Take(3)
// 1, 2, 3
go
func (s Stream[T]) Take(n int) Stream[T]

Skip ​

Drops the first n elements and emits the rest.

Skip(2) turns [1, 2, 3, 4, 5] into [3, 4, 5]
go
rest := goflow.Of(ctx, 1, 2, 3, 4, 5).Skip(2)
// 3, 4, 5
go
func (s Stream[T]) Skip(n int) Stream[T]

Peek ​

Calls fn for each element as a side-effect and forwards the element unchanged. Useful for logging or debugging.

Peek calls fn for each of [1, 2, 3] and passes them on unchanged
go
s.Peek(func(_ context.Context, n int) {
    log.Printf("processing: %d", n)
})
go
func (s Stream[T]) Peek(fn func(context.Context, T)) Stream[T]

Throttle ​

Rate-limits the stream to at most one element per duration d.

Throttle(100ms) passes [1, 2, 3] on, one element every 100ms
go
limited := s.Throttle(100 * time.Millisecond)
go
func (s Stream[T]) Throttle(d time.Duration) Stream[T]

Consumers ​

Collect ​

Drains the stream and returns all elements as a slice.

Collect turns [1, 2, 3] into the slice []int{1, 2, 3}
go
items := goflow.Of(ctx, 1, 2, 3).Collect()
go
func (s Stream[T]) Collect() []T

Count ​

Returns the number of elements in the stream.

Count of [1, 2, 3] is 3
go
n := goflow.Of(ctx, 1, 2, 3).Count()
// 3
go
func (s Stream[T]) Count() int

ForEach ​

Consumes the stream, calling fn for each element. Returns the first error from fn or the context, nil when fully consumed. With WithErrorHandler(h), errors from fn go to h and consumption continues; ForEach then only returns context errors or nil.

ForEach calls fn(a), fn(b), fn(c)
go
err := s.ForEach(func(_ context.Context, item Item) error {
    return db.Save(item)
})
go
func (s Stream[T]) ForEach(fn func(context.Context, T) error, opts ...Option) error

Reduce ​

Folds all elements into a single value using fn. Returns the accumulated result or the first error from fn.

Reduce(0, acc + n) of [1, 2, 3] is 6
go
sum, err := goflow.Reduce(goflow.Of(ctx, 1, 2, 3), 0,
    func(_ context.Context, acc, n int) (int, error) {
        return acc + n, nil
    },
)
go
func Reduce[T, U any](s Stream[T], initial U, fn func(context.Context, U, T) (U, error)) (U, error)

ToMap ​

Collects all stream elements into a map using the key and value functions. If duplicate keys occur, the last value wins.

ToMap turns [{1 A}, {2 B}, {1 C}] into the map {1: C, 2: B}
go
m := goflow.ToMap(s, func(u User) string { return u.ID }, func(u User) User { return u })
go
func ToMap[T any, K comparable, V any](s Stream[T], key func(T) K, value func(T) V) map[K]V

FindFirst ​

Returns the first element and true, or the zero value and false if the stream is empty.

FindFirst of [7, 8, 9] is 7, true
go
first, ok := s.FindFirst()
go
func (s Stream[T]) FindFirst() (T, bool)

FindFirstMatch ​

Returns the first element matching the predicate and true, or the zero value and false if no element matches.

FindFirstMatch(admin) of [bob, ann, eve] is ann, true
go
admin, ok := s.FindFirstMatch(func(_ context.Context, u User) bool {
    return u.Role == "admin"
})
go
func (s Stream[T]) FindFirstMatch(fn func(context.Context, T) bool) (T, bool)

AllMatch ​

Returns true if all elements match the predicate. Short-circuits on the first non-matching element. Returns true for an empty stream.

AllMatch(n > 0) of [3, 5, -1, 8] is false
go
allPositive := s.AllMatch(func(_ context.Context, n int) bool { return n > 0 })
go
func (s Stream[T]) AllMatch(fn func(context.Context, T) bool) bool

AnyMatch ​

Returns true if any element matches the predicate. Short-circuits on the first matching element. Returns false for an empty stream.

AnyMatch(n < 0) of [3, -2, 5] is true
go
hasNegative := s.AnyMatch(func(_ context.Context, n int) bool { return n < 0 })
go
func (s Stream[T]) AnyMatch(fn func(context.Context, T) bool) bool

NoneMatch ​

Returns true if no elements match the predicate. Short-circuits on the first matching element. Returns true for an empty stream.

NoneMatch(is err) of [ok, err, ok] is false
go
noErrors := s.NoneMatch(func(_ context.Context, r Result) bool { return r.Err != nil })
go
func (s Stream[T]) NoneMatch(fn func(context.Context, T) bool) bool

Min ​

Returns the minimum element according to cmp and true, or the zero value and false if the stream is empty.

Min of [3, 1, 2] is 1, true
go
smallest, ok := s.Min(func(a, b int) int { return a - b })
go
func (s Stream[T]) Min(cmp func(T, T) int) (T, bool)

Max ​

Returns the maximum element according to cmp and true, or the zero value and false if the stream is empty.

Max of [3, 1, 2] is 3, true
go
largest, ok := s.Max(func(a, b int) int { return a - b })
go
func (s Stream[T]) Max(cmp func(T, T) int) (T, bool)

Process ​

Consumes the stream with n long-lived workers. All errors are collected and returned via errors.Join. With WithErrorHandler(h), errors go to h instead and are not retained. Use WithGroupOptions(...) to pass gofuncy.GroupOption values to the worker group.

Process runs fn on 4 workers at the same time
go
err := s.Process(4, func(ctx context.Context, item Item) error {
    return upload(ctx, item)
})
go
func (s Stream[T]) Process(n int, fn func(context.Context, T) error, opts ...Option) error

Concurrency ​

FanOut ​

Distributes elements round-robin across n output streams.

FanOut(3) sends [1..6] round-robin to 3 output streams
go
streams := s.FanOut(3)
go
func (s Stream[T]) FanOut(n int) []Stream[T]

WARNING

FanOut blocks on the slowest consumer. If one output stream is not being consumed, the entire pipeline stalls.

FanIn ​

Combines multiple streams into a single stream. Elements arrive in non-deterministic order as they become available. Uses the context and options from the first stream.

FanIn merges 3 streams into one stream
go
merged := goflow.FanIn(streams)
go
func FanIn[T any](streams []Stream[T]) Stream[T]

FanMap ​

Fans out a stream into n partitions, maps each concurrently, and fans in the results. Output order is non-deterministic. This is a shorthand for FanIn(MapEach(s.FanOut(n), fn)).

FanMap runs fn on 3 workers and merges the results
go
results := goflow.FanMap(s, 4, func(ctx context.Context, url string) (Response, error) {
    return httpGet(ctx, url)
})
go
func FanMap[T, U any](s Stream[T], n int, fn func(context.Context, T) (U, error)) Stream[U]

FanMapFilter ​

Fans out, applies MapFilter concurrently, and fans in the results. Equivalent to FanIn(MapFilterEach(s.FanOut(n), fn)).

FanMapFilter runs fn on 3 workers, skips b and merges the results
go
results := goflow.FanMapFilter(s, 4, func(ctx context.Context, id string) (User, bool, error) {
    u, err := fetchUser(ctx, id)
    if err != nil {
        return User{}, false, nil // skip failures
    }
    return u, true, nil
})
go
func FanMapFilter[T, U any](s Stream[T], n int, fn func(context.Context, T) (U, bool, error)) Stream[U]

Tee ​

Broadcasts every element to n output streams. Unlike FanOut which round-robins, Tee sends each element to all streams.

Tee(2) sends every element of [1, 2, 3] to 2 output streams
go
copies := s.Tee(2)
// copies[0] and copies[1] both receive every element
go
func (s Stream[T]) Tee(n int) []Stream[T]

WARNING

Tee blocks on the slowest consumer. If any output stream is not consumed, the pipeline stalls. Consider adding buffered channels via Pipe if consumers have different speeds.

MapEach ​

Applies Map to each stream in a slice, returning a slice of transformed streams.

MapEach applies fn to each stream in a slice
go
transformed := goflow.MapEach(streams, fn)
go
func MapEach[T, U any](streams []Stream[T], fn func(context.Context, T) (U, error)) []Stream[U]

MapFilterEach ​

Applies MapFilter to each stream in a slice.

MapFilterEach applies fn to each stream in a slice and skips 2
go
filtered := goflow.MapFilterEach(streams, fn)
go
func MapFilterEach[T, U any](streams []Stream[T], fn func(context.Context, T) (U, bool, error)) []Stream[U]

Combinators ​

Concat ​

Returns a stream that emits all elements from each input stream in order: first all elements from streams[0], then streams[1], and so on. Uses the context and options from the first stream.

Concat joins s1, s2, s3 in order: [1, 2, 3, 4, 5]
go
all := goflow.Concat(s1, s2, s3)
go
func Concat[T any](streams ...Stream[T]) Stream[T]

Utility Methods ​

Context ​

go
func (s Stream[T]) Context() context.Context

Returns the stream's bound context.

WithOptions ​

go
func (s Stream[T]) WithOptions(opts ...gofuncy.GoOption) Stream[T]

Returns a shallow copy of the stream with the given options appended. Options are propagated to all downstream operators.

Chan ​

Chan returns the channel under the stream
go
func (s Stream[T]) Chan() <-chan T

Returns the underlying receive channel.

Iter ​

Iter yields 1, 2, 3 to a range loop
go
func (s Stream[T]) Iter() iter.Seq[T]

Returns an iter.Seq[T] that yields each element of the stream. The returned iterator drains the stream and can only be used once.

Iter2 ​

Iter2 yields 0 a, 1 b, 2 c to a range loop
go
func (s Stream[T]) Iter2() iter.Seq2[int, T]

Returns an iter.Seq2[int, T] that yields each element with its zero-based index. The returned iterator drains the stream and can only be used once.