Rust Iterators and Closures: 10 Patterns That Replace Loops

You've written the loop a hundred times. For i in 0..len, index in, bounds-check every access, push into a fresh vector, then loop again to filter it, then loop a third time to sum it. It works, and that's exactly why it's dangerous — nobody questions code that works.

But that triple loop walks 30 million elements to process 10 million, and it keeps two throwaway buffers alive for no reason. On a laptop you won't feel it. On a production ingest worker handling 40 GB shards, you'll feel it as a p99 that tripled overnight and an OOM killer that picked your process at 3 AM.

Rust's answer isn't a faster loop. It's refusing to write the loop at all. Iterators turn nested, index-driven code into a single declarative chain that the compiler fuses into one pass — often with the bounds checks removed entirely.

There's a catch, and it's why juniors bounce off this topic. Laziness means your map closure might never run. Collect needs type annotations that look like line noise. And closures capture variables in three different ways that produce errors reading like riddles.

By the time you're done here, you'll read any chain like a sentence, you'll know which Fn trait a closure satisfies before the compiler tells you, and you'll have the production stories — including a real outage caused by collecting a stream twice — that make these rules stick.

The Iterator Trait: next, Item, and the Only Two Things That Matter

Strip away every adapter and the Iterator trait is almost insultingly small. One associated type named Item. One required method named next that returns Option<Item>. Return Some(value) while values remain, return None when you're done, and every consumer in the standard library — for loops, collect, sum, count — works with your type for free. That contract is the entire protocol, and seniors read unfamiliar iterator code by asking only these two questions: what is Item, and what does next do at the boundary.

The for loop is sugar over this protocol, and seeing the desugar once ends half of all iterator confusion. For x in iter translates to a match on iter.into_iter() that calls next in a loop and breaks on None. That single fact explains why for takes ownership of the expression, why you can't reuse the same by-value iterator afterward, and why .iter() versus .into_iter() changes whether the collection survives the loop. If a junior asks why their vector is gone after a loop, you point at into_iter and the lesson lands in seconds.

IntoIterator is the quiet partner trait that makes all of this ergonomic. Anything implementing it can appear on the right side of for and can be passed to functions taking impl IntoIterator. Slices give you .iter() for shared borrows and .iter_mut() for exclusive borrows, Vec gives you .into_iter() for ownership, and ranges like 0..n implement it directly. Writing function signatures that accept impl IntoIterator<Item = T> instead of &[T] costs nothing and accepts arrays, vectors, ranges, and map closures alike.

The final piece is the fuse contract. After next returns None once, well-behaved iterators keep returning None forever. Most adapters rely on this silently, and a hand-rolled iterator that alternates between None and Some will corrupt take, position, and for_each in ways that look like data loss. When you implement the trait yourself — which you'll do in the custom-iterator section — either guarantee fused behavior by construction or wrap the result in .fuse() at the boundary. Two methods, one option type, one fuse rule: that's the whole foundation everything below builds on.

Ownership flavors of iteration deserve a second look because they decide whether the source collection survives. Iterating &v auto-refs into slice::iter and borrows; iterating &mut v yields exclusive references for in-place mutation; iterating v by value moves elements out and ends the collection. Method resolution picks these automatically in for loops, which is convenient until a function body uses v after the loop and meets E0382. The thirty-second diagnosis is always the same question: did this loop borrow or consume? Answer it by reading the expression after for...in, and the fix — adding & or switching to .iter() — writes itself.

One more structural fact pays rent forever: adapters return anonymous types you cannot name. The type of v.iter().map(f) is Map<Iter<i32>, ClosureType>, spelled out nowhere in your source and unspellable by hand. Functions returning chains therefore declare impl Iterator<Item = T>, letting the compiler hide the concrete stack of wrappers. When a chain must cross a dynamic boundary — trait objects, heterogeneous branches — Box<dyn Iterator<Item = T>> erases the type at the cost of one vtable call per next. Seniors keep impl Iterator at every internal boundary and pay for Box only where dynamism is real, because a heap-allocated iterator in a hot loop is a performance regression wearing an abstraction costume.

Closure-built iterators cover the cases too small for a struct. std::iter::from_fn wraps a closure returning Option<T> as a full iterator — counters, generators, and stateful walks in five lines with inferred types. Repeat_with produces infinite streams from a repeated closure for load shapes and fuzz inputs, always paired with take for termination. Successors chains each item from the previous — Collatz sequences, linked-list walks, retry-state machines — expressing recurrence directly instead of manual loop state. These constructors inherit every adapter instantly, which makes them the fastest path from an idea to a tested pipeline when a named type would be ceremony.

fn main() {
    // for-loop desugar: this is what `for x in v.iter()` compiles to.
    let v = vec![10, 20, 30];
    let mut iter = v.iter().into_iter();
    loop {
        match iter.next() {
            Some(x) => println!("saw {x}"),
            None => break,
        }
    }
    // The vector is untouched: `.iter()` only borrowed it.
    assert_eq!(v.len(), 3);
    // Generic over anything convertible into an iterator.
    print_all(&v);
    print_all(0..3);
}
fn print_all<I>(items: I)
where
    I: IntoIterator<Item = i32>,
{
    for x in items {
        println!("item {x}");
    }
}

Lazy Adapters vs Consuming Consumers: Why Your map Never Ran

The most reported iterator bug in every team you've joined has the same shape: a chain ending in map or filter with a semicolon, doing absolutely nothing. Adapters are lazy constructors. Calling .map(f) allocates a small struct holding your iterator and your closure — it processes zero elements. Work starts only when a consumer pulls values through next, which means a chain without a consumer is dead code that compiles cleanly. The compiler even warns you with unused_must_use if you let it, and enabling that lint crate-wide catches this class of bug at write time.

Adapters split into two families and you should be able to classify any method in one glance. Transforming adapters — map, filter, filter_map, flat_map, take, skip, enumerate, zip, chain — take an iterator and return a new iterator, so they compose indefinitely and stay lazy. Consuming adapters, usually just called consumers — collect, sum, count, fold, for_each, partition, find, any, last — take an iterator by value and return a concrete result, ending the chain. The signature tells you which family you're holding: anything returning impl Iterator or Self stays lazy, anything returning Vec, bool, or u64 ends it.

This split drives real design decisions. Chaining five adapters costs one allocation at most and one pass at runtime, because each next call cascades down the stack of wrappers. Inserting a single collect in the middle breaks that fusion: you pay a full materialization, a second walk, and allocator pressure that shows up as a latency cliff on large inputs. The senior default is zero intermediate collects, with exceptions granted only for reuse (iterate the same data three times), parallelism boundaries (hand a Vec to scoped threads), or measured cache wins on tiny hot inputs.

There's a subtle middle ground worth knowing: by_ref. Calling it.by_ref().take(10).collect() borrows the iterator instead of consuming it, letting you sip the head off a stream and keep pulling the tail afterward. Log processors use this to peek at headers before dispatching the body. It looks like a trick the first time you see it, and after the third production use it becomes muscle memory for any incremental parsing work.

Size hints ride along invisibly and decide how many times your collects reallocate. Every iterator reports size_hint as a lower bound plus an optional upper bound; map and enumerate forward it exactly, filter degrades the upper bound to None because it can't predict rejections, and collect uses the lower bound to pre-reserve. That is why mapping a 5M-element slice collects with one allocation while filtering it reallocates logarithmically — the information was lost at the filter step. ExactSizeIterator restores precision where length is genuinely known, and len() on those iterators is O(1) truth rather than a guess. When profiling shows realloc spikes inside collect, the cause is usually a degraded hint two adapters upstream, not the collect itself.

Debugging lazy chains needs its own tool because println inside adapters never fires without a consumer pulling. The inspect adapter exists for exactly this: .inspect(|x| eprintln!("stage2={x:?}")) passes values through untouched while logging each one, composing with the chain instead of breaking it. Pair inspect with take(5) during development to preview head behavior on massive inputs without running the full stream. Between inspect for values, size_hint for allocations, and by_ref for partial consumption, you hold the complete diagnostic kit for any chain that misbehaves — and none of it costs a line of restructuring.

Short-circuiting consumers deserve explicit recognition because they change complexity, not just style. Any, all, find, and position stop pulling the moment the answer is decided — a 10M-row scan for one match costs one comparison instead of ten million. Contrast with count or last, which must walk everything; reaching for find when you need existence, rather than filter plus count, converts linear work into early exit. Reviewers should treat filter-then-count as a smell when the predicate result is boolean — any and all say what the code means and stop when it means it.

fn main() {
    // Lazy: this chain does ZERO work until `sum` pulls values through.
    let total: u64 = (1..=1_000_000u64)
        .filter(|x| x % 2 == 0)
        .map(|x| x * x)
        .sum();
    println!("total = {total}");
    // by_ref: sip 3 items off the front, keep the same iterator alive.
    let data = vec![1, 2, 3, 4, 5];
    let mut it = data.iter();
    let head: Vec<i32> = it.by_ref().take(3).cloned().collect();
    let tail: Vec<i32> = it.cloned().collect();
    assert_eq!(head, vec![1, 2, 3]);
    assert_eq!(tail, vec![4, 5]);
    println!("head={head:?} tail={tail:?}");
}

collect and the Turbofish: Turning Streams Into Anything

Collect is the most used consumer in Rust and the most misunderstood, because its signature hides the magic. Collect takes self and returns B where B: FromIterator<Self::Item> — meaning the target type is inferred from context, not from the call. When context is missing, the compiler errors with E0277 and asks for an annotation, which is where turbofish enters: .collect::<Vec<_>>() pins the container while leaving the element type to inference. Seniors reach for turbofish reflexively at collection sites because it documents intent at the exact line a reader needs it.

FromIterator is the trait doing the real work, and its implementors read like a greatest-hits list: Vec, VecDeque, HashMap, HashSet, String, and — the one juniors miss — Result and Option. Collecting an iterator of Result<T, E> into Result<Vec<T>, E> short-circuits on the first error, which turns ten lines of error-plumbing into a single collect call. The same trick works for Option. Once you see Result as a collection target, whole classes of manual loops collapse into one-liners that propagate errors correctly by construction.

String collection deserves its own callout because it surprises people twice. Collecting Iterator<Item = char> into String just works, and collecting Iterator<Item = &str> joins without a separator — which is rarely what you want for human output. For delimited output, use slice::join or itertools-style manual folds; collect is for lossless reconstruction. Knowing which join you need before writing the chain saves the classic write-collect-rewrite cycle.

Performance-wise, collect into Vec pre-reserves capacity when the iterator reports a trustworthy size_hint, so mapped slices collect with a single allocation. Filtered iterators report a vacuous upper bound, so collecting them may reallocate logarithmically — harmless at small sizes, measurable past a million elements. When you know the output size, Vec::with_capacity plus extend beats collect by skipping the hint dance entirely. That's a micro-optimization you apply after profiling, not before, but you should know it exists before the profiler tells you.

String assembly has three doors and picking the wrong one is a rite of passage. Collecting an iterator of char into String reconstructs text losslessly — the canonical chars().filter().collect() pipeline. Collecting an iterator of &str concatenates with no separator, which is correct for rebuilding tokens and wrong for human-readable lists. Delimited output belongs to slice::join(", ") or to a fold seeding a String with push_str, not to collect at all. The failure mode is always discovered in review: a collect producing "alicebobcarol" where "alice, bob, carol" was wanted, fixed by swapping one method call.

HashMap collection carries its own quiet rule: duplicate keys keep the last value with no warning. Grouping rows by key therefore can't be a bare collect — it needs entry-API folding, counts via map-of-counts, or a multimap built with fold pushing into per-key Vecs. Seniors spot this instantly in review because the shape is distinctive: .collect::<HashMap<_, _>>() over data where keys repeat means silent overwrites, and the fix is always a fold the author considered too verbose. Verbose and correct beats terse and lossy every time the data has duplicates, which in production means nearly always.

Lesser-known FromIterator targets solve specific shapes elegantly. BTreeMap collection yields sorted output for deterministic snapshots and golden-file tests where HashMap's random order would flake. VecDeque targets queue-shaped pipelines with O(1) pops from both ends. Collecting into Box<[T]> freezes a snapshot that can't grow, documenting intent at the type level. And Option as a target — collecting Iterator<Item = Option<T>> into Option<Vec<T>> — bails to None on the first missing value, the total-success counterpart to Result's short-circuit. Knowing the full target list turns collect from a Vec-builder into a universal constructor. Even effect-only chains collect: Iterator<Item = ()> gathers into () for pipelines run purely for side effects.

use std::collections::HashMap;
fn parse_pair(s: &str) -> Option<(String, u64)> {
    let (k, v) = s.split_once('=')?;
    Some((k.to_string(), v.parse().ok()?))
}
fn main() -> Result<(), String> {
    let rows = vec!["a=1", "b=2", "oops", "c=3"];
    // Turbofish pins the target; `_` leaves the element type inferred.
    let map: HashMap<String, u64> =
        rows.iter().filter_map(|r| parse_pair(r)).collect();
    assert_eq!(map["b"], 2);
    // Collect into Result: first error short-circuits the whole build.
    let nums = vec!["10", "20", "xx", "30"];
    let parsed: Result<Vec<u64>, _> =
        nums.iter().map(|s| s.parse::<u64>()).collect();
    assert!(parsed.is_err());
    println!("map has {} keys; bad input short-circuits", map.len());
    Ok(())
}

Fn, FnMut, FnOnce: Reading Capture Modes Like a Senior

Every closure implements exactly one of three traits, and the compiler picks the least powerful one the closure's body allows. If the body only reads captured variables through shared borrows, you get Fn. If it mutates a capture or calls an FnMut method on one, you get FnMut. If it moves a capture out — by value return, by passing ownership onward, or by the move keyword forcing ownership — you get FnOnce, callable exactly one time. The hierarchy is Fn implies FnMut implies FnOnce, so an Fn closure satisfies any bound while an FnOnce closure satisfies only FnOnce.

Adapter signatures advertise their bound openly and you should read them as contracts. Map takes FnMut because it calls your closure once per element. Filter takes FnMut for the same reason. Sort_by takes FnMut for comparators, while thread::spawn and tokio::spawn demand FnOnce plus Send and 'static because the closure crosses into another thread exactly once and may outlive the spawner. When a junior pastes a closure into spawn and gets a borrow error, the fix is almost always the move keyword — and understanding why takes ten seconds once you read bounds as contracts.

Move deserves careful treatment because it changes capture mode, not just location. A move closure takes ownership of every capture it mentions, which downgrades the closure toward FnOnce whenever a captured value is consumed. But move alone doesn't force FnOnce: a move closure that only reads its owned captures through &self still implements Fn, which is why move closures work fine with map. The rule of thumb is mechanical — add move when the closure escapes the current frame (threads, async tasks, returned impl Fn), skip it for local chains where borrowing keeps things flexible.

The classic footgun is capturing all of self when you wanted one field. Writing .map(|x| x + self.factor) inside a method borrows self for the chain's lifetime, which collides with any later &mut self use and produces errors far from the cause. Copy the field into a local first — let factor = self.factor; .map(move |x| x + factor) — and the borrow vanishes. Clippy's needless_borrow lints help, but the habit of narrowing captures before writing the closure prevents the error instead of diagnosing it.

Generic functions over closures reveal the same three traits from the consumer side. Declaring fn apply<F: Fn(u32) -> u32>(f: F) accepts functions, non-capturing closures, and Fn closures while rejecting mutating ones — the bound is a filter on caller behavior. Impl-trait sugar fn apply(f: impl Fn(u32) -> u32) says the same thing with less syntax, and &dyn Fn(u32) -> u32 erases the type for heterogeneous storage at one indirection per call. Sort_by_key demonstrates the idiom in the wild: it takes FnMut returning an orderable key, calls it per comparison, and never constrains more than it uses. Writing your own helpers with minimal bounds — FnMut where mutation is possible, Fn where it isn't — keeps APIs maximally reusable.

Copy-type captures deserve a final note because they quietly dodge the borrow checker. Capturing a u64 or bool copies the bits into the closure with no borrow at all, which is why small config values never cause lifetime trouble while String and Vec captures do. The clone-then-move pattern exploits this deliberately: let owned = expensive.clone() before a move closure gives the closure its own copy while the original stays usable. Costs a clone, buys independence — the right trade whenever the clone is cheap or the closure escapes to another thread.

FnOnce appears in everyday APIs far beyond thread spawning, and recognizing the bound predicts behavior. Unwrap_or_else and map_or_else take FnOnce because the fallback runs at most once — no mutability needed, no repetition possible. Option::and_then takes FnOnce for the same reason: one value in, one call, one result. Once you see the pattern, bounds read as usage documentation — Fn means called repeatedly and concurrently-safe to share, FnMut means sequential repeated calls, FnOnce means a single shot. That reading turns unfamiliar signatures into known quantities in seconds.

fn main() {
    // Fn: only reads its capture — satisfies every bound.
    let factor = 3;
    let v = vec![1, 2, 3];
    let scaled: Vec<i32> = v.iter().map(|x| x * factor).collect();
    // FnMut: mutates its capture — map is fine, `f` is borrowed mutably.
    let mut seen = 0;
    let doubled: Vec<i32> = v.iter().map(|x| {
        seen += 1;
        x * 2
    }).collect();
    // FnOnce: moves its capture out — callable exactly once.
    let label = String::from("result");
    let consume = || label;
    let owned: String = consume();
    // `consume` cannot be called again here: value moved.
    assert_eq!(scaled, vec![3, 6, 9]);
    assert_eq!((seen, doubled.len(), owned), (3, 3, String::from("result")));
    println!("captures ok");
}

move Closures, Threads, and Ownership Across Boundaries

Move closures are where ownership theory meets production reality, because every concurrency boundary in Rust demands them. Thread::spawn requires FnOnce() -> T + Send + 'static, and the 'static bound is the one that bites: borrowed captures can't satisfy it unless the borrowed data itself lives forever. The move keyword transfers ownership of captures into the closure, severing the lifetime tie to the spawning frame. That single keyword is the difference between code that compiles on a worker pool and a wall of lifetime errors that reads like a different language.

Scoped threads changed the calculus and seniors use them deliberately. std::thread::scope lets spawned threads borrow stack data because the scope joins every thread before returning, which guarantees borrows can't outlive their source. Inside a scope you often skip move entirely and share slices by reference across eight workers with zero copies. Outside a scope — raw spawn, tokio tasks, callbacks stored for later — move is mandatory. Knowing which spawn you're holding determines whether you copy megabytes or borrow them, and the wrong choice in a hot path shows up directly in allocator flame graphs.

Ownership also governs what happens to collections fed into chains. Into_iter on a Vec moves every element, giving owned values to map closures that need to transform and re-store them. Iter borrows, giving &T references that keep the source alive for later passes. The decision is a one-line diff with architectural consequences: into_iter ends the collection's life, iter preserves it. Review every into_iter in a function that touches the collection afterward — the compiler will catch the mistake, but catching it in review saves a CI round-trip.

The subtlest move interaction is partial moves inside closures. A closure that moves one field out of a captured struct while borrowing another will fight you, because moving a field out of a borrowed struct is illegal. Destructure first — let Cache { hot, .. } = cache; then move hot into the closure — or clone the owned piece when the cost is trivial. These restructures look like ceremony until you've debugged the alternative: a 40-line error about conflicting borrow and move semantics pointing at a closure three frames from the actual decision.

Channels extend the same ownership thinking across pipeline stages. An mpsc sender moved into a producer thread transfers values downstream with ownership, and each stage's move closure owns exactly what it needs — no shared borrows, no lifetime negotiation between stages. The pattern scales cleanly: parse threads own readers, transform threads own lookup tables (or borrow them under scope), sink threads own writers. When a stage needs shared read access instead of ownership, Arc replaces move-capture — and the handoff stays explicit at the type level rather than hidden behind a global.

Async runtimes replay the 'static requirement with higher stakes. Tokio::spawn demands Send plus 'static because tasks migrate across worker threads and outlive any stack frame, so borrowed captures are rejected outright. The standard idiom is cloning Arcs per task — let db = Arc::clone(&db) inside the spawn loop — giving each task shared ownership of connection pools, config, and caches. Engineers who fight this by cloning entire structs per task pay in memory and stale-copy bugs; engineers who reach for Arc first get cheap shared ownership with a single allocation. The rule transfers directly: borrowed data for scoped threads, Arc-shared data for spawned tasks, owned data for one-shot handoffs.

Async move blocks replay the same ownership lesson inside futures. An async move block takes ownership of its captures exactly like a move closure, and the resulting future is Send only when every capture is Send — one Rc capture poisons Send for the whole task and fails spawning with an error pointing nowhere near the capture. The Arc-clone idiom transfers directly: clone Arcs before the async move block, keep non-Send handles (like Rc caches or raw pointers) behind spawn_blocking boundaries. Thread::Builder adds the operational finishing touch for OS threads — named threads turn anonymous pool-worker panics into attributable log lines, which on-call reads as a gift.

use std::thread;
fn main() {
    // Scoped threads: borrow stack data with zero copies.
    let data = vec![1u64, 2, 3, 4, 5, 6, 7, 8];
    let mut total = 0u64;
    thread::scope(|s| {
        let mut handles = Vec::new();
        for chunk in data.chunks(2) {
            // `chunk` is borrowed; scope guarantees it outlives workers.
            handles.push(s.spawn(move || chunk.iter().sum::<u64>()));
        }
        for h in handles {
            total += h.join().unwrap();
        }
    });
    assert_eq!(total, 36);
    // Owned handoff: `move` severs the lifetime tie to this frame.
    let name = String::from("worker");
    let h = thread::spawn(move || format!("hello from {name}"));
    println!("{} total={total}", h.join().unwrap());
}

Method Chains That Replace Loops: map, filter, fold, and Friends

The mechanical translation from loop to chain follows one pattern, and drilling it until it's automatic is worth more than memorizing twenty adapters. Indexing loops become iter plus map. Push-if loops become filter plus collect. Accumulator loops become fold. Search loops become find or position. Counting loops become count or filter-count. Each translation deletes a mutable buffer, deletes an index variable, and deletes the off-by-one error hiding in the boundary condition. The result reads as a statement of intent rather than a sequence of machine steps, which is why chain code survives refactors that kill loop code.

Fold is the adapter seniors reach for when juniors write accumulator loops, and it deserves a slow explanation because its signature looks hostile. Fold takes an initial value and a closure receiving the running accumulator plus each item, returning the next accumulator: iter.fold(0, |acc, x| acc + x). Sum, product, and max are just prebuilt folds, but fold itself handles histograms, running statistics, and string building without a single mut binding. The accumulator's type can differ from the item type entirely — folding an iterator of log lines into a HashMap of counters is a one-liner that replaces fifteen lines of entry-API loop code.

Enumerate, zip, and position handle the cases where loop indices carried real information. Enumerate pairs each item with its index, killing the manual counter that always drifted. Zip walks two iterators in lockstep and stops at the shorter, which replaces paired-index loops and their mismatched-length panics. Position returns the index of the first match instead of the value, which is what hand-rolled search loops actually wanted. Partition splits one iterator into two collections by predicate in a single pass — successes and failures, valid and invalid — where loop code needed two buffers and two pushes.

The readability limit is real and seniors respect it. A chain of three to five adapters with named closures reads beautifully; a chain of nine anonymous closures is write-only code that nobody can debug at 2 AM. The remedies are mechanical: extract closures into named functions or small fns, bind intermediate results to named variables with clear types, and break chains that mix abstraction levels. Code review guidance that works is a number — more than five adapters in one expression gets split — because it removes taste from the argument entirely.

Boolean queries compress flag loops even further than transforms do. Any returns true the moment a predicate matches, short-circuiting the rest of the walk; all verifies every element with the same early exit. Hand-rolled loops for these carry a mutable found flag plus a break, which is three lines of state where one method call suffices. Find_map merges find and map for the search-and-extract shape — locating the first parseable row and returning its value — collapsing a loop-with-break plus a transform into a single pass that stops early.

Stateful scans fill the gap between pure maps and full folds. Scan threads a mutable state through the iterator while yielding one output per input, which expresses running totals, deduplication-with-memory, and carry-forward defaults that map cannot touch. Take_while and skip_while bound consumption by predicate — headers then body, preamble then payload — and map_while fuses mapping with termination for parse-until-invalid shapes. Together with fold for aggregates and position for searches, these adapters cover every loop skeleton reviewers encounter, leaving raw loops for control flow that genuinely mixes effects, early returns, and branching in ways no combinator expresses cleanly.

Extremum and reduction adapters finish the loop-replacement toolkit. Max_by_key and min_by_key replace tracking loops carrying a best-so-far variable — longest line, cheapest route, newest timestamp — in one call with no mutable state. Sum and product need their target type pinned (let total: u64 = iter.sum()) since numeric generics can't infer from nothing. Reduce handles non-Default reductions like string concatenation without a seed, returning Option for empty inputs. Each one deletes a loop-shaped bug class: stale bests, wrong seeds, and empty-input panics that hand-rolled reductions always risk.

use std::collections::HashMap;
fn main() {
    let words = vec!["apple", "avocado", "berry", "apricot", "cherry"];
    // Loop-shaped problem, chain-shaped solution: histogram by first letter.
    let hist: HashMap<char, usize> = words
        .iter()
        .filter(|w| w.len() > 4)
        .fold(HashMap::new(), |mut acc, w| {
            *acc.entry(w.chars().next().unwrap()).or_insert(0) += 1;
            acc
        });
    assert_eq!(hist[&'a'], 3);
    // Position replaces hand-rolled search loops with index tracking.
    let data = vec![4, 8, 15, 16, 23, 42];
    let idx = data.iter().position(|x| *x > 20);
    assert_eq!(idx, Some(4));
    // Zip walks two slices in lockstep; stops at the shorter one.
    let names = vec!["a", "b", "c"];
    let pairs: Vec<(usize, &str)> = (0..10).zip(names.iter().cloned()).collect();
    assert_eq!(pairs.len(), 3);
    println!("hist={hist:?} idx={idx:?}");
}

flat_map, filter_map, and partition: The Three Workhorses

Three adapters do disproportionate work in production code, and each one collapses a two-step dance into a single step. Filter_map applies a closure returning Option and drops the Nones — parsing, validation, and conditional extraction in one pass. Flat_map applies a closure returning an inner iterator and splices the results into one flat stream — nested loops over lines-then-words, users-then-orders, shards-then-records. Partition consumes the iterator once and returns two collections split by predicate — valid versus invalid, retryable versus fatal. If your chain has a filter immediately followed by a map, you almost certainly want filter_map; if it has a map producing vectors followed by flatten, you want flat_map.

Filter_map earns its place through error-shaped data. Parsing user input, environment variables, or CSV rows yields Option or Result per item, and filter_map with .ok() silently drops failures while keeping successes — perfect for best-effort telemetry, wrong for billing. The failure mode is silent data loss, so the senior habit is pairing filter_map with a counter: increment a rejected metric inside the closure or partition first and log the rejects. Teams that skip this end up explaining to finance why 3% of transactions vanished without a single error log, which is a conversation you have exactly once.

Flat_map is the nested-loop killer with one sharp edge: the inner iterators must all yield the same Item type, and the closure runs lazily per outer item, which means side effects inside interleave with consumption. Lines-to-words tokenizers, config expansion, and fan-out joins all read naturally as flat_map, and chars plus split_whitespace compose into surprisingly expressive text pipelines. Watch the closure's allocation behavior — returning a fresh Vec per item in a hot flat_map over millions of outer items creates allocation churn that a reused buffer or an itertools-style batching adapter would avoid. Profile before optimizing, but know where to look.

Partition closes the loop on two-output problems. Validating a batch yields good records and bad records, and partition collects both in one pass instead of filtering twice — which would walk the data twice and recompute the predicate twice. The return is a tuple of two collections of the same container type, so let (ok, err): (Vec<_>, Vec<_>) = iter.partition(pred) reads cleanly. Combined with Result collection, partition-then-collect gives you validated successes plus an error report in three lines where loop code needed twenty and usually got the error path wrong.

Flatten has quiet superpowers through the IntoIterator impls on Option and Result. An iterator of Options flattens with .flatten() into just the Some values — no closure required — and the same holds for iterator-of-Result streams when you want values with errors dropped (plus a counter, per the rule above). Flat_map over lines-then-words is the famous case, but flat_map over user-then-orders, shard-then-records, and directory-then-files all share the shape: one outer stream, one inner expansion per item, one flat result. Recognizing the shape is the skill; the method call is trivial once you do.

Unzip is partition's sibling for pair-shaped streams and completes the family. Where partition splits by predicate into two collections, unzip splits pairs into two collections — keys and values, timestamps and readings, ids and payloads — in one pass with no predicate at all. The two compose naturally: partition a stream of Results into oks and errs, then unzip the oks into parallel vectors for columnar processing. Three adapters, two passes total, zero manual buffers — the kind of pipeline that reads as a specification and runs as a single fused walk.

Index-tracking across nested expansions has a clean idiom worth memorizing. Flat_map discards inner positions, so chains needing global indices pair .enumerate() on the outer stream with inner lengths — or flatten first and enumerate the flat result for absolute positions. Filter_map with Result::ok compresses parse-and-keep into a fragment, but prefer an explicit match with a reject counter wherever drops need visibility. And flat_map over references (lines.iter().flat_map(|l| l.split())) borrows through the whole chain, keeping zero-copy pipelines alive where an owned intermediate would allocate per item.

fn main() {
    // filter_map: parse what parses, drop the rest, count rejects.
    let raw = vec!["10", "xx", "20", "", "30"];
    let mut rejected = 0u32;
    let nums: Vec<u64> = raw
        .iter()
        .filter_map(|s| match s.parse::<u64>() {
            Ok(n) => Some(n),
            Err(_) => {
                rejected += 1;
                None
            }
        })
        .collect();
    assert_eq!((nums.clone(), rejected), (vec![10, 20, 30], 2));
    // flat_map: one flat stream of words out of nested lines.
    let lines = vec!["hello world", "rust iterators"];
    let words: Vec<&str> =
        lines.iter().flat_map(|l| l.split_whitespace()).collect();
    assert_eq!(words.len(), 4);
    // partition: one pass, two buckets — no double walk.
    let (evens, odds): (Vec<i32>, Vec<i32>) =
        (1..=10).partition(|x| x % 2 == 0);
    assert_eq!((evens.len(), odds.len()), (5, 5));
    println!("nums={nums:?} words={words:?}");
}

Writing Your Own Iterator: Struct, next, and Size Hints

Sooner or later a domain needs an iterator the standard library doesn't ship: a retry backoff sequence, a paginated API walker, a sliding window over sensor frames, a Fibonacci stream for load-test shapes. Implementing Iterator yourself is a thirty-line exercise that pays off every time a consumer — for loops, take, collect, zip — works with your type untouched. The shape is fixed: a struct holding iteration state, an impl block with a constructor, and impl Iterator specifying Item and next. State goes in the struct, transition logic goes in next, termination is returning None.

The backoff example is the canonical first custom iterator because its state machine is honest. It holds attempt count, base delay, and max delay; each next computes base times two-to-the-attempt capped at max, increments the counter, and returns the duration. Consumers decide policy: .take(5) caps retries, .zip with request attempts pairs delays with tries, .collect materializes a schedule for logging. The iterator itself knows nothing about sleeping or HTTP — it produces values, and policy lives at the consumption site. That separation is what makes custom iterators compose instead of calcify.

Size_hint is the optional method you should rarely skip. It returns a lower bound and an optional upper bound on remaining length, and collect plus Vec::extend use it to pre-reserve capacity. A backoff iterator with a known attempt cap reports an exact hint; an unbounded Fibonacci reports (usize::MAX-ish lower, None). Wrong hints are worse than vague ones — an overstated lower bound over-allocates, an understated one just reallocates — so report exact when you know it and (0, None) when you don't. ExactSizeIterator and DoubleEndedIterator are opt-in upgrades: implement them only when len and next_back are genuinely O(1), because consumers will trust those bounds in capacity math.

Testing custom iterators is straightforward and frequently skipped, which is how off-by-one terminations reach production. Assert the first three values, assert the termination boundary with take-plus-count, assert fused behavior by calling next twice past the end, and assert the size_hint against actual remaining length at several points. Four small tests, each under ten lines, covering the exact properties downstream adapters assume. The paginated-API walker that passed these tests survived three backend schema changes without modification — the iterator absorbed the drift because its contract was pinned.

Double-ended iteration is the natural upgrade when consumption runs from both ends. Implementing DoubleEndedIterator with next_back lets consumers call rev(), rfind(), and rposition on your type — deque drains, reverse-chronological log walks, and palindrome checks all compose for free. The contract mirrors next: next_back yields from the tail, the two cursors must never cross (return None once they meet), and size_hint should shrink from both directions. A chunk iterator yielding fixed-size windows from a slice demonstrates the shape well: head and tail indices converging, exact hints throughout, rev() working on day one because the trait was implemented alongside the base.

Peekable adapters solve the lookahead problem without touching your implementation. Wrapping any iterator in .peekable() adds peek() for one-element lookahead, which powers tokenizers, CSV header detection, and run-length grouping that need to see the next item before committing. Peek_next patterns compose with by_ref for incremental parsers that sip headers, dispatch bodies, and never materialize the stream. Between custom state machines for generation, peekable for lookahead, and fuse for contract safety, hand-rolled iterators cover every streaming shape production has shown me — and each one inherits the full adapter library the moment Item and next compile.

The FusedIterator marker trait is the promise your fused types should advertise. Implementing std::iter::FusedIterator (an empty unsafe-marker-free trait) declares that None is forever, letting generic consumers skip defensive re-polling and enabling optimizations in adapters like zip and take. It costs one line — impl FusedIterator for Backoff {} — and documents the contract in the type system instead of comments. Pair it with a test calling next() twice past exhaustion asserting None both times, and the guarantee is pinned mechanically rather than trusted socially.

use std::time::Duration;
/// Capped exponential backoff: 100ms, 200ms, 400ms, ... up to `max`.
struct Backoff {
    attempt: u32,
    base_ms: u64,
    max_ms: u64,
    cap: u32,
}
impl Backoff {
    fn new(base_ms: u64, max_ms: u64, cap: u32) -> Self {
        Backoff { attempt: 0, base_ms, max_ms, cap }
    }
}
impl Iterator for Backoff {
    type Item = Duration;
    fn next(&mut self) -> Option<Duration> {
        if self.attempt >= self.cap {
            return None; // fused: stays None forever after this.
        }
        let shift = self.attempt.min(20);
        let ms = (self.base_ms.saturating_mul(1 << shift)).min(self.max_ms);
        self.attempt += 1;
        Some(Duration::from_millis(ms))
    }
    fn size_hint(&self) -> (usize, Option<usize>) {
        let left = (self.cap - self.attempt) as usize;
        (left, Some(left))
    }
}
impl ExactSizeIterator for Backoff {}
fn main() {
    let schedule: Vec<Duration> = Backoff::new(100, 800, 5).collect();
    assert_eq!(schedule.len(), 5);
    assert_eq!(schedule[4], Duration::from_millis(800));
    println!("schedule={schedule:?}");
}

Zero-Cost in Practice: Fusion, Bounds Checks, and Benchmarks

Zero-cost is a claim you verify, not a slogan you repeat. The mechanism is fusion: because adapters are generic structs monomorphized per chain, the optimizer inlines each next call into the consumer's loop and eliminates the wrapper layers entirely. A filter-plus-map-plus-sum chain over a slice becomes a single loop with a branch and an accumulate — the same assembly you'd get from hand-written code, byte for byte on current LLVM. Godbolt comparisons confirm this regularly, and the habit of checking codegen for hot chains separates engineers who hope from engineers who know.

Bounds-check elision is the second half of the performance story and the reason slice iteration beats indexing. Indexing with v[i] inserts a runtime bounds check per access, while slice::iter yields references through unchecked pointer walks the compiler can prove safe — and auto-vectorize. Chains over .iter() routinely compile to SIMD instructions where the equivalent index loop stalls on branches. The practical rule is blunt: never index in a loop when an iterator reaches the same elements, and treat every clippy::needless_range_loop warning as a missed vectorization opportunity rather than style nitpicking.

Ordering adapters correctly is the optimization juniors can apply on day one. Filter before map so rejected items skip the expensive closure — on a 10M-row ingest where 70% of rows fail the predicate, hoisting filter first cut mapping work by 2.4 seconds per batch. Prefer take and position for early exit over collecting everything then slicing. And remember that sum, fold, and for_each stream through registers while collect materializes heap memory: when you only need an aggregate, collecting first is pure waste at allocator speed.

Benchmarks need release mode and a fixed methodology or they lie. Debug builds leave bounds checks and unoptimized closures in place, making iterators look 4-8x slower than they ship. The workflow is cargo test --release for timing harnesses or a proper criterion bench comparing chain versus hand loop on identical inputs, with cache-warm runs and pinned CPU frequency for anything you intend to quote. Twice now I've watched teams rewrite chains into loops based on debug-mode numbers, then rewrite back after release numbers showed parity — measure in the mode you ship or don't measure at all.

Copy semantics interact with codegen in ways worth knowing precisely. For Copy element types, .iter().copied() and .into_iter() compile to identical walks — the optimizer sees through both — so prefer the borrowing form whenever the source must survive. Cloned() on non-Copy types pays per-element clone costs that dwarf adapter overhead, which makes it the first suspect when a chain profiles slower than its hand-rolled twin. Chunks_exact beats chunks where the remainder branch matters: the exact variant skips per-chunk length checks and vectorizes cleanly, while chunks carries a remainder path that can block SIMD. These are second-order effects behind fusion and filter ordering, but they decide the last fifteen percent.

For_each sits outside the fusion story and reviewers should notice. Because it executes side effects per element, the optimizer cannot reorder or eliminate its body the way it can with pure map closures feeding a sum. Chains ending in for_each still skip intermediate allocations, but the per-element work stays exactly as written — no vectorization across opaque side effects. That is fine for sinks, loggers, and buffer writes; it is a reason to keep transforms pure and pushed into map where the optimizer can see them. Purity isn't philosophy here, it's the precondition the optimizer needs to do its best work.

Benchmark hygiene decides whether your numbers mean anything. Wrap chain outputs in std::hint::black_box so the optimizer can't delete supposedly dead computation — without it, a sum over an unused result compiles to nothing and your benchmark measures an empty loop. Warm caches with a throwaway pass, pin inputs across compared variants, and report medians over repeated runs rather than single bests. For custom iterators, mark tiny next methods #[inline] so monomorphized consumers fuse across crate boundaries; without inlining, the call survives as a real function call and fusion dies at the module edge.

use std::time::Instant;
fn chain_sum(data: &[u64]) -> u64 {
    // Filter first: rejected items never enter the expensive map closure.
    data.iter().filter(|x| *x % 2 == 0).map(|x| x * x).sum()
}
fn loop_sum(data: &[u64]) -> u64 {
    let mut acc = 0u64;
    for x in data.iter() {
        if *x % 2 == 0 {
            acc += *x * *x;
        }
    }
    acc
}
fn main() {
    // Always benchmark optimized code: debug numbers lie about iterators.
    let data: Vec<u64> = (0..5_000_000).collect();
    for f in [chain_sum, loop_sum] {
        let t = Instant::now();
        let r = f(&data);
        println!("result={r} elapsed={:?}", t.elapsed());
    }
}

Pitfalls That Bite Seniors: Double Collects and Invalidation Myths

The double-collect is the incident that opens this article, and the general rule behind it is simple: every collect is a memory allocation with your name on it, so each one needs a justification. Collecting twice from re-created sources doubles peak memory. Collecting mid-chain to appease the borrow checker, then continuing to iterate the Vec, splits one pass into two and doubles cache traffic. The fixes rank in order of preference: fuse into a single pass computing both results, collect once and borrow the Vec twice, or bound the materialization with take when only a prefix is needed. Clippy's needless_collect lint flags the laziest instances, but architectural double-collects need reviewers who ask why this Vec exists.

Iterator invalidation is the C++ trauma that doesn't apply — and misunderstanding the difference causes both over-caution and real bugs. In C++, mutating a vector can invalidate outstanding iterators silently. In Rust, the borrow checker makes that state unrepresentable: you cannot hold an iter borrow and push to the Vec simultaneously, full stop. The myth leads engineers to clone defensively where borrowing sufficed, copying megabytes per call to dodge a hazard the compiler already prevents. Trust the borrow checker here; it is strictly stronger than the C++ convention it replaces.

What actually bites is subtler: logic errors the type system can't see. Mutating shared state inside map closures makes results order-dependent and breaks the day you parallelize with rayon. Forgetting that sort and dedup require the Vec form — iterators have no ordering methods because they're streams, not buffers — leads to collect-sort-collect round trips that should have been one sort_by on the Vec. And zip's silent truncation at the shorter input has corrupted more joins than any other single behavior: always assert equal lengths before zipping parallel arrays, or use an explicit indexed loop that fails loudly.

The last pitfall is size_hint abuse in both directions. Ignoring hints when you know the output size forfeits pre-allocation and pays logarithmic reallocations past a million elements — Vec::with_capacity plus extend is the fix. Fabricating optimistic hints in custom iterators over-allocates on every collect downstream. Report exact bounds when the math is exact, (0, None) when it isn't, and let the consumers do their job. Between honest hints, single-pass chains, and borrow-checked invalidation safety, the pitfall list shrinks to discipline problems — which is exactly what code review is for.

Ordering prerequisites bite in two classic spots. Dedup removes only consecutive duplicates, so unsorted input keeps scattered repeats — sort first or reach for a HashSet filter when order must be preserved. Windows borrows the slice for the iterator's lifetime, which collides with mutation inside the loop body; collect the windows or restructure into indexed chunks when the body must write back. Both errors read as borrow complaints pointing at innocent lines, and both resolve the moment you name the prerequisite out loud.

Infinite iterators complete the hazard tour. Cycle, repeat, and repeat_with never return None, so collecting them hangs forever and count never terminates — every infinite chain must end in take, take_while, or a short-circuiting consumer like find or any. Nth skips efficiently without materializing discarded items, step_by panics on zero at construction, and last walks the entire stream (there is no shortcut to the final element of a lazy chain). Knowing which consumers terminate early and which walk everything is the difference between a streaming pipeline and a hang that pages on-call at midnight.

Exact-length trust has its own sharp edge in ExactSizeIterator::len. Adapters like zip and take consult len for capacity and short-circuiting, so a custom len that lies — undercounting after next_back calls, for instance — corrupts downstream pre-allocation silently. Implement len as front-minus-back arithmetic on your own cursors and test it after mixed next/next_back consumption. And treat TrustedLen as what it is: an unsafe promise for standard-library internals, not a tool for application code. Honest hints plus tested len cover every legitimate optimization without a single unsafe block.

fn main() {
    let data: Vec<i32> = (1..=1_000_000).collect();
    // ONE pass, two results: no intermediate Vec, no double walk.
    let (sum, count) = data.iter().filter(|x| *x % 2 == 0).fold(
        (0i64, 0u64),
        |(s, c), x| (s + *x as i64, c + 1),
    );
    println!("sum={sum} count={count}");
    // zip truncates silently: assert lengths first, always.
    let a = vec![1, 2, 3];
    let b = vec![4, 5];
    assert_eq!(a.len(), b.len(), "parallel arrays diverged");
    let _paired: Vec<(i32, i32)> = a.iter().cloned().zip(b.iter().cloned()).collect();
    // Pre-reserve when you know the size: skips realloc churn.
    let mut out = Vec::with_capacity(data.len() / 2);
    out.extend(data.iter().filter(|x| *x % 2 == 0));
    assert_eq!(out.len() as u64, count);
}