Advanced 5 min · March 16, 2026

NumPy Loop vs Vectorisation: 45-Minute Training Bottleneck

Python for-loop over 50M rows caused 48-minute single-core bottleneck.

Naren · Founder

Plain-English first. Then code. Then the interview question.

⚡Quick Answer

A Python loop over a NumPy array is 10–100x slower than a vectorised operation — each iteration pays interpreter overhead
np.vectorize() is NOT a performance tool — it wraps a Python loop in a convenience API, same speed
Most loop patterns have a NumPy equivalent: np.diff, np.cumsum, np.clip, broadcasting, boolean masking
Vectorised operations run in compiled C with no per-element type checks or reference counting
For recurrences that cannot be vectorised, Numba @njit compiles to machine code — 50–200x faster than Python loops
Biggest mistake: reaching for np.vectorize expecting speed — it is syntax sugar, not optimisation

The Cost of Python Loops

Every iteration of a Python for-loop over a NumPy array pays a fixed interpreter overhead that has nothing to do with the operation you're trying to compute. The runtime pays for: extracting the element from the array buffer and converting it to a Python float object (boxing), executing the loop body in Python bytecode, managing the reference count on every temporary object, and deallocating those objects at the end of each iteration.

This overhead is approximately 300 nanoseconds per element. It doesn't scale with the complexity of the operation — a simple addition and a complex trigonometric function each cost roughly the same interpreter overhead per iteration. The actual computation is a small fraction of the total cost for simple operations.

A vectorised call like arr.sum() takes a completely different path. NumPy passes a raw C pointer directly to a compiled function that processes the entire array using SIMD (Single Instruction, Multiple Data) instructions. Modern CPUs with AVX2 support process 4 double-precision floats per instruction. There is no Python overhead per element — zero boxing, zero reference counting, zero bytecode interpretation.

The overhead isn't proportional to the work per element — it's a fixed tax. For a trivial operation like addition, the Python overhead is roughly 99% of the total runtime. For a genuinely complex per-element computation, the C code dominates and the Python overhead shrinks in proportion. This is why the gap is largest for simple element-wise operations (100–150x) and smaller for complex per-element work.

In production pipelines, this overhead compounds badly. A 50M-element array processed in a Python loop at 300ns per element takes 15 seconds. The same array processed with a vectorised ufunc takes roughly 100 milliseconds. That's a 150x difference between a pipeline that meets its SLA and one that blows through it by 45 minutes.

Replacing Common Loop Patterns

Most Python loops over NumPy arrays are not doing anything fundamentally sequential. They just look sequential because element-by-element thinking is the default mental model. The patterns that appear most often in production codebases all have direct vectorised replacements — once you learn to recognise the pattern, the replacement is mechanical.

Element-wise arithmetic is the simplest: any loop that computes output[i] = f(input[i]) using arithmetic operators is directly replaceable with broadcasting. NumPy broadcasts scalar operations across the entire array in a single compiled call.

np.diff handles adjacent differences: any loop computing output[i] = arr[i+1] - arr[i] is a np.diff(arr) call.

np.cumsum and np.cumprod handle running totals and products. They also enable the rolling window trick, which is the most impactful pattern to recognise.

np.clip handles value clamping without a conditional per element. np.where handles two-branch conditionals. np.select handles multiple branches.

The rolling window pattern deserves particular attention because it appears constantly in time-series and signal processing code and is non-obvious to vectorise at first glance. The key insight is that a rolling sum of window W at position i is: sum(arr[i-W:i]) = cumsum[i] - cumsum[i-W]. If you have the prefix sum array precomputed, every window sum is a single subtraction — O(1) per element, no inner loop, no temporary array allocation. Rolling mean follows immediately. Rolling variance uses the computational formula Var(X) = E[X²] - E[X]², decomposed the same way with a second prefix sum over the squared values.

When You Cannot Avoid a Loop — np.vectorize vs Numba

Some operations genuinely cannot be vectorised because each output depends on a previously computed output — not on the input at the same position. Running maximums, exponential smoothing, Fibonacci-style recurrences, and certain signal filters all have this structure. These are not problems of pattern recognition — they are structurally sequential. The question then is not whether to loop, but where the loop should run.

np.vectorize is the most misunderstood function in NumPy. Engineers see 'vectorize' in the name and assume it implies compiled execution or parallelism. It implies neither. Reading the NumPy documentation directly: 'The vectorized function evaluates pyfunc over successive tuples of the input arrays like the python map function, except it uses the broadcasting rules of numpy.' The implementation is a Python loop with thin type-coercion wrapping on each side. It provides convenience — you can pass array arguments to a function that expects scalars — not speed. Benchmarking np.vectorize against a hand-written Python loop reliably shows the same runtime, or slightly worse due to the extra function call overhead per element.

For genuinely sequential operations that must run fast, Numba is the correct tool. The @njit decorator ('no Python, just-in-time') compiles the decorated function to LLVM machine code on the first call. The compiled function runs with zero Python overhead — the sequential logic is preserved, but the interpreter is completely bypassed. Compilation takes roughly 0.5–2 seconds on the first call, after which the compiled version is cached. For a loop over 10M elements that previously took 3 seconds in Python, the Numba-compiled version typically runs in 20–50 milliseconds — a 60–150x improvement.

The Numba constraint is that all types must be statically inferable from the function signature. This means: no Python dicts or sets inside @njit functions, no arbitrary Python objects, and no calls to Python functions that aren't themselves @njit-compiled. For data pipeline work using NumPy arrays and scalars, this is almost never a limitation in practice.

Cython is an alternative when the team already uses it and wants a separate build step with explicit type annotations in a .pyx file. The performance outcome is comparable to Numba. Numba requires less code change — a decorator on an existing Python function versus a rewrite in Cython syntax — which makes it the right default for most teams.

Broadcasting and Memory Layout: The Hidden Performance Factors

Vectorisation eliminates Python interpreter overhead, but two additional factors determine how fast the resulting C-compiled operations run: whether NumPy can apply broadcasting without creating intermediate copies, and whether the memory access pattern aligns with CPU cache lines.

Broadcasting lets NumPy perform operations between arrays of different shapes without allocating expanded copies. When you subtract a mean vector of shape (1000,) from a matrix of shape (1000, 50), NumPy broadcasts the subtraction without materialising a (1000, 50) copy of the mean vector. This is fast. When the operation requires an intermediate result that doesn't fit the broadcast rules, NumPy allocates a full-size temporary array — and that allocation, which happens in C but still consumes memory bandwidth, can dominate the runtime for large arrays.

Memory layout — C-order (row-major) versus Fortran-order (column-major) — matters because CPU caches load contiguous memory efficiently. A C-order NumPy array stores rows contiguously. Iterating over rows, or applying operations along axis=1, accesses contiguous memory and benefits from cache prefetching. Iterating over columns accesses every nth element in memory, causing cache misses on every access. For large arrays, cache-unfriendly access patterns can cost 5–10x in throughput versus cache-friendly access, independent of Python versus C execution.

In-place operations avoid allocation entirely. arr += 1.0 modifies the array in place and allocates no temporary. arr + 1.0 allocates a new array of the same size to hold the result, then the original may be garbage collected. For a 1GB array, that allocation difference is the difference between a 1GB and 2GB peak memory usage — which may or may not matter, but it's always worth being explicit about.

NumPy Loop Optimisation Methods: Complete Comparison

Method	Speedup vs Python Loop	When to Use	Trade-off
Vectorised ufunc / broadcasting	100–300x	Element-wise arithmetic, comparisons, math functions, aggregations — any operation where output[i] depends only on input[i] or the whole array	Requires expressing the logic as NumPy operations. Not all logic translates directly, but most element-wise operations do.
Boolean masking / np.where / np.select	100–200x	Conditional assignment per element — if/else logic, value replacement, multi-branch selection	np.where evaluates both branches for all elements before selecting — use boolean masking when one branch is expensive and rarely true, to evaluate only the matching subset.
np.diff / np.cumsum / np.clip / rolling cumsum trick	50–150x	Adjacent differences, running totals, value clamping, rolling window statistics — specific mathematical patterns with direct NumPy equivalents	Only applies when the operation fits one of these patterns. Rolling cumsum only works for associative operations (sum, sum of squares) — not median, quantile, or mode.
Numba @njit	50–200x	Sequential dependencies (result[i] uses result[i-1]), recurrences, exponential smoothing, running statistics that require prior output values	One-time LLVM compilation cost of 0.5–2s on first call. Types must be statically inferable — no arbitrary Python objects inside @njit functions. Numba must be installed separately.
np.vectorize	0–5% (negligible, sometimes negative)	API convenience only — applying a scalar Python function to an array input when you do not care about performance and want to avoid writing the loop yourself	No performance benefit at all. The name is misleading. Benchmark against the raw loop before using — the runtimes will match. Never use this expecting speedup.
Cython	50–200x	Large production codebases that already use Cython, or when explicit C-level type annotations are preferred over Numba's type inference	Requires separate .pyx files, explicit type annotations, and a build step integrated into the package setup. Higher setup overhead than Numba for new code.
In-place operations + keepdims broadcasting	1.5–3x (additive after other optimisations)	Large arrays (>100MB) where memory allocation time is a measurable fraction of total operation time	Modifies the original array — requires explicit .copy() if the original must be preserved. Cannot always replace out-of-place operations when intermediate results are needed.

Key Takeaways

Python loops over NumPy arrays pay ~300ns per element in interpreter overhead — a fixed tax independent of operation complexity. Vectorised ufuncs pay ~0.2ns per element in compiled C. The 150x gap is structural, not incidental.
np.vectorize is API convenience with zero performance benefit — it runs a Python loop. The name is misleading. Benchmark it against the raw loop to confirm they match, then use the right tool instead.
Most loop patterns have a direct NumPy replacement: np.diff for adjacent differences, np.cumsum for running totals, np.clip for clamping, np.where for conditionals. The rolling window pattern — the most common hidden bottleneck — is vectorisable using the prefix sum trick.
For loops with genuine sequential dependencies (result[i] depends on result[i-1]), use Numba @njit — it compiles the exact same logic to machine code with 50–200x speedup over the Python loop while preserving the sequential structure.
Scaling hardware does not fix algorithmic inefficiency — the GIL serialises Python loops regardless of core count. Profile first, identify the actual bottleneck, then vectorise or compile before reaching for a larger instance.

Common Mistakes to Avoid

Using np.vectorize expecting a performance speedup
Symptom: Code is 'vectorised' with np.vectorize, benchmarks show identical runtime to the raw Python loop, engineer concludes the loop 'cannot be optimised further'
Fix: np.vectorize is a Python loop with type-coercion wrapping — it provides zero execution speedup. For actual performance improvement: if the operation is expressible as NumPy ufuncs or broadcasting, write it that way. If the loop has genuine sequential dependencies, use Numba @njit. The name is the trap — benchmark np.vectorize against the raw loop first to confirm they match, then choose the right tool.
Writing a Python loop for element-wise arithmetic instead of broadcasting
Symptom: Simple scaling, shifting, or clipping operations run 100x slower than expected — monitoring shows one CPU core at 100%, others idle
Fix: Replace with broadcasting expressions: arr * scale, arr + offset, np.clip(arr, lo, hi), np.exp(arr), np.log(arr). NumPy broadcasts scalars and shape-compatible arrays across the full array in compiled C. These are single function calls with zero Python overhead per element.
Writing a Python loop for rolling window statistics
Symptom: Feature engineering pipeline runs for minutes instead of seconds — profiling shows the loop creating temporary array allocations per iteration, RSS growing and shrinking in a sawtooth pattern
Fix: Use the cumulative sum trick: cs = np.cumsum(np.insert(arr, 0, 0)); rolling_sum = cs[window:] - cs[:-window]; rolling_mean = rolling_sum / window. Extend to variance with a second prefix sum over squared values. No Python loop, no temporary allocations, O(n) with a small constant. Clip variance to zero before sqrt to handle floating-point precision artifacts.
Scaling hardware to fix a Python loop bottleneck
Symptom: Instance upgraded from 4 to 16 vCPU and runtime improves by less than 10% — monitoring confirms one core at 100% and the rest idle throughout the run
Fix: The GIL prevents multiple threads from executing Python bytecode simultaneously. More cores do not parallelize a Python loop. Vectorise the loop with NumPy or compile it with Numba @njit. Fix the algorithm — then evaluate whether additional cores provide further benefit through NumPy's internal BLAS/LAPACK threading.
Optimising the wrong loop — spending time on a 2% bottleneck while ignoring the 95% bottleneck
Symptom: Engineer spends a day vectorising a loop that shows no measurable improvement to total pipeline runtime — the real bottleneck is I/O, memory allocation, or a different loop entirely
Fix: Profile first with %timeit or cProfile before writing any optimisation code. Use line_profiler for per-line timing within functions. Identify that the loop is actually the bottleneck before optimising it. A rule of thumb: if the loop accounts for less than 20% of total runtime, optimising it will not move the needle on the metric you care about.

Interview Questions on This Topic

QWhy is iterating over a NumPy array with a Python for loop slow?JuniorReveal
The root cause is that Python for-loops over NumPy arrays execute in the Python interpreter, paying a fixed overhead per element that has nothing to do with the computation being performed. For each iteration, the interpreter: extracts the element from the array's C buffer, boxes it as a Python float object (allocating a Python object on the heap), executes the loop body in Python bytecode, manages reference counts on every temporary object, and deallocates those objects at the end of the iteration. This overhead runs to approximately 300 nanoseconds per element — regardless of whether the operation inside the loop is a trivial addition or a complex computation. A vectorised call like arr.sum() takes a completely different path: NumPy hands a raw C pointer directly to a compiled function. The compiled function runs a C-level loop with SIMD instructions — modern CPUs process 4 float64 values per instruction with AVX2. There is no Python overhead per element at all. For a 1M-element array with simple addition, the Python loop takes roughly 300ms and the vectorised call takes roughly 2ms — a 150x difference. The gap is largest for simple operations where the Python overhead dominates the actual computation, and shrinks for complex per-element work where the C computation starts to take significant time itself.
QWhat does np.vectorize actually do under the hood?JuniorReveal
np.vectorize is a Python function that calls your provided function once per element of the input array, in a Python for-loop. It does not compile your function to C or machine code. It does not parallelise execution. It does not use SIMD instructions. The NumPy documentation says explicitly: 'The vectorized function evaluates pyfunc over successive tuples of the input arrays like the python map function' and 'The vectorize function is provided primarily for convenience, not for performance.' What np.vectorize actually provides is: automatic handling of NumPy broadcasting for scalar functions (so a function that expects scalar inputs works with array inputs), automatic stacking of scalar return values into an output array, and optional support for specifying output dtype and excluded parameters. The performance is identical to — or marginally worse than — a hand-written Python loop, because it adds a thin layer of type-coercion and broadcasting handling on top of the same per-element Python function calls. The name is the problem. Engineers see 'vectorize' and assume it implies compiled execution. It doesn't. For actual performance improvement on non-vectorisable operations, use Numba's @njit decorator, which compiles the decorated function to LLVM machine code on first call and runs with no Python overhead on subsequent calls.
QA feature engineering pipeline processes a 50M-row dataset with a rolling window loop. The loop computes rolling mean and std for a window of 100. How would you optimise this?SeniorReveal
This is the rolling window pattern, and it's fully vectorisable using prefix sums — no sequential dependency exists between windows despite appearances. For rolling mean: compute the prefix sum with a zero prepended, then use strided indexing to get the window sum at each position. cumsum = np.cumsum(np.insert(arr, 0, 0.0)) rolling_sum = cumsum[window:] - cumsum[:-window] rolling_mean = rolling_sum / window For rolling variance, use the computational formula Var(X) = E[X²] - E[X]²: cumsum2 = np.cumsum(np.insert(arr2, 0, 0.0)) rolling_sum2 = cumsum2[window:] - cumsum2[:-window] rolling_var = rolling_sum2 / window - rolling_mean2 rolling_std = np.sqrt(np.maximum(rolling_var, 0.0)) # clip negatives from float precision The np.maximum(rolling_var, 0.0) before sqrt is important — floating-point precision loss in the subtraction of two large numbers can produce small negative values that cause sqrt to return NaN. For a 50M-row dataset with window=100, this implementation runs in approximately 11 seconds versus 48 minutes for the Python loop version — roughly a 260x improvement on the same hardware. This approach only works for associative operations — sum, sum of squares, count. For rolling median, rolling quantile, or rolling mode, the prefix sum trick doesn't apply. For those, scipy.ndimage provides optimised implementations, or a Numba-compiled loop is the right tool.
QWhen would you choose Numba @njit over a vectorised NumPy approach, and what are the practical limitations?SeniorReveal
Numba @njit is the right choice when the operation has genuine sequential dependencies — when computing result[i] requires result[i-1] or earlier outputs — making the loop structurally impossible to vectorise with standard NumPy operations. Canonical examples: exponential weighted moving average (EWMA), running maximum or minimum, certain IIR filters, Viterbi algorithm, and any finite state machine computed over a sequence. These all have the property that the output at each position depends on the output at the prior position, not just the input. For these operations, Numba @njit compiles the decorated function to LLVM machine code on the first call (typically 0.5–2 seconds of one-time overhead). Subsequent calls run at near-C speed with no Python overhead. For a 10M-element running maximum, Numba typically achieves roughly 170x speedup over the Python loop equivalent. Practical limitations: All types used inside the @njit function must be statically inferable — Numba performs type inference at compile time. This means no Python dicts or sets (use structured arrays), no arbitrary Python objects, and no calls to Python functions that aren't themselves @njit-compiled. On first encountering a limitation, the error message from Numba is specific about what construct caused the issue, which makes debugging straightforward. For one-off analysis scripts where Numba installation adds complexity, scipy.ndimage provides optimised implementations of many rolling statistics. For pandas-integrated pipelines, bottleneck provides highly optimised replacements for common pandas/NumPy rolling operations. Numba is the right choice when you need a custom sequential algorithm that none of these libraries cover.

Frequently Asked Questions

Is np.vectorize() faster than a Python loop?

No — or so marginally that the difference is noise. The NumPy documentation states directly that np.vectorize is 'provided primarily for convenience, not for performance.' It calls your Python function once per element in a Python loop, adding thin type-coercion handling on each side. Benchmarking consistently shows np.vectorize matching or slightly underperforming a hand-written Python loop on the same operation.

Use np.vectorize when you want the convenience of applying a scalar Python function to array inputs without writing the loop yourself, and when you have no performance requirement. Do not use it when you expect speedup — you will be disappointed and confused.

What is the fastest way to apply a custom function to each element of a NumPy array?

It depends on what the function does.

If the function can be expressed using NumPy operations — arithmetic, comparisons, math functions, boolean logic — express it that way directly. This is the fastest option: the loop runs in compiled C with SIMD and there is no Python overhead per element.

If the function cannot be expressed with NumPy operations because it has sequential dependencies, use Numba: decorate with @numba.njit and it compiles to LLVM machine code on the first call. Subsequent calls run at near-C speed. This is the right answer for recurrences, state machines, and operations where output[i] depends on output[i-1].

If the function calls external Python code or libraries that Numba cannot compile, there is no way to eliminate the per-element Python overhead. In that case, focus on batching the calls to reduce the per-element fixed cost, or use multiprocessing if each element's computation is large enough to justify serialization overhead.

Can I parallelise a Python loop over a NumPy array using multiprocessing?

Technically yes, practically rarely the right answer. The serialisation cost of passing large NumPy arrays between processes — pickling, copying, unpickling — often consumes more time than the computation being parallelised. For arrays above ~10MB, the IPC overhead typically dominates.

For CPU-bound element-wise operations, vectorisation or Numba @njit is faster and simpler because the computation stays in a single process with no serialisation overhead.

Multiprocessing earns its place when each element requires significant independent computation that dwarfs the serialisation cost — for example, running ML model inference or a complex simulation per element, where each call takes tens of milliseconds. In that scenario, the per-element work dominates and parallelism across processes provides real throughput improvement. Use concurrent.futures.ProcessPoolExecutor for this pattern and benchmark with actual array sizes before committing to the architecture.

How do I know if my loop can be vectorised?

Ask one question: does computing result[i] require the value of result[i-1] or any earlier output?

If the answer is no — result[i] depends only on input values, not on prior outputs — the loop can be vectorised. The computation at each position is independent of all others, which is the only structural requirement for vectorisation. Find the NumPy equivalent (broadcasting, ufunc, np.where, cumsum trick) and replace the loop.

If the answer is yes — computing result[i] requires result[i-1] — the loop has a sequential dependency and cannot be vectorised with standard NumPy. Use Numba @njit to preserve the sequential structure while compiling it to machine code.

A common source of confusion: loops that read from input[i-1] but write to result[i] are NOT sequential in the relevant sense. Reading from the input array at any position is fine — the input doesn't change. The constraint is only on reading from previously computed output values.

🔥

That's Python Libraries. Mark it forged?

5 min read · try the examples if you haven't