NumPy Boolean Indexing and Fancy Indexing

Once you understand that NumPy arrays support masks and index arrays as index objects, a whole class of loop-free data manipulation opens up. Instead of iterating over rows to filter data, you describe the condition once and let NumPy handle the rest.

But that power comes with traps. Boolean masks with mismatched shapes crash silently. Fancy indexing with repeated indices produces copies you can't modify in-place. This article covers the mechanics, the performance reality, and the production failures that tripped us up.

Boolean Indexing — Filtering by Condition

Boolean indexing uses a True/False array of the same shape (or broadcastable) to select elements. It's the foundation for vectorized filtering — no Python loops. The resulting array is always a copy, but you can assign to the masked positions in-place using the same mask on the left-hand side.

import numpy as np

scores = np.array([72, 85, 91, 60, 78, 95, 55, 88])

# Students who passed (≥70)
passed = scores[scores >= 70]
print(passed)  # [72 85 91 78 95 88]

# Modify in-place: cap scores at 90
scores[scores > 90] = 90
print(scores)  # [72 85 90 60 78 90 55 88]

# Multiple conditions
print(scores[(scores >= 70) & (scores < 85)])  # [72 78]

np.where — Conditional Selection and Replacement

np.where is the vectorized conditional operator. With three arguments, it replicates x where cond is True and y where False — equivalent to an element-wise if‑else. With one argument, it returns the indices where the condition holds. This is essential for masking, clipping, and selection without a loop.

import numpy as np

temp = np.array([18.5, 22.1, 35.4, 8.2, 30.0])

# Replace values above 30 with 30 (clip)
clipped = np.where(temp > 30, 30.0, temp)
print(clipped)  # [18.5 22.1 30.  8.2 30. ]

# np.where with one argument returns indices
idxs = np.where(temp > 25)
print(idxs)  # (array([2, 4]),)
print(temp[idxs])  # [35.4 30. ]

Fancy Indexing and np.ix_

Fancy indexing uses integer arrays to select elements along each dimension. It's powerful for reordering rows, extracting submatrices, and complex selections. Without np.ix_, selecting a 2D submatrix requires careful broadcasting; np.ix_ builds the necessary broadcastable index arrays automatically. Fancy indexing always returns a copy, not a view.

import numpy as np

m = np.arange(16).reshape(4, 4)

# Select rows [0, 2] and columns [1, 3] — submatrix
print(m[np.ix_([0, 2], [1, 3])])
# [[ 1  3]
#  [ 9 11]]

# Sort by a column
data = np.array([[3, 1], [1, 4], [2, 0]])
sorted_by_col0 = data[data[:, 0].argsort()]
print(sorted_by_col0)
# [[1 4]
#  [2 0]
#  [3 1]]

Combining Boolean and Fancy Indexing

You can mix boolean masks and integer arrays in the same indexing expression. For example, arr[mask, cols] applies the mask to the rows and selects specific columns from those rows. This is powerful but easy to get wrong — the mask applies only to the axis it appears on. Common use: filter rows with a mask, then select a subset of columns by index.

import numpy as np

arr = np.arange(20).reshape(5, 4)
mask = arr[:, 0] > 5  # rows where first column > 5
selected = arr[mask, [0, 2]]  # from those rows, take columns 0 and 2
print(selected)
# [[ 8 10]
#  [12 14]
#  [16 18]]

# Equivalent using np.ix_:
rows = np.where(mask)[0]
selected2 = arr[np.ix_(rows, [0, 2])]
print(np.array_equal(selected, selected2))  # True

Performance Traps: Copy vs View and Memory Allocation

Boolean indexing and fancy indexing always return copies. Slicing returns a view. This difference has major performance implications: copying a large array can double memory usage and slow down operations. Knowing when you get a view vs copy saves both memory and debugging time.

import numpy as np

arr = np.ones((1000, 1000), dtype=np.float64)

# Slicing — view, no copy
view = arr[:500, :500]
print(view.base is arr)  # True

# Boolean indexing — copy
mask = arr[:, 0] > 0.5
copy = arr[mask, :]
print(copy.base is arr)  # True (copy, base is arr? Actually copy has own memory, base is None)

# Check memory usage after operations
import tracemalloc
tracemalloc.start()
_ = arr[arr > 0]
snapshot = tracemalloc.take_snapshot()
print("Peak memory during indexing:", snapshot.statistics('lineno')[0].size / 1e6, "MB")

Frequently Asked Questions