Indexing in DBMS — Missing Foreign Key Broke Checkout
A missing foreign key index on a 2-million-row table caused 12-second scans and 100% CPU, taking down checkout —avoid this with proper indexing..
20+ years shipping production systems from the metal up. Written from production experience, not tutorials.
- ✓Solid grasp of fundamentals
- ✓Comfortable reading code examples
- ✓Basic production concepts
- An index is a sorted lookup structure that maps column values to disk locations, avoiding full table scans
- B-Tree indexes are default and support equality and range queries; Hash indexes are O(1) for equality only
- Clustered indexes reorder rows on disk (one per table); non-clustered indexes are separate structures requiring key lookups
- Performance: Index seek on 1M rows is ~0.1ms vs full scan ~2s — a 20,000x difference
- Production trap: Every index adds write overhead — 8 indexes on a 10K writes/sec table means 80K extra I/O ops per second
- Biggest mistake: Indexing low-cardinality columns alone — the planner will ignore the index at >20% row selectivity
Imagine a 1,000-page cookbook with no table of contents. To find the recipe for 'chocolate cake' you'd flip every single page. An index at the back of the book says 'chocolate cake → page 847' and you jump straight there. A database index works exactly the same way — it's a separate, pre-sorted lookup structure that tells the database engine exactly where your data lives on disk, so it never has to flip through every page.
Without an index, your database degrades to a full table scan on every query—reading millions of rows just to find ten. That’s why indexing is the single most impactful performance lever you control. Get the wrong index or none at all, and your users will feel the delay; pick the right one, and your queries will stay sub‑millisecond even as data grows.
Why Indexing Is Not Optional
An index is a separate data structure — typically a B+ tree — that maps column values to row locations, enabling O(log n) lookups instead of O(n) full table scans. Without it, every query degenerates into a sequential read of all pages, which is the single most common cause of production latency. The core mechanic is simple: trade write overhead for read speed by maintaining a sorted or hashed copy of a subset of columns.
In practice, a B+ tree index stores key-pointer pairs in leaf nodes linked as a linked list, allowing both point lookups and range scans in logarithmic time. The index is physically separate from the heap or clustered table, so a query that uses an index performs two I/O operations: one to traverse the tree, another to fetch the row. The selectivity of the indexed column determines whether the optimizer will use it — a cardinality below 1% of total rows is the typical threshold.
Use indexes on columns that appear in WHERE, JOIN, ORDER BY, or GROUP BY clauses, especially foreign keys. The canonical failure is a missing foreign key index causing a cascading full table scan on every join — exactly what broke checkout in the motivating example. Every foreign key column should be indexed unless you have measured and proven otherwise.
How a Database Actually Finds Data Without an Index
Before we talk about indexes, we need to understand what they replace. When you run a query like SELECT * FROM orders WHERE customer_id = 4821, the database has two options: scan every row in the table checking each customer_id one by one, or consult an index to jump directly to the answer.
The first option is a Full Table Scan. It sounds terrible, but for a table with 500 rows it's actually fine — the overhead of an index lookup can even be slower for tiny datasets. The problem explodes when your orders table has 50 million rows and you're running that query 10,000 times per second.
Disk I/O is the bottleneck. Data lives in fixed-size blocks (usually 8KB or 16KB) on disk. A full scan means reading every block. An index is designed so that finding a specific value requires reading only a handful of blocks — typically O(log n) reads for a B-Tree index — regardless of whether the table has 1,000 rows or 1 billion.
This is the core contract: indexes trade extra disk space and slower writes for dramatically faster reads. That trade-off is the single most important idea in this entire article.
B-Tree vs Hash Indexes — Picking the Right Tool for the Job
There isn't just one type of index. The right choice depends entirely on the kind of queries you run.
B-Tree (Balanced Tree) is the default in every major RDBMS — PostgreSQL, MySQL, Oracle, SQL Server. It stores index entries in a sorted tree structure. The sorted order means it handles equality lookups (=), range queries (BETWEEN, <, >), and ORDER BY efficiently. 90% of the time, B-Tree is what you want.
Hash Index stores a hash of the column value mapped to a row pointer. Hash lookups are O(1) — theoretically faster than B-Tree's O(log n) for equality checks. The catch: hash indexes are useless for range queries. WHERE price > 100 on a hash index results in a full scan because hashed values have no sorted relationship to each other.
Bitmap Index (Oracle, PostgreSQL partial support) is ideal for low-cardinality columns — columns with very few distinct values like status (active/inactive) or country_code. It stores a bit array per distinct value and uses bitwise AND/OR to combine multiple conditions. Never use bitmap indexes on high-cardinality columns like email — the storage overhead becomes catastrophic.
Choosing wrong here is a real interview red flag.
WHERE status = 'pending' in an orders table can be 50x smaller than a full index on status, because 98% of orders are 'completed' and never queried that way. Smaller index = fits in memory = faster. If your queries consistently filter on a specific subset of rows, a partial index is almost always the right call.Clustered vs Non-Clustered Indexes — The One That Actually Moves Your Data
This distinction trips people up in interviews constantly, so let's nail it.
A Clustered Index physically reorders the rows on disk to match the index order. Because the data itself IS the index, there can only ever be one clustered index per table. In MySQL InnoDB, the PRIMARY KEY is always the clustered index. SQL Server lets you choose which column to cluster on. When you read data via the clustered index, you get the actual row immediately — no second lookup needed.
A Non-Clustered Index is a completely separate structure that stores the indexed column values and a pointer back to the actual row (called a Row Identifier or RID in SQL Server, or the primary key in InnoDB). When the query engine uses a non-clustered index, it first finds the matching pointer in the index, then does a second lookup in the main table to fetch the remaining columns. This second lookup is called a Key Lookup or Bookmark Lookup and shows up in execution plans.
The practical implication: if a non-clustered index query needs many columns beyond what's in the index, those Key Lookups add up. The fix is a Covering Index — an index that includes all the columns a query needs, so the engine never has to touch the main table at all.
Covering indexes are one of the highest-impact performance optimisations you can make with zero application code changes.
When NOT to Add an Index — The Write Performance Tax
Here's what nobody tells beginners: indexes are never free. Every index you add is a liability on write-heavy tables, and getting this wrong in production is painful.
Every time you INSERT, UPDATE, or DELETE a row, the database must update every index on that table to keep it consistent. A table with 8 indexes doesn't just write once — it performs up to 9 writes per operation. On a table receiving 10,000 inserts per second, that's 80,000 extra I/O operations per second before you've even handled reads.
The situation gets worse with UPDATE. If you update an indexed column, the database must delete the old index entry and insert a new one — two write operations per index, per updated row.
The right mental model is to think of each index as a maintenance contract. The database pays that contract on every write, forever. You only justify that contract when read performance gains outweigh the write penalty.
Rules of thumb from the trenches: - Tables with > 80% writes and < 20% reads: index sparingly, only the absolute hottest read paths. - Tables with > 80% reads: index aggressively, your queries will thank you. - Event/audit log tables: almost never index beyond the primary key — they're append-only and rarely read column-by-column. - Foreign key columns: almost always index these — they're hit on every JOIN.
pg_stat_user_indexes in PostgreSQL (or sys.dm_db_index_usage_stats in SQL Server) to find indexes with zero or near-zero usage. These are burning disk space, slowing every write, and contributing to table bloat. Dropping an unused index on a write-heavy table can give you an immediate, measurable throughput boost with zero risk to read performance.Composite Indexes — Column Order Makes or Breaks Performance
A composite index spans multiple columns. The order of those columns determines which queries it can serve efficiently. This is one of the most misunderstood concepts in indexing.
The Leading Column Rule: The index can be used for queries that filter on the leading column alone, or on the leading column plus any subsequent columns. It cannot efficiently support a query that skips the leading column — that forces a full index scan or full table scan.
(department_id, hire_date)WHERE department_id = 7→ uses index (good)WHERE department_id = 7 AND hire_date > '2024-01-01'→ uses index very efficientlyWHERE hire_date > '2024-01-01'→ cannot use leading column; likely full scan
Selector First Rule: Put the column with the highest selectivity (most unique values) first. If department_id has 100 distinct values and hire_date has 10,000, then hire_date is more selective. But if queries always filter on department_id first, keep it leading.
The real-world strategy: List all your hot queries, group them by filter columns, then design composite indexes that cover multiple related queries by varying the column order. One index cannot serve all; you often need two or three well-placed indexes rather than fifteen scattered ones.
Covering indexes (using INCLUDE) can add extra columns without affecting the sort order, preserving the selectivity of the leading columns.
- Leading column determines the primary sort order — queries must include it to get the index benefit.
- Second column is sorted within each leading value — useful for refining after the first filter.
- Skipping the leading column forces a full index scan (still reads all leaf pages) or a full table scan.
- Design indexes around your actual query patterns, not theoretical coverage — look at slow query logs.
Multilevel Indexing — When a Single B-Tree Won't Fit in Memory
You just read about B-Trees. Good. Now imagine your index itself is too large to fit in RAM. This happens when your table has billions of rows. A single-level B-Tree still requires too many disk IOs for the root lookup. Multilevel indexing solves this by building an index on top of the index. Think of it as a directory of directories. The outer index points to blocks of the inner index. Each level shrinks the search space exponentially. Your database almost certainly uses this under the hood for large tables. You never see it. The query planner handles it. But you need to understand why a query that touches 1% of a 10-billion-row table can still return in milliseconds. It's not magic. It's multiple layers of sparse indexes. Each level stores only a few thousand entries. The top level fits entirely in cache. That means you find your data with 3-4 disk reads instead of 20. The cost? Insertions now update multiple index levels. Write-heavy tables pay this tax. If your workload is 90% reads, multilevel indexing is your best friend. If it's 90% writes, reconsider your schema.
Index Scan vs Seek — Why Range Queries Kill Hash Indexes
You slapped a hash index on the user email column. Great for login queries. Terrible for 'find all users registered in January'. Hash indexes only support equality lookups. No range scans. No ORDER BY. No prefix matching. The database has to fall back to a full table scan for anything that isn't an exact match. This is the #1 mistake junior devs make. They see 'index' and think it works everywhere. It doesn't. B-Tree indexes support both seek (exact match) and scan (range). When you query WHERE created_at BETWEEN '2025-01-01' AND '2025-01-31', the B-Tree navigates to the first matching leaf page, then walks the linked list of leaf pages forward. That's an index scan — sequential reads from the index, not the table. If your query filters on a leading column of a composite index, the same logic applies. The database decides between seek and scan based on selectivity. High selectivity (few rows) -> seek. Low selectivity (many rows) -> scan. You can force a seek with a hint in some databases, but don't. Trust the optimizer. It knows the statistics.
B-Tree vs LSM-Tree: Storage Engine Comparison
While B-trees dominate relational databases, LSM-trees (Log-Structured Merge-Trees) power many modern NoSQL systems like Cassandra, RocksDB, and LevelDB. The core difference lies in write amplification: B-trees update data in-place, causing random disk I/O, whereas LSM-trees buffer writes in memory (memtable) and flush them as immutable sorted SSTables, merging them in the background. This makes LSM-trees superior for write-heavy workloads—up to 10x faster writes—but they suffer from read amplification: a point query may need to check multiple SSTables and the memtable. B-trees excel at reads, especially range scans, because data is stored contiguously. For example, an e-commerce checkout system with frequent order inserts (write-heavy) might prefer LSM-trees, while a reporting system with complex range queries (read-heavy) benefits from B-trees. Bloom filters in LSM-trees reduce unnecessary SSTable lookups, but they add memory overhead. In practice, PostgreSQL uses B-trees by default, while MySQL's MyRocks engine uses LSM-trees. Choosing between them requires understanding your workload's read/write ratio and latency tolerance.
Index-Only Scans and Covering Indexes
An index-only scan occurs when a query can be satisfied entirely from the index without accessing the table's heap (data pages). This is possible when the index contains all columns referenced in the query (SELECT, WHERE, JOIN, ORDER BY). Such an index is called a covering index. For example, consider a table orders(id, user_id, total, status) with a composite index on (user_id, total). The query SELECT total FROM orders WHERE user_id = 123 can use an index-only scan because total is in the index. If the query also needed status, the index would not cover it, forcing a table lookup. To maximize index-only scans, include frequently selected columns in the index, but beware of index bloat—adding too many columns increases size and write overhead. In PostgreSQL, you can check if an index-only scan is used via EXPLAIN ANALYZE; look for "Index Only Scan" in the output. In MySQL, covering indexes are indicated by "Using index" in the Extra column. A practical example: in an e-commerce checkout, if you often query SELECT order_id, total FROM orders WHERE user_id = ? AND status = 'paid', create a covering index on (user_id, status, order_id, total). This avoids heap lookups, speeding up the checkout flow.
Partial, Expression, and Conditional Indexes
Not all indexes need to cover every row. Partial indexes (PostgreSQL, SQL Server) index only a subset of rows based on a WHERE clause, reducing index size and maintenance overhead. For example, in an orders table, you might index only rows where status = 'pending' because those are the ones queried frequently. Expression indexes (PostgreSQL, MySQL) index the result of a function or expression, enabling fast lookups on computed values. For instance, CREATE INDEX idx_lower_email ON users (LOWER(email)) speeds up case-insensitive email searches. Conditional indexes (SQL Server filtered indexes) are similar to partial indexes. A practical example: in a checkout system, you might have a partial index on (user_id, created_at) WHERE status = 'abandoned' to quickly find abandoned carts. Without it, a full index on status would include millions of 'completed' rows that are rarely queried. Expression indexes are useful for date truncation: CREATE INDEX idx_order_month ON orders (DATE_TRUNC('month', created_at)) to accelerate monthly reports. However, these indexes require the exact same expression in queries—otherwise they are ignored. In MySQL, expression indexes are supported from 8.0.13; earlier versions require generated columns. Use them judiciously: they optimize specific queries but add complexity.
The Missing Foreign Key Index That Took Down Checkout at Midnight
SELECT * FROM order_items WHERE order_id = ? took 12 seconds each, even though the table had only 2 million rows.order_id would cover all queries on the orders table. They didn't realize that a query filtering on customer_id — a foreign key column with no index — would force a full table scan regardless of the primary key.orders table had 2 million rows. The query SELECT * FROM orders WHERE customer_id = 4821 had no index on customer_id, so the database performed a sequential scan. Under normal load (5 queries/sec) this was tolerable at ~800ms. During Black Friday, the query ran 200 times per second, saturating disk I/O and causing all other queries to queue.customer_id: CREATE INDEX idx_orders_customer_id ON orders (customer_id);. Query time dropped from 800ms to 2ms. CPU dropped from 100% to 15%.- Every foreign key column needs an index — queries that JOIN or filter on FK columns will otherwise scan the whole table.
- Query performance is not linear — a single missing index under load causes system-wide degradation, not just one slow query.
- Monitor index usage with
pg_stat_user_indexesorsys.dm_db_index_usage_stats; unused indexes are a write tax, missing indexes are a read disaster.
EXPLAIN ANALYZE SELECT ... WHERE ...;Look for 'Index Scan', 'Index Seek' (good) vs 'Seq Scan' (bad).| File | Command / Code | Purpose |
|---|---|---|
| full_scan_vs_index.sql | CREATE TABLE orders ( | How a Database Actually Finds Data Without an Index |
| index_types_comparison.sql | CREATE TABLE products ( | B-Tree vs Hash Indexes |
| clustered_vs_nonclustered.sql | CREATE TABLE employees ( | Clustered vs Non-Clustered Indexes |
| index_write_overhead_demo.sql | CREATE TABLE event_log_over_indexed ( | When NOT to Add an Index |
| composite_index_order.sql | CREATE TABLE orders100k ( | Composite Indexes |
| multilevel_index_demo.sql | CREATE TABLE orders ( | Multilevel Indexing |
| index_scan_vs_seek.sql | CREATE TABLE events ( | Index Scan vs Seek |
| lsm_vs_btree_benchmark.sql | CREATE TABLE orders_btree (id SERIAL PRIMARY KEY, data TEXT); | B-Tree vs LSM-Tree |
| covering_index_example.sql | CREATE INDEX idx_covering ON orders (user_id, status) INCLUDE (order_id, total); | Index-Only Scans and Covering Indexes |
| partial_index_example.sql | CREATE INDEX idx_pending_orders ON orders (user_id, created_at) WHERE status = '... | Partial, Expression, and Conditional Indexes |
Key takeaways
pg_stat_user_indexes (PostgreSQL) or sys.dm_db_index_usage_stats (SQL Server) to find indexes with zero usage and drop themInterview Questions on This Topic
What is the difference between a clustered and non-clustered index, and why can a table have only one clustered index?
Frequently Asked Questions
20+ years shipping production systems from the metal up. Written from production experience, not tutorials.
That's DBMS. Mark it forged?
9 min read · try the examples if you haven't