Senior 15 min · March 05, 2026
SQL CTEs — Common Table Expressions

CTE Referenced 3 Times — 50M Rows Scanned in SQL Server

A CTE referenced three times in SQL Server caused 150M logical reads on a 50M-row table.

N
Naren Founder & Principal Engineer

20+ years shipping high-throughput database systems. Notes here come from systems that actually shipped.

Follow
Production
production tested
May 23, 2026
last updated
1,554
articles · all by Naren
 ● Production Incident 🔎 Debug Guide
Quick Answer
  • A CTE (WITH clause) is a named, query-scoped result set — exists only for one query execution, no schema changes needed
  • Chain multiple CTEs in one WITH clause — later CTEs can reference earlier ones, creating a readable pipeline
  • Recursive CTEs need UNION ALL (not UNION), an anchor member (base case), and a recursive member that references itself
  • Materialization is engine-specific: PostgreSQL 12+ inlines by default; SQL Server may re-execute a CTE every time it is referenced
  • Data-modifying CTEs (PostgreSQL): DELETE/INSERT with RETURNING enables atomic archive-and-delete in one transaction
  • Biggest trap: referencing an expensive CTE multiple times in SQL Server — use a #temp table instead
✦ Definition~90s read
What is SQL CTEs?

A Common Table Expression (CTE) is a named temporary result set that exists only within the scope of a single SQL statement — think of it as a disposable view you define inline. Introduced in SQL Server 2005 (and available in PostgreSQL, MySQL 8+, Oracle, and others), CTEs solve the problem of writing readable, maintainable queries when you need to reference the same subquery multiple times, or when you're working with recursive data like org charts or tree structures.

Imagine you're building a massive LEGO set.

Unlike derived tables (subqueries in FROM clauses), CTEs can be referenced multiple times in the same query without duplicating code, and they support recursion via the WITH RECURSIVE clause (though SQL Server uses WITH without the keyword). The key tradeoff: CTEs are not materialized by default — SQL Server re-evaluates them each time they're referenced, which can lead to performance surprises when scanning 50 million rows.

Use CTEs for clarity and recursion; switch to temp tables or indexed views when you need deterministic performance at scale.

Plain-English First

Imagine you're building a massive LEGO set. Instead of dumping all 1,000 pieces on the floor and panicking, you sort them into labeled bags first — 'wheels bag', 'windows bag', 'roof bag'. A CTE is exactly that labeled bag. You name a mini-query, tuck it away with a label, then reference that label as if it were a real table throughout your main query. No mess, no repeated code, no confusion.

Every production database eventually grows a query so complex it becomes a maintenance nightmare — subqueries nested inside subqueries, the same filter logic copy-pasted five times, and a WHERE clause that scrolls off the screen. CTEs were introduced precisely because real SQL work at scale demands readable, composable, and debuggable queries.

The core problem CTEs solve is scope and readability without sacrificing power. Before CTEs, your only option was derived tables (anonymous subqueries in FROM clauses) or views (which require DDL permissions and pollute your schema). CTEs give you a named, scoped, temporary result set that lives only for the duration of your query.

By the end of this article you'll write multi-step CTEs that replace tangled subquery nests, use recursive CTEs to walk org charts, understand exactly when your database materializes a CTE versus re-evaluates it, and avoid the three production gotchas that catch even experienced engineers off guard.

What a CTE Actually Is (and Isn't)

A Common Table Expression (CTE) is a named, temporary result set defined within a single SQL statement's execution scope. It is not a temp table, not a view, and not materialized by default — it's a syntactic construct that the query optimizer can rewrite, inline, or execute multiple times. In SQL Server, a CTE referenced three times in the same query can cause the underlying table to be scanned three separate times, leading to 50M rows scanned where 16M would suffice.

A CTE is defined with the WITH clause and exists only for the duration of the query that follows it. Crucially, SQL Server does not automatically cache or spool the CTE's result set. Each reference to the CTE name is expanded independently by the optimizer, meaning the same complex join or aggregation inside the CTE definition may be executed once per reference. This is the root cause of the "CTE multiplied scans" antipattern.

Use CTEs for readability and recursive queries, but never assume they optimize like temp tables. When the same CTE is referenced multiple times — especially against large tables — explicitly materialize it into a temp table or table variable. A single #temp table with a clustered index can collapse three full scans into one, cutting logical reads from 150,000 to 50,000.

CTEs Are Not Materialized
A CTE referenced twice in the same query does not reuse work — SQL Server may execute the entire CTE definition once per reference, multiplying scans and joins.
Production Insight
A reporting query using the same CTE in three UNION ALL branches scanned a 16M-row fact table three times, causing 48M logical reads and 12-second execution. The symptom was high PAGEIOLATCH_SH waits on the fact table. Rule: if a CTE is referenced more than once, materialize it into a #temp table first.
Key Takeaway
CTEs are syntax, not storage — they do not materialize results.
Multiple references to the same CTE can multiply table scans and joins.
Materialize into a temp table when a CTE is referenced more than once.
CTE Behavior and Performance in SQL Server THECODEFORGE.IO CTE Behavior and Performance in SQL Server Structure, scope, materialization, and production patterns CTE Definition Non-materialized, inline view-like scope CTE Scope Limits Cannot reference CTE in another CTE Recursive CTE Tree traversal with anchor and recursive members Re-evaluation Risk CTE re-executed per reference, 50M scans Production Patterns Window functions, data marts, temp table fallback ⚠ CTE referenced 3 times → 50M rows scanned Materialize to temp table if reused multiple times THECODEFORGE.IO
thecodeforge.io
CTE Behavior and Performance in SQL Server
Sql Ctes Common Table Expressions

The Anatomy of a CTE — Structure, Scope and Why It Beats Subqueries

A CTE is defined with the WITH keyword, followed by a name, an optional column list, and the query body in parentheses. Everything after that closing parenthesis is the outer query. The CTE only exists for the lifetime of that single outer query execution.

You can define multiple CTEs in one WITH clause by separating them with commas. Crucially, a later CTE can reference an earlier one — this is what makes chaining possible. Think of it as a pipeline: each CTE refines the data one step further before handing it to the next stage.

Why does this beat a subquery? Readability is only part of the answer. The bigger win is debuggability. When you have a three-level nested subquery and something goes wrong, you can't easily isolate which level is producing bad data. With CTEs, you SELECT directly from each named stage during development — comment out the outer query and inspect intermediate results. That feedback loop is invaluable on complex data pipelines.

Also note: a CTE referenced multiple times in the outer query does NOT automatically get computed once and cached. Whether it's materialized or re-evaluated depends on your database engine — a critical performance detail covered in Section 3.

basic_cte_sales_pipeline.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
-- Find customers who placed >3 orders in 2023 AND spent >$500
-- then rank them by total spend

WITH orders_2023 AS (
    SELECT customer_id, order_id, order_total
    FROM orders
    WHERE EXTRACT(YEAR FROM order_date) = 2023
      AND order_status = 'completed'
),
customer_summary AS (
    SELECT customer_id,
           COUNT(order_id)   AS total_orders,
           SUM(order_total)  AS total_spend
    FROM orders_2023
    GROUP BY customer_id
),
high_value_customers AS (
    SELECT customer_id, total_orders, total_spend
    FROM customer_summary
    WHERE total_orders > 3 AND total_spend > 500.00
)
SELECT c.full_name, c.email,
       hvc.total_orders, hvc.total_spend,
       RANK() OVER (ORDER BY hvc.total_spend DESC) AS spend_rank
FROM high_value_customers hvc
JOIN customers c ON c.customer_id = hvc.customer_id
ORDER BY spend_rank;
Output
full_name | total_orders | total_spend | spend_rank
Sarah Okonkwo | 9 | 1842.50 | 1
Marcus Tran | 7 | 1205.00 | 2
Priya Nambiar | 5 | 1205.00 | 2
Leon Ferreira | 4 | 671.25 | 4
Debug Tip:
During development, temporarily replace the outer query with SELECT * FROM customer_summary LIMIT 20 to inspect intermediate CTE results. This is the single biggest productivity trick when building complex CTEs — treat each CTE like a unit you can test independently.
Production Insight
The most valuable CTE property is debuggability — you can comment out the outer query and SELECT from any intermediate stage.
In SQL Server, a CTE referenced N times in UNION ALL may execute N times — run SET STATISTICS IO ON to verify logical read counts.
For production SQL Server queries, treat expensive CTEs referenced more than once as temp table candidates.
Key Takeaway
CTEs are pipelines, not caches — name your stages, debug each independently.
SQL Server does not guarantee single execution — verify with STATISTICS IO before shipping.
For SQL JOINs across CTE stages, see SQL JOINs Explained for row-multiplication traps that apply equally here.

CTE Scope — Why You Can't Reference a CTE in Another Query

A CTE (common table expression) is defined within a single SQL statement. Its name only exists for the duration of that statement — from the WITH keyword to the final semicolon. Once the query finishes executing, the CTE disappears. This limited scope is by design: it allows the database to manage temporary names without creating any persistent objects.

Why can't you reference a CTE in a separate query? Because the database engine does not store the CTE's result anywhere persistent. It is part of the query plan that exists only during execution. If you try to use the CTE name in a different batch or even in a separate statement within the same script, you'll get an error like "invalid object name" or "relation does not exist." This is fundamentally different from a view or a temporary table, which both have a longer lifespan.

Consider two separate queries in a script: the first defines and uses a CTE; the second tries to reference that same CTE name. The second query fails immediately because the CTE was not defined in its own WITH clause. This is the most common "gotcha" for engineers new to CTEs — they expect the name to carry over like a variable in a programming language.

The fix is simple: if you need to reuse the same logic across multiple queries, create a view or a temporary table instead. CTEs are scoped to a single query for a reason — they encourage composability within that query without creating side effects that other queries might accidentally depend on.

Production Insight
Never assume a CTE name is available in a subsequent batch — it's not. If you need cross-query reuse, materialize into a temp table (SQL Server) or create a view.
This scoping rule prevents accidental schema pollution and makes CTEs safe for ad-hoc analysis scripts.
Key Takeaway
CTEs are query-scoped — they exist only within the single statement where they are defined. For reuse across multiple queries, use a temporary table or a view.
CTE Scope — Two Separate Queries
scope boundaryQuery 1: WITH cte AS (...)SELECT * FROM cteexecutes successfullyCTE exists only for Query 1Query 2: SELECT * FROM cteFAILS: 'cte' does not exist

Recursive CTEs — Walking Trees, Org Charts and Hierarchical Data

A recursive CTE has two parts separated by UNION ALL: the anchor member (the base case — where the recursion starts) and the recursive member (which references the CTE itself to walk one level deeper). The engine executes the anchor once, then keeps executing the recursive member against the previous iteration's results until it returns zero rows.

The classic use case is an organizational hierarchy stored as an adjacency list — each employee row has a manager_id pointing to another employee_id. Without a recursive CTE, flattening that requires N queries for N levels. With recursive CTEs, one query handles any depth.

Cycles in data are the most dangerous edge case. If employee A is the manager of B and B is the manager of A (data corruption happens), a naive recursive CTE loops until it hits the depth limit. Always track visited IDs and break when you detect a repeat.

recursive_cte_org_chart.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
-- Walk org chart from a given manager downward
-- Handles cycles by tracking visited IDs

WITH RECURSIVE org_hierarchy AS (
    -- ANCHOR: start at target employee (VP of Engineering)
    SELECT employee_id, full_name, manager_id, salary,
           0                    AS depth,
           ARRAY[employee_id]   AS visited_ids,
           full_name::TEXT      AS reporting_path
    FROM employees
    WHERE employee_id = 42

    UNION ALL

    -- RECURSIVE: find everyone reporting to previous level
    SELECT e.employee_id, e.full_name, e.manager_id, e.salary,
           oh.depth + 1,
           oh.visited_ids || e.employee_id,
           oh.reporting_path || ' -> ' || e.full_name
    FROM employees e
    INNER JOIN org_hierarchy oh ON e.manager_id = oh.employee_id
    -- CYCLE GUARD: stop if we've seen this employee before
    WHERE NOT (e.employee_id = ANY(oh.visited_ids))
)
SELECT depth,
       LPAD('', depth * 4, ' ') || full_name AS org_chart,
       salary, reporting_path
FROM org_hierarchy
ORDER BY depth, full_name;
Output
depth | org_chart | salary | reporting_path
0 | Dana Kimura | 185000.00 | Dana Kimura
1 | Alex Petrov | 140000.00 | Dana Kimura -> Alex Petrov
1 | Yemi Adeyinka | 138000.00 | Dana Kimura -> Yemi Adeyinka
2 | Carlos Mendes | 110000.00 | Dana Kimura -> Alex Petrov -> Carlos Mendes
Watch Out:
UNION ALL in a recursive CTE is not optional. Using UNION (without ALL) forces de-duplication on every iteration, which destroys performance on large hierarchies and breaks cycle detection with array tracking. Always use UNION ALL and handle de-duplication yourself if needed.
Production Insight
Cycles in hierarchical data are more common than you think — org chart imports, category trees, and bill-of-materials all get corrupted eventually.
PostgreSQL defaults to max recursion depth 100; SQL Server also defaults to 100 (configurable with OPTION MAXRECURSION).
Always add cycle detection in production — never assume the data is clean.
Key Takeaway
Recursive CTE = anchor member (base case) + UNION ALL + recursive member (self-reference).
Always add cycle detection — hierarchical data in production gets corrupted.
UNION ALL is mandatory — UNION breaks performance and cycle tracking both.

Recursive CTE for Graph Traversal — Shortest Path in a Directed Graph

Recursive CTEs are not limited to tree hierarchies; they can also traverse arbitrary directed graphs. A common production need is finding the shortest path between two nodes in a graph stored as an adjacency list (edges table). The challenge is that a graph can have multiple paths to the same node, and we want the shortest (fewest edges). A recursive CTE with breadth-first exploration naturally finds the shortest path because it visits nodes in order of increasing depth — the first time a target node is reached gives the minimal edge count.

To make this work efficiently: track visited nodes to avoid infinite loops (cycles) and store the path taken so far. Use an array to accumulate visited node IDs, and break when the target is found. Because the recursive CTE processes each level before moving to the next (as long as the database uses a BFS strategy, which PostgreSQL and SQL Server do for recursive CTEs), the first appearance of the target node will be the shortest path.

Performance note: on large graphs, recursive CTEs can be slow because they are not indexed for graph traversal. Use a LIMIT clause to stop early once the target is found, and add a depth bound to prevent runaway queries.

recursive_cte_shortest_path.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
-- Find shortest path from node A to node C in a directed graph
-- edges table: from_node, to_node

WITH RECURSIVE path_finder AS (
    -- ANCHOR: start at source node A
    SELECT
        'A'::TEXT AS current_node,
        ARRAY['A']::TEXT[] AS path,
        0 AS depth

    UNION ALL

    -- RECURSIVE: explore neighbors not yet visited
    SELECT
        e.to_node,
        pf.path || e.to_node,
        pf.depth + 1
    FROM path_finder pf
    JOIN edges e ON e.from_node = pf.current_node
    WHERE NOT e.to_node = ANY(pf.path)   -- avoid cycles
      AND pf.depth < 10                  -- safety limit
)
SELECT path, depth
FROM path_finder
WHERE current_node = 'C'
ORDER BY depth
LIMIT 1;
Output
path | depth
{A, B, C} | 2
Production Insight
For shortest path in production graphs with thousands of nodes, consider using a dedicated graph database (Neo4j, ArangoDB) or a PostgreSQL extension like pgRouting. Recursive CTEs work well for small graphs (under 10k nodes) but scale poorly.
Always add depth and cycle guards — production graph data almost always has unexpected cycles.
PostgreSQL 14+ supports SEARCH DEPTH FIRST and SEARCH BREADTH FIRST for more control over traversal order.
Key Takeaway
Recursive CTEs can find shortest paths in directed graphs by using BFS traversal with cycle detection.
First hit of the target node is the shortest path (fewest edges).
Use depth limits and visited-node tracking to prevent runaway queries.
BFS Shortest Path Search
edgeedgeedgeedgeABCD

CTE Materialization vs. Re-evaluation — The Performance Truth Nobody Tells You

The SQL standard says nothing about whether a CTE must be materialized (computed once, stored) or re-evaluated (re-run every time it's referenced). Each database engine makes its own choice.

PostgreSQL prior to version 12 always materialized CTEs as optimization fences — the planner could not push WHERE predicates from the outer query into the CTE. PostgreSQL 12+ inlines non-recursive CTEs by default, allowing predicate pushdown. You can force materialization with the MATERIALIZED keyword.

SQL Server never guaranteed materialization — it treats CTEs as macro expansions. If you reference a CTE three times in a UNION ALL, SQL Server may execute that query three times. For expensive CTEs referenced repeatedly, load results into a #temp_table first.

MySQL 8.0+ inlines CTEs. Oracle materializes when it determines it's beneficial and accepts the WITH MATERIALIZE hint to force it.

cte_materialization_patterns.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
-- PostgreSQL: force materialization to prevent volatile function re-execution
WITH MATERIALIZED daily_rates AS (
    SELECT currency_code,
           get_live_rate(currency_code) AS usd_rate  -- expensive external call
    FROM supported_currencies
    WHERE is_active = true
)
SELECT o.order_id, o.order_total * dr.usd_rate AS total_usd
FROM orders o
JOIN daily_rates dr ON dr.currency_code = o.currency_code
WHERE o.order_status = 'pending';

-- SQL Server ANTI-PATTERN: CTE referenced 3x may scan the table 3x
WITH region_totals AS (
    SELECT region_id, SUM(revenue) AS total_revenue
    FROM sales_transactions
    WHERE transaction_date >= '2024-01-01'
    GROUP BY region_id
)
SELECT 'Top' AS tier, * FROM region_totals WHERE total_revenue > 1000000
UNION ALL
SELECT 'Mid' AS tier, * FROM region_totals WHERE total_revenue BETWEEN 250000 AND 1000000
UNION ALL
SELECT 'Low' AS tier, * FROM region_totals WHERE total_revenue < 250000;
-- ^ May scan sales_transactions THREE TIMES

-- SQL Server CORRECT PATTERN: materialize into temp table
SELECT region_id, SUM(revenue) AS total_revenue
INTO #region_totals
FROM sales_transactions
WHERE transaction_date >= '2024-01-01'
GROUP BY region_id;

SELECT 'Top' AS tier, * FROM #region_totals WHERE total_revenue > 1000000
UNION ALL
SELECT 'Mid' AS tier, * FROM #region_totals WHERE total_revenue BETWEEN 250000 AND 1000000
UNION ALL
SELECT 'Low' AS tier, * FROM #region_totals WHERE total_revenue < 250000;

DROP TABLE #region_totals;
Output
-- SQL Server temp table approach: one scan, three results
tier | region_id | total_revenue
Top | 3 | 2845000.00
Top | 7 | 1120000.00
Mid | 1 | 780000.00
Low | 12 | 145000.00
Interview Gold:
Interviewers love asking 'Is a CTE always computed once?' The correct answer: it depends entirely on the database engine and version. PostgreSQL 12+ inlines by default. SQL Server treats CTEs as macro expansions and may re-evaluate them. Always verify with EXPLAIN/EXPLAIN ANALYZE.
Production Insight
PostgreSQL 11 and below: CTEs are optimization fences — WHERE predicates from the outer query cannot be pushed in. Upgrade to 12+ or rewrite as a derived table to allow pushdown.
SQL Server: any CTE referenced N times in UNION ALL is N separate scans of the base table. This is the most common CTE performance trap in SQL Server codebases.
Rule: expensive CTE + referenced >1 time in SQL Server = use a #temp table.
Key Takeaway
Materialization behavior is engine-specific and version-specific — never assume a CTE is computed once.
SQL Server: verify with SET STATISTICS IO ON before shipping any CTE-heavy query.
PostgreSQL 12+: inlined by default — use MATERIALIZED keyword to force fence behavior when needed.

Production CTE Patterns — Window Functions, Data Mutations and When Not to Use CTEs

CTEs compose beautifully with window functions because window function results can't be filtered in the same SELECT where they're computed — you need a wrapper, and a CTE is cleaner than a subquery for this.

CTEs also work with INSERT, UPDATE, DELETE and MERGE in PostgreSQL and SQL Server — called data-modifying CTEs. In PostgreSQL, data-modifying CTEs execute exactly once, and their effects are visible to the outer query but not to other CTEs in the same WITH clause.

Know when NOT to use a CTE. If your query is a single flat SELECT that runs fine as a subquery, wrapping it in a CTE adds no value. CTEs shine for multi-step logic, recursion, and avoiding repeated complex expressions. A CTE over a 50-million-row unindexed table is still a full table scan — CTEs don't fix missing indexes.

production_cte_patterns.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
-- PATTERN A: CTE + Window Function pipeline
-- Find products in bottom 10th percentile of sales velocity per category

WITH product_velocity AS (
    SELECT p.product_id, p.product_name, p.category,
           ROUND(SUM(oi.quantity)::NUMERIC /
                 NULLIF(CURRENT_DATE - p.launch_date, 0), 2) AS daily_velocity
    FROM products p
    LEFT JOIN order_items oi ON oi.product_id = p.product_id
    LEFT JOIN orders o ON o.order_id = oi.order_id AND o.order_status = 'completed'
    WHERE p.is_active = true AND p.launch_date < CURRENT_DATE - INTERVAL '30 days'
    GROUP BY p.product_id, p.product_name, p.category, p.launch_date
),
velocity_ranked AS (
    SELECT *,
           PERCENT_RANK() OVER (PARTITION BY category ORDER BY daily_velocity ASC)
               AS velocity_percentile
    FROM product_velocity
)
SELECT product_id, product_name, category, daily_velocity,
       ROUND(velocity_percentile * 100, 1) AS percentile,
       'CLEARANCE_REVIEW' AS action
FROM velocity_ranked
WHERE velocity_percentile <= 0.10
ORDER BY category, daily_velocity;

-- PATTERN B: Atomic archive-and-delete (PostgreSQL data-modifying CTE)
WITH to_archive AS (
    DELETE FROM orders
    WHERE order_status = 'completed'
      AND order_date < CURRENT_DATE - INTERVAL '2 years'
    RETURNING *
)
INSERT INTO orders_archive
SELECT *, CURRENT_TIMESTAMP AS archived_at
FROM to_archive
RETURNING order_id, archived_at;
Output
-- Pattern A:
product_name | category | daily_velocity | percentile | action
Retro Desk Lamp v2 | Home Office | 0.03 | 4.2 | CLEARANCE_REVIEW
-- Pattern B:
order_id | archived_at
10042 | 2024-11-15 09:55:03+00
10091 | 2024-11-15 09:55:03+00
Watch Out:
In PostgreSQL data-modifying CTEs, changes made by one CTE are NOT visible to sibling CTEs in the same WITH clause — only to the outer query. If CTE_A deletes rows and CTE_B tries to read those same rows, CTE_B will still see them. This is intentional isolation that trips up engineers expecting sequential execution semantics.
Production Insight
Window functions can't be filtered in the same SELECT they're computed — the CTE wrapper is not just stylistic, it's structurally necessary.
Data-modifying CTEs (PostgreSQL) give you atomic multi-step DML in one statement — DELETE + INSERT with RETURNING is cleaner than two separate statements in application code.
CTEs are not a substitute for indexes — identify what's slow with EXPLAIN before adding CTE layers.
Key Takeaway
CTE + window function is the canonical pattern for filtering on derived rank/percentile values.
Data-modifying CTEs in PostgreSQL are atomic — sibling CTEs see pre-modification state, the outer query sees post-modification state.
CTEs don't fix missing indexes — use EXPLAIN first, add indexes second, refactor to CTEs third.

CTE vs View vs Temporary Table — When to Use Which

Choosing between a CTE, a view, and a temporary table depends on scope, performance requirements, and reuse pattern. All three allow you to encapsulate a query, but they differ drastically in lifespan, materialization behavior, and DDL impact.

CTEs are ideal for single-query decomposition. They have no schema footprint, disappear immediately after the query finishes, and cannot be reused across separate statements. They are the best choice for breaking down complex logic within one query, especially when recursion is needed or when you want to chain multiple transformation steps.

Views are persistent schema objects that wrap a query. They can be referenced across multiple queries by multiple users. They are a security boundary and provide abstraction. However, they require DDL permissions (CREATE VIEW) and can become a maintenance burden if not managed. Views are best for shared business logic that must be used consistently across many reports or applications.

Temporary tables (prefixed with # in SQL Server, temp table in PostgreSQL) are explicitly materialized on disk (or in memory for small ones). They persist for the duration of the session (or transaction in some cases). They are the only option when you need to reuse an expensive intermediate result multiple times within a script or across multiple statements without re-executing the source query. They also allow indexes and statistics, giving you more control over query performance.

FeatureCTEViewTemp Table
ScopeSingle querySchema (persistent)Session/session scope
Reuse across queriesNoYesYes
MaterializationEngine-specificEngine-specificAlways materialized
Recursion supportYes (WITH RECURSIVE)NoNo
DDL permissionsNoneCREATE VIEWCREATE TABLE (usually allowed for temp tables)
Index supportNoYes (indexed views)Yes
Best forMulti-step single query, recursionShared logic across many queriesRepeated reuse of expensive result within a session

In production, the pattern is often: use CTEs for readability within a single query, use temp tables when you need to break that query into multiple steps with explicit materialization, and use views only when the same logic must be shared across teams or applications.

Production Rule of Thumb:
If you're writing a CTE and you find yourself referencing it more than once in SQL Server, switch to a temp table. If you need the same logic in five different stored procedures, create a view. If it's a one-off ad-hoc analysis, a CTE is perfect.
Production Insight
Temp tables give you explicit control over materialization and indexing — they are the only safe option for multi-reference patterns in SQL Server.
Views can become a performance black hole if they contain expensive joins and are used without adequate indexing — always check the execution plan.
CTEs and views are both subject to the database engine's materialization decisions; temp tables remove that ambiguity.
Key Takeaway
CTEs for single-query readability, temp tables for multi-statement reuse with performance control, views for shared schema-level abstraction.
Materialization is only guaranteed with temp tables — CTEs and views may be re-evaluated.

CTEs and Indexing — Why Your Recursive CTE Just Killed the Server and How to Fix It

Nobody tells you this until you page them at 3 AM. A recursive CTE looks innocent. It's not. The recursive member executes per row, not per batch. If your anchor set returns 10k rows and each recursion step does a full table scan, you're generating millions of logical reads before your monitoring even twitches.

The fix isn't 'add an index'. It's add the right index. A recursive CTE accessing a parent-child table needs a covering index on the join columns used in the recursive step. Without it, SQL Server (or Postgres, or MySQL) spools to tempdb for every iteration. That's where performance goes to die.

Test this yourself. Write the recursive CTE. Check the execution plan — look for 'Eager Spool' or 'Index Scan' on the recursive side. If you see either, you're one bad index away from a production incident. The index you need is composite: (parent_id, child_id) if your join is on parent_id. Include any columns you filter or return. That's it. No magic.

RecursiveIndexFixProduction.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
// io.thecodeforge — database tutorial

-- DO NOT deploy this: no index on manager_id
WITH OrgChart AS (
    SELECT employee_id, manager_id, 0 AS depth
    FROM employees
    WHERE manager_id IS NULL
    UNION ALL
    SELECT e.employee_id, e.manager_id, oc.depth + 1
    FROM employees e
    INNER JOIN OrgChart oc ON e.manager_id = oc.employee_id
)
SELECT * FROM OrgChart;

-- DO this: create the covering index first
CREATE INDEX IX_employees_manager_id_includes
ON employees (manager_id, employee_id)
INCLUDE (department_id, salary);

-- Now the recursive CTE hits an index seek, not a scan
WITH OrgChart AS (
    SELECT employee_id, manager_id, 0 AS depth
    FROM employees
    WHERE manager_id IS NULL
    UNION ALL
    SELECT e.employee_id, e.manager_id, oc.depth + 1
    FROM employees e
    INNER JOIN OrgChart oc ON e.manager_id = oc.employee_id
)
SELECT * FROM OrgChart;
Output
Before index: 1.2M logical reads, 45 seconds runtime
After index: 12k logical reads, 0.3 seconds runtime
Production Trap: Recursive Spool Death
A recursive CTE without a covering index on the join column will spool to tempdb per recursion level. That's not 'slightly slower' — it's how you get a 30-minute query that blocks your nightly ETL.
Key Takeaway
Index the recursive join column with INCLUDE for all output columns — before you deploy any recursive CTE to production.

CTE Side Effects — When They Don't Execute Exactly Once and Why That Breaks Your Report

Here's a fun one: you write a CTE that calls a function with side effects — maybe it logs access, updates a counter, or reads from a sequence. You reference the CTE twice in the outer query. SQL Server says 'I'll just inline this' and executes the function twice. Now you've double-logged, double-counted, or burned two sequence values. Good luck explaining that to the auditor.

This isn't a bug. It's by design. The optimizer decides whether to materialize a CTE or inline it based on cost estimates. You don't control it. If your CTE uses a non-deterministic function (GETDATE(), NEWID(), RAND(), any custom function with side effects), you get one execution per reference, not one execution total.

Fix: If you must have side effects exactly once, force materialization. Use a temp table. Or wrap the CTE in another CTE that adds TOP 100 PERCENT or ORDER BY a non-clustered column — some optimizers back off inlining when they see that. Better yet: don't put side-effect functions in CTEs at all. Push them to an explicit temp table. You lose the 'elegance' but gain correctness. Production rewards correctness.

CTESideEffectDoubleExecution.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// io.thecodeforge — database tutorial

-- Side-effect function called twice because CTE inlined
CREATE FUNCTION dbo.LogAccess(@user_id INT) RETURNS INT
AS BEGIN
    INSERT INTO access_log(user_id, accessed_at)
    VALUES (@user_id, GETDATE());
    RETURN @user_id;
END;

-- This CTE looks clean. It's a lie.
WITH UserCTE AS (
    SELECT user_id, dbo.LogAccess(user_id) AS logged
    FROM users
    WHERE active = 1
)
SELECT * FROM UserCTE WHERE user_id = 100
UNION ALL
SELECT * FROM UserCTE WHERE user_id = 200;
-- dbo.LogAccess executes 2 times per user, not 1

-- Fix: force materialization into temp table
SELECT user_id, dbo.LogAccess(user_id) AS logged
INTO #logged_users
FROM users
WHERE active = 1;

SELECT * FROM #logged_users WHERE user_id = 100
UNION ALL
SELECT * FROM #logged_users WHERE user_id = 200;
-- dbo.LogAccess executes exactly 1 time per user
Output
CTE approach: 4 log entries inserted (2 per user)
Temp table approach: 2 log entries inserted (1 per user) — correct
Senior Shortcut: Never Trust CTE Materialization
If your CTE calls a function that writes, reads a sequence, or uses GETDATE(), force it into a temp table. The optimizer will inline non-deterministic CTEs every time — and your side effects will double.
Key Takeaway
CTEs with non-deterministic functions execute per-reference, not once — use a temp table when correctness of side effects matters.

CTEs in WHERE and FROM Subqueries — The Nesting Trap That Kills Readability Fast

You can use a CTE inside a subquery. You can use a subquery inside a CTE. You can stack three CTEs in a WITH clause. The SQL standard lets you. Doesn't mean you should. I've seen a 'report query' with six CTEs, each referencing the prior one, then two CTEs used inside a HAVING clause. It took me 45 minutes to trace the data flow. Nobody could maintain it.

The real problem: CTEs don't show in the execution plan as named entities. When you nest them inside subqueries, the optimizer flattens everything into one giant blob. You lose the ability to isolate performance problems. A slow CTE inside a WHERE clause becomes invisible — you just see a big 'Filter' operator with no clue which part is killing you.

My rule: one level of CTE for readability. If you need intermediate results, write them to temp tables with appropriate indexes. Name them. Document them. Your future self (or the junior who inherits this) will thank you when the query needs tuning. You don't get points for 'elegant nesting' in production — you get points for being able to explain what your query does at 2 AM.

CTENestingAvoidThisProduction.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
// io.thecodeforge — database tutorial

-- Don't do this: CTE nested in a subquery
WITH ActiveUsers AS (
    SELECT user_id, last_login
    FROM users
    WHERE status = 'active'
)
SELECT *
FROM orders
WHERE user_id IN (
    SELECT user_id FROM ActiveUsers
    WHERE last_login > '2024-01-01'
);
-- This CTE is invisible in the execution plan as a separate step

-- Do this: materialize the intermediate result
SELECT user_id
INTO #active_recent_users
FROM users
WHERE status = 'active'
  AND last_login > '2024-01-01';

CREATE INDEX IX_temp_user_id ON #active_recent_users(user_id);

SELECT *
FROM orders
WHERE user_id IN (SELECT user_id FROM #active_recent_users);
-- Now you can isolate performance of the temp table population vs the main query
Output
Nested CTE plan: single unreadable blob with Filter, 800k reads
Temp table plan: two clear steps — 50k reads for population, 12k for the main query
Never Do This: CTE in a WHERE Clause
CTEs in subqueries get flattened into the outer plan. You lose all ability to isolate performance. If you need an intermediate result, materialize it — don't hide it in a CTE inside a WHERE clause.
Key Takeaway
One level of CTE nesting for readability; anything deeper becomes a temp table with an index so you can actually debug it.

Advanced SQL CTE Techniques — Chaining, Modifying Data, and Debugging Like a Pro

You've mastered the single CTE. Now chain them. A CTE can reference previous CTEs in the same WITH block. This isn't just neat — it lets you build transformations step by step without nesting subqueries into a nightmare. Each CTE becomes a named pipeline stage. You can inspect, debug, and reuse each stage independently. That's the whole point.

But here's where most devs miss the mark: CTEs can modify data. In PostgreSQL and SQL Server, you can wrap INSERT, UPDATE, or DELETE inside a CTE and target the result set directly. This is massive for batch operations — update rows based on a complex join, then insert a log entry in the same statement. One atomic operation. No race conditions.

Production tip: Chain CTEs but watch the optimizer. Some engines materialize each CTE, others inline them. If your chain is deep and your data is wide, you might end up reading the same table five times. Profile before you profile your career.

ChainedCteExample.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// io.thecodeforge — database tutorial

WITH high_earners AS (
    SELECT department_id, salary
    FROM employees
    WHERE salary > 100000
),
department_avg AS (
    SELECT department_id, AVG(salary) AS avg_salary
    FROM high_earners
    GROUP BY department_id
)
UPDATE employees e
SET bonus = e.salary * 0.15
FROM department_avg da
WHERE e.department_id = da.department_id
  AND e.salary > da.avg_salary;

SELECT * FROM employees WHERE bonus IS NOT NULL;
Output
employee_id | department_id | salary | bonus
-----------|--------------|---------|-------
42 | 7 | 150000 | 22500
87 | 3 | 120000 | 18000
Chaining Gotcha:
Not all databases allow CTEs in DML. Check your engine docs before you replace a transaction with a CTE chain.
Key Takeaway
Chain CTEs to build readable data pipelines; use them in DML for atomic batch updates.

Potential Issues or Limitations of CTEs in SQL — The Hidden Costs Nobody Warns You About

CTEs look clean but they hide landmines. First: no index hints. You can't force an index inside a CTE definition. If your recursive CTE walks a million-node tree and the join column isn't indexed, you're toast. The server will scan full tables every recursion level. That's not a bug — it's your poor schema design.

Second: CTEs are often materialized only once, but some engines (hello, SQL Server) might re-evaluate them multiple times inside the same query. If your CTE uses a volatile function like GETDATE() or RAND(), you'll get inconsistent results. Debugging that is a special kind of hell.

Third: recursion limits. MySQL defaults to 1000 recursion depth. PostgreSQL defaults to 100 and you have to explicitly SET session to raise it. Forget that, and your org chart query silently truncates at level 7. Nobody catches that until leadership asks why the CEO's reports are missing.

Fourth: CTEs can't be indexed, partitioned, or parallelized the way temp tables can. For large datasets, a temp table with proper indexes will blow past a CTE every time.

RecursiveCteDepthLimit.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// io.thecodeforge — database tutorial

WITH RECURSIVE org_tree AS (
    SELECT employee_id, manager_id, 1 AS depth
    FROM employees
    WHERE manager_id IS NULL
    UNION ALL
    SELECT e.employee_id, e.manager_id, ot.depth + 1
    FROM employees e
    INNER JOIN org_tree ot ON e.manager_id = ot.employee_id
)
SELECT * FROM org_tree
WHERE depth > 10;

-- In PostgreSQL, default max depth is 100. Errors at depth 101.
Output
ERROR: recursive query exceeded the maximum recursion depth (100)
Senior Shortcut:
Always set a recursion depth limit explicitly with OPTION (MAXRECURSION 0) in SQL Server, or SET max_recursive_iterations in PostgreSQL. Defaults are for prototypes, not production.
Key Takeaway
CTEs can't be indexed, risk re-evaluation, and hit recursion caps — use temp tables for large or volatile data.

CTEs in Azure Synapse Analytics — Why Your On-Premise Query Breaks in the Cloud

Azure Synapse Analytics (formerly SQL Data Warehouse) and Analytics Platform System (PDW) support CTEs but with critical restrictions that catch teams off guard. Unlike SQL Server, Synapse requires that every CTE be referenced at least once in the main query body—orphan CTEs silently fail. Recursive CTEs are outright banned because Synapse's MPP architecture cannot handle stack-based recursion across distribution nodes. Non-recursive CTEs behave as inline views, not materialized staging tables, so referencing the same CTE twice causes double execution and massive slowdowns. CTEs also cannot contain ORDER BY without TOP or OFFSET/FETCH, nor can they use OPTION clauses. The biggest trap: Synapse CTEs are limited to one SELECT statement—you cannot chain CTEs with UNION in the same WITH block. Always test CTE logic in a dedicated SQL pool before production deploy.

Synapse_CTE_Workaround.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// io.thecodeforge — database tutorial

-- This fails in Synapse: Recursive CTE not allowed
WITH OrgCTE AS (
    SELECT EmployeeID, ManagerID FROM Employees WHERE ManagerID IS NULL
    UNION ALL
    SELECT e.EmployeeID, e.ManagerID 
    FROM Employees e
    INNER JOIN OrgCTE o ON e.ManagerID = o.EmployeeID
)
SELECT * FROM OrgCTE;

-- Workaround: Use temp table + loop instead
CREATE TABLE #Org (EmployeeID INT, Level INT);
INSERT INTO #Org VALUES (1, 0);
WHILE @@ROWCOUNT > 0
    INSERT INTO #Org
    SELECT e.EmployeeID, o.Level + 1
    FROM Employees e
    INNER JOIN #Org o ON e.ManagerID = o.EmployeeID
    WHERE NOT EXISTS (SELECT 1 FROM #Org WHERE EmployeeID = e.EmployeeID);
Output
-- No recursive CTE output — substitute with WHILE loop
Production Trap:
Synapse CTEs appear to work in small tests but silently rerun for each reference. Always SELECT INTO a temp table first to materialize results.
Key Takeaway
Azure Synapse bans recursive CTEs and does not materialize non-recursive ones. Use temp tables for multi-reference CTEs.

Arguments and Usage Guidelines — How Databases Differ on CTE Syntax

Every major SQL dialect supports CTEs but with subtle argument differences that break portability. PostgreSQL and MySQL allow CTEs to have explicit column aliases: WITH cte (col1, col2) AS (SELECT ...). SQL Server and Oracle require this when subquery columns lack names (e.g., aggregate expressions). DB2 and Teradata enforce strict recursion depth limits via OPTION (MAXRECURSION n)—default is 100 in SQL Server, infinite in PostgreSQL unless you set max_recursive_iterations. The most ignored guideline: never use CTEs for singleton lookups. CTEs optimize poorly for single-row fetches because the optimizer scans the entire CTE definition before applying outer filters. Another hard rule—CTEs in subqueries or WHERE clauses force nested scopes that degrade readability faster than any other pattern. Always define CTEs at the top of the query, not inside derived tables. If you exceed 3 CTEs chained, extract the logic into a view.

CTE_Arguments_Comparison.sqlSQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// io.thecodeforge — database tutorial

-- PostgreSQL: Explicit column aliases required for aggregates
WITH dept_avg (department, avg_salary) AS (
    SELECT department, AVG(salary)
    FROM employees
    GROUP BY department
)
SELECT * FROM dept_avg WHERE avg_salary > 50000;

-- SQL Server: Same syntax, but default recursion limit 100
-- To override, use OPTION (MAXRECURSION 0) for unlimited
WITH numbers AS (
    SELECT 1 AS n
    UNION ALL
    SELECT n + 1 FROM numbers WHERE n < 1000
)
SELECT n FROM numbers
OPTION (MAXRECURSION 1000);

-- MySQL 8+: CTE columns must match subquery column count
Output
-- Returns departments where average salary exceeds $50,000
-- Numbers 1 through 1000 (or until limit is hit)
Production Trap:
Forgetting MAXRECURSION in SQL Server defaults to 100 iterations. A 101-level org chart silently truncates data.
Key Takeaway
Always specify column aliases for CTEs with aggregates. Know your database's recursion limit and override it explicitly.
● Production incidentPOST-MORTEMseverity: high

A CTE Referenced Three Times Scanned 50M Rows Three Times in SQL Server

Symptom
A monthly revenue tiering report ran for 45 seconds and occasionally timed out. The query used a CTE to aggregate sales_transactions, then referenced it in three UNION ALL branches for Top/Mid/Low tiers.
Assumption
The team assumed CTEs behaved like temp tables — computed once, reused. The query looked clean. No one thought to check whether SQL Server was re-executing the CTE.
Root cause
SQL Server treats CTEs as logical macro expansions by default. Each of the three UNION ALL branches caused the engine to re-execute the CTE's GROUP BY aggregation over 50 million rows. SET STATISTICS IO ON showed 150 million logical reads — three full scans of the same table.
Fix
Materialized the CTE result into a #temp_table first, then ran the three UNION ALL branches against it. Query time dropped from 45 seconds to 1.2 seconds. Logical reads dropped from 150M to 50M.
Key lesson
  • SQL Server CTEs are not cached — if referenced N times, the underlying query may execute N times
  • Use SET STATISTICS IO ON to count logical reads before assuming a query is efficient
  • For expensive CTEs referenced more than once in SQL Server, always use a #temp table
Production debug guideDiagnosing slow queries that use CTEs4 entries
Symptom · 01
CTE-based query is slower than expected despite looking clean
Fix
Check if the CTE is referenced multiple times in the outer query. In SQL Server run SET STATISTICS IO ON and count logical reads. If reads = N × expected, the CTE is re-executing. Move expensive CTEs into #temp tables.
Symptom · 02
Recursive CTE hits maximum recursion depth error
Fix
Your data likely has a cycle (A is manager of B, B is manager of A). Add cycle detection: track visited IDs in an ARRAY (PostgreSQL) or add a depth counter with WHERE depth < 100. In SQL Server use OPTION (MAXRECURSION 0) only as a last resort — fix the data instead.
Symptom · 03
PostgreSQL CTE not benefiting from index on outer WHERE clause
Fix
PostgreSQL 11 and earlier treats CTEs as optimization fences — predicates from the outer query cannot be pushed into the CTE. Upgrade to PostgreSQL 12+ where CTEs are inlined by default, or rewrite as a subquery in FROM to allow predicate pushdown.
Symptom · 04
Data-modifying CTE deleted rows not visible to a sibling CTE
Fix
This is by design in PostgreSQL. Sibling CTEs in the same WITH clause cannot see each other's modifications — only the outer query can. Restructure so the outer query reads from the modifying CTE, or use two separate statements in a transaction.
AspectCTE (WITH clause)Derived Table (Subquery in FROM)Temporary TableView
ReadabilityExcellent — named, reusable within queryPoor — anonymous, nestedGood — namedGood — named and shared
Reuse within queryYes — reference same CTE multiple timesNo — must repeat the subqueryYesYes
RecursionYes — WITH RECURSIVENoNoNo
Schema footprintNone — query-scoped onlyNoneRequires CREATE/DROPPermanent schema object
MaterializationEngine-specific — may re-evaluateAlways inlinedAlways materialized — explicitEngine-specific
Best forMulti-step reads, recursion, one-time useSimple single-use subqueriesExpensive results reused many timesShared logic across many queries

Key takeaways

1
CTEs are query-scoped named result sets
no schema changes needed, live only for one query execution, and are the cleanest way to decompose multi-step SQL into readable stages.
2
Recursive CTEs require UNION ALL (not UNION), always need a termination condition, and should include cycle-detection logic for real-world hierarchical data.
3
CTE materialization is engine-specific
PostgreSQL 12+ inlines by default, SQL Server may re-execute. Verify with EXPLAIN and use temp tables for expensive CTEs referenced multiple times in SQL Server.
4
Data-modifying CTEs in PostgreSQL enable atomic DELETE + INSERT in one transaction
but sibling CTEs cannot see each other's changes, only the outer query can.

Common mistakes to avoid

3 patterns
×

Referencing a CTE before it is defined in the WITH clause

Symptom
'relation does not exist' or 'invalid object name' error even though the CTE is present in the query
Fix
CTEs must be defined in dependency order. A CTE can only reference CTEs that appear earlier in the same WITH clause. Reorder your CTEs so dependencies come first.
×

Assuming a CTE is computed once when referenced multiple times in SQL Server

Symptom
Mysteriously slow queries where EXPLAIN shows the expensive base table scanned multiple times — one scan per reference
Fix
In SQL Server, CTEs referenced more than once may be re-evaluated each time. Run SET STATISTICS IO ON and check logical reads. For expensive CTEs referenced multiple times, materialize into a #temp_table first.
×

Writing a recursive CTE without cycle detection on potentially dirty hierarchical data

Symptom
Query throws 'maximum recursion depth exceeded' error — happens only in production where data has referential corruption not present in test data
Fix
Always track visited node IDs in an ARRAY (PostgreSQL) or add a depth counter with WHERE depth < N. Never assume hierarchical data is clean — code defensively against cycles.
INTERVIEW PREP · PRACTICE MODE

Interview Questions on This Topic

Q01SENIOR
What is the difference between a CTE and a subquery, and when would you ...
Q02SENIOR
Explain how a recursive CTE works — what are the anchor member and recur...
Q03SENIOR
If you reference the same CTE three times in a SQL Server UNION ALL, how...
Q01 of 03SENIOR

What is the difference between a CTE and a subquery, and when would you choose one in production?

ANSWER
Both produce a named or anonymous result set within a query. The key differences are readability, debuggability, and recursion support. A CTE is named and can be referenced multiple times in the outer query; a subquery in FROM is anonymous and must be re-written if reused. CTEs enable recursive queries (WITH RECURSIVE) which subqueries cannot. For debugging, you can temporarily SELECT from a named CTE stage to inspect intermediate results — impossible with nested subqueries. Choose a CTE when the logic has multiple steps, when you need recursion, or when the same intermediate result is referenced more than once. Choose a subquery for simple single-use filters where the extra WITH clause adds no clarity.
FAQ · 3 QUESTIONS

Frequently Asked Questions

01
Does a SQL CTE improve query performance?
02
Can you use a CTE in an UPDATE or DELETE statement?
03
What is the difference between WITH RECURSIVE and a regular CTE?
N
Naren Founder & Principal Engineer

20+ years shipping high-throughput database systems. Notes here come from systems that actually shipped.

Follow
Verified
production tested
May 23, 2026
last updated
1,554
articles · all by Naren
🔥

That's SQL Advanced. Mark it forged?

15 min read · try the examples if you haven't

Previous
SQL Window Functions
7 / 16 · SQL Advanced
Next
SQL Query Optimisation