Home DSA LU Decomposition — Factoring Matrices for Efficient Solving

LU Decomposition — Factoring Matrices for Efficient Solving

Where developers are forged. · Structured learning · Free forever.
📍 Part of: Linear Algebra → Topic 3 of 5
Learn LU decomposition — how Gaussian elimination produces L and U factors, why it enables O(n²) solving for multiple right-hand sides, and its role in determinant computation.
⚙️ Intermediate — basic DSA knowledge assumed
In this tutorial, you'll learn:
  • LU: A = LU where L is lower triangular (1s on diagonal), U is upper triangular. O(n³) to compute.
  • Once factored, solving Ax=b = two O(n²) triangular substitutions: Ly=b, Ux=y.
  • Determinant: det(A) = product of U's diagonal elements — O(n²) once LU is computed.
✦ Plain-English analogy ✦ Real code with output ✦ Interview questions
⚡ Quick Answer
LU decomposition factors a matrix A into a lower triangular matrix L and upper triangular matrix U such that A = LU. Once you have L and U, solving Ax = b for any b takes only O(n²) — just two triangular substitutions. If you need to solve the same system for 1000 different right-hand sides, LU decomposition pays for itself immediately.

LU decomposition is Gaussian elimination reorganised as a matrix factorisation. Where Gaussian elimination solves one system Ax=b, LU decomposition produces L and U factors that enable solving any number of systems with the same A matrix in O(n²) each — versus O(n³) per system without decomposition.

LU decomposition is used for: solving linear systems (factorize A once, solve for many b), computing matrix determinants (det(A) = det(L)×det(U) = product of diagonal elements), and computing matrix inverses (though np.linalg.solve is preferred). NumPy and SciPy use LAPACK's DGETRF for LU factorisation.

LU Decomposition Algorithm

lu_decomp.py · PYTHON
1234567891011121314151617181920212223242526272829303132333435
def lu_decompose(A):
    """LU decomposition without pivoting. A = L @ U."""
    n = len(A)
    L = [[1.0 if i==j else 0.0 for j in range(n)] for i in range(n)]
    U = [row[:] for row in A]
    for col in range(n):
        for row in range(col+1, n):
            if abs(U[col][col]) < 1e-12:
                raise ValueError('Singular matrix — use pivoting')
            factor = U[row][col] / U[col][col]
            L[row][col] = factor
            for j in range(col, n):
                U[row][j] -= factor * U[col][j]
    return L, U

def forward_sub(L, b):
    n = len(b)
    y = [0.0] * n
    for i in range(n):
        y[i] = b[i] - sum(L[i][j]*y[j] for j in range(i))
    return y

def back_sub(U, y):
    n = len(y)
    x = [0.0] * n
    for i in range(n-1, -1, -1):
        x[i] = (y[i] - sum(U[i][j]*x[j] for j in range(i+1,n))) / U[i][i]
    return x

A = [[2,1,-1],[4,3,1],[2,-1,3]]
L, U = lu_decompose(A)
b = [8,13,5]
y = forward_sub(L, b)   # Solve Ly = b
x = back_sub(U, y)       # Solve Ux = y
print(f'Solution: {[round(v,4) for v in x]}')
▶ Output
Solution: [2.0, 3.0, -1.0]

Determinant via LU

det_lu.py · PYTHON
123456789101112
from functools import reduce
from operator import mul

def determinant_lu(A):
    """det(A) = product of U's diagonal elements."""
    _, U = lu_decompose(A)
    return reduce(mul, (U[i][i] for i in range(len(U))), 1.0)

import numpy as np
A = [[2,1,-1],[4,3,1],[2,-1,3]]
print(f'det via LU: {determinant_lu(A):.4f}')
print(f'numpy det:  {np.linalg.det(A):.4f}')
▶ Output
det via LU: 8.0000
numpy det: 8.0000

🎯 Key Takeaways

  • LU: A = LU where L is lower triangular (1s on diagonal), U is upper triangular. O(n³) to compute.
  • Once factored, solving Ax=b = two O(n²) triangular substitutions: Ly=b, Ux=y.
  • Determinant: det(A) = product of U's diagonal elements — O(n²) once LU is computed.
  • With pivoting: PA = LU where P is a permutation matrix. Required for numerical stability.
  • In practice: scipy.linalg.lu for decomposition, scipy.linalg.solve for solving — never roll your own in production.

Interview Questions on This Topic

  • QHow does LU decomposition improve over repeated Gaussian elimination for multiple right-hand sides?
  • QHow do you compute the determinant using LU?
  • QWhy is pivoting required in LU decomposition?

Frequently Asked Questions

When is LU better than Cholesky decomposition?

Cholesky (A = LL^T) requires A to be symmetric positive definite — but is twice as fast as LU and numerically more stable when applicable. Use Cholesky for covariance matrices, positive definite systems (FEM, Gaussian processes), and normal equations in least squares. Use LU for general square systems.

🔥
Naren Founder & Author

Developer and founder of TheCodeForge. I built this site because I was tired of tutorials that explain what to type without explaining why it works. Every article here is written to make concepts actually click.

← PreviousGaussian Elimination — Solving Linear SystemsNext →Eigenvalues and Eigenvectors
Forged with 🔥 at TheCodeForge.io — Where Developers Are Forged