LU Decomposition — Factoring Matrices for Efficient Solving
- LU: A = LU where L is lower triangular (1s on diagonal), U is upper triangular. O(n³) to compute.
- Once factored, solving Ax=b = two O(n²) triangular substitutions: Ly=b, Ux=y.
- Determinant: det(A) = product of U's diagonal elements — O(n²) once LU is computed.
LU decomposition is Gaussian elimination reorganised as a matrix factorisation. Where Gaussian elimination solves one system Ax=b, LU decomposition produces L and U factors that enable solving any number of systems with the same A matrix in O(n²) each — versus O(n³) per system without decomposition.
LU decomposition is used for: solving linear systems (factorize A once, solve for many b), computing matrix determinants (det(A) = det(L)×det(U) = product of diagonal elements), and computing matrix inverses (though np.linalg.solve is preferred). NumPy and SciPy use LAPACK's DGETRF for LU factorisation.
LU Decomposition Algorithm
def lu_decompose(A): """LU decomposition without pivoting. A = L @ U.""" n = len(A) L = [[1.0 if i==j else 0.0 for j in range(n)] for i in range(n)] U = [row[:] for row in A] for col in range(n): for row in range(col+1, n): if abs(U[col][col]) < 1e-12: raise ValueError('Singular matrix — use pivoting') factor = U[row][col] / U[col][col] L[row][col] = factor for j in range(col, n): U[row][j] -= factor * U[col][j] return L, U def forward_sub(L, b): n = len(b) y = [0.0] * n for i in range(n): y[i] = b[i] - sum(L[i][j]*y[j] for j in range(i)) return y def back_sub(U, y): n = len(y) x = [0.0] * n for i in range(n-1, -1, -1): x[i] = (y[i] - sum(U[i][j]*x[j] for j in range(i+1,n))) / U[i][i] return x A = [[2,1,-1],[4,3,1],[2,-1,3]] L, U = lu_decompose(A) b = [8,13,5] y = forward_sub(L, b) # Solve Ly = b x = back_sub(U, y) # Solve Ux = y print(f'Solution: {[round(v,4) for v in x]}')
Determinant via LU
from functools import reduce from operator import mul def determinant_lu(A): """det(A) = product of U's diagonal elements.""" _, U = lu_decompose(A) return reduce(mul, (U[i][i] for i in range(len(U))), 1.0) import numpy as np A = [[2,1,-1],[4,3,1],[2,-1,3]] print(f'det via LU: {determinant_lu(A):.4f}') print(f'numpy det: {np.linalg.det(A):.4f}')
numpy det: 8.0000
🎯 Key Takeaways
- LU: A = LU where L is lower triangular (1s on diagonal), U is upper triangular. O(n³) to compute.
- Once factored, solving Ax=b = two O(n²) triangular substitutions: Ly=b, Ux=y.
- Determinant: det(A) = product of U's diagonal elements — O(n²) once LU is computed.
- With pivoting: PA = LU where P is a permutation matrix. Required for numerical stability.
- In practice: scipy.linalg.lu for decomposition, scipy.linalg.solve for solving — never roll your own in production.
Interview Questions on This Topic
- QHow does LU decomposition improve over repeated Gaussian elimination for multiple right-hand sides?
- QHow do you compute the determinant using LU?
- QWhy is pivoting required in LU decomposition?
Frequently Asked Questions
When is LU better than Cholesky decomposition?
Cholesky (A = LL^T) requires A to be symmetric positive definite — but is twice as fast as LU and numerically more stable when applicable. Use Cholesky for covariance matrices, positive definite systems (FEM, Gaussian processes), and normal equations in least squares. Use LU for general square systems.
Developer and founder of TheCodeForge. I built this site because I was tired of tutorials that explain what to type without explaining why it works. Every article here is written to make concepts actually click.