Counting Sort & Radix Sort — Negative Number Pitfalls
Java's modulus returns negative remainders, crashing Radix Sort on negative inputs.
20+ years shipping performance-critical code where algorithms decide the bill. Drawn from code that ran under real load.
- ✓Solid grasp of fundamentals
- ✓Comfortable reading code examples
- ✓Basic production concepts
- Counting Sort and Radix Sort break O(n log n) by avoiding comparisons entirely.
- Counting Sort works for small integer ranges (k << n), placing each element directly.
- Radix Sort processes digits one at a time, using stable Counting Sort on each digit.
- Base 256 Radix Sort sorts 32-bit integers in 4 passes, outperforming base 10's 10 passes.
- Production risk: Counting Sort's memory blows up if value range is huge; Radix Sort fails silently with negative numbers.
Imagine you're sorting a massive pile of exam papers by grade (A, B, C, D, F). Instead of comparing papers against each other one by one, you just put each paper straight into the matching labelled box — then read out the boxes in order. That's Counting Sort. Radix Sort is like doing that same trick multiple times: first sort by the last digit, then the tens digit, then the hundreds digit, and suddenly the whole list is sorted without ever comparing two numbers head-to-head.
Every developer learns the classic sorting algorithms — QuickSort, MergeSort, HeapSort — and accepts the O(n log n) ceiling as an immovable law of nature. But it isn't a law. It's only the floor for comparison-based sorting. When your data has a bounded range — like ages, zip codes, exam scores, or pixel colour values — you can blow past that ceiling entirely and sort in O(n) time. That's not a theoretical curiosity; it's a genuine production win when you're processing millions of records.
Counting Sort & Radix Sort — The Non-Comparison Sorting Duo
Counting sort and radix sort are integer sorting algorithms that bypass the Ω(n log n) lower bound of comparison-based sorts by exploiting key structure. Counting sort works by tallying occurrences of each distinct key value, then computing prefix sums to determine final positions — it runs in O(n + k) time and O(k) space, where k is the range of input values. Radix sort extends this by sorting numbers digit by digit, typically using counting sort as its stable subroutine, achieving O(d * (n + b)) where d is digit count and b is the base.
In practice, counting sort is only efficient when k is not significantly larger than n — for example, sorting 1 million 32-bit integers (k = 2^32) would require 4 GB of auxiliary memory, making it impractical. Radix sort avoids this by processing digits, but its performance depends on digit width and base choice; a base of 256 (byte-wise) is common because it aligns with memory hierarchy and reduces passes. Both algorithms are stable, which is critical when sorting by multiple keys.
Use counting sort when the key range is small and known — e.g., sorting ages in a census (0–120) or grades (A–F). Use radix sort for fixed-width integers or strings of equal length, like sorting IP addresses or 64-bit timestamps in a log processing pipeline. These algorithms shine in systems where predictable, linear-time performance matters more than in-place memory — think database sort operators or network packet classification.
Why Comparison-Based Sorts Hit a Wall at O(n log n)
Before we build anything, you need to understand the problem these algorithms solve. Every comparison-based sort — QuickSort, MergeSort, you name it — works by asking 'is A greater than B?' over and over. Information theory proves you need at least log₂(n!) comparisons to sort n items, which simplifies to O(n log n). You simply cannot do better if comparisons are your only tool.
The key insight behind Counting Sort and Radix Sort is that they don't compare elements against each other at all. They exploit extra information: the range of possible values. If you know all your values sit between 0 and 999, you can use that constraint to skip comparisons entirely and place each element directly into its correct position.
This is the fundamental trade-off: you exchange time for domain knowledge. That trade is fantastic when sorting integers in a known range, but it completely breaks down for arbitrary objects, floats, or unbounded data. Knowing when the trade is worth it is what separates a senior engineer from someone who just memorised an algorithm.
Counting Sort Deep Dive — The Stable, Prefix-Sum Version You Actually Need
The simple version of Counting Sort shown above works for integers but loses stability — equal elements might not preserve their original relative order. That matters the moment you're sorting objects (like Student records by grade). The production-grade version uses a prefix-sum trick to guarantee stability, and it's what feeds directly into Radix Sort.
Here's the three-pass approach: first, count frequencies as before. Second, convert the frequency array into a prefix-sum array — each position now stores the starting output index for that value. Third, iterate the original input left-to-right, placing each element at the index the prefix-sum array points to, then increment that pointer.
Why left-to-right? Because iterating in order preserves the relative sequence of equal elements, making the sort stable. This stability is non-negotiable for Radix Sort to work correctly — so understand it here and Radix Sort becomes trivial.
Radix Sort — Sorting Digit by Digit Using Counting Sort as the Engine
Radix Sort answers a practical question: what do you do when values are too large for Counting Sort alone? Sorting 10 million phone numbers directly with Counting Sort would need a 10-billion-slot array. Radix Sort sidesteps this by sorting one digit at a time — always using a small, bounded range (0-9 for decimal digits).
The critical rule: always sort from the least significant digit to the most significant digit (LSD Radix Sort). This feels backwards, but it works because each pass is stable. After sorting by units, then tens, then hundreds, the previous passes are preserved by stability, and the array ends up fully sorted.
Think of it like sorting a deck of cards with two-digit numbers. Round 1: sort into 10 piles by the right digit, stack them in order. Round 2: sort those into 10 piles by the left digit, stack them. Done. The key is that Round 1's order is still respected inside each pile from Round 2.
Time complexity is O(d × (n + k)) where d is the number of digits and k is the digit base (10 for decimal). For fixed-length keys like phone numbers or zip codes, d is constant, so this collapses to O(n).
Choosing Between Counting Sort, Radix Sort, and Comparison Sorts
You now know three families of sorts. When should you use each? The decision depends on two dimensions: the size of the data (n) and the range of values (k or number of digits d).
Counting Sort wins when k is small — roughly k <= n and k <= 10^6 for practical memory. For exam scores (0-100), it's unbeatable. Radix Sort wins when k is large but the keys are fixed-width (e.g., 32-bit integers, IPv4 addresses). Comparison sorts win when you need generic sorting of arbitrary objects, floats, or when k is unbounded.
- Can you map your data to integers in a known range? -> Non-comparison candidate.
- Is k small (<= n)? Use Counting Sort.
- Is k large but keys have fixed width? Use Radix Sort (base 256 for performance).
- Otherwise, use a comparison sort (QuickSort, MergeSort, or Java's
Arrays.sort()).
Memory also matters: Counting Sort uses O(n+k) and can blow up. Radix Sort uses O(n+b) per pass where b is base (256). Comparison sorts typically use O(n) or O(log n) extra space.
- Counting Sort budget: k slots. If k = 1B, that's 4 GB just for the count array.
- Radix Sort budget: base slots (256 for byte) per pass, plus n for output.
- Comparison sort budget: O(n) auxiliary space for MergeSort, O(log n) for QuickSort.
- Rule of thumb: if k > 10*n, Radix Sort or comparison sort is better.
Gotchas, Real-World Use Cases and When NOT to Use These Sorts
Counting Sort shines for tightly bounded integer data: ages (0-120), star ratings (1-5), HTTP status codes (100-599), pixel channel values (0-255). It's used in image processing pipelines where sorting millions of pixel brightness values per frame is a real workload. The moment k (range) grows much larger than n (count), you're wasting memory and time.
Radix Sort is the go-to for fixed-width keys: IP addresses, phone numbers, postal codes, database primary keys within a known range, and sorting strings of equal length lexicographically. Some of the fastest network packet classifiers in the world use Radix Sort variants. It's also used in suffix array construction, a building block for search engines and genome sequencing tools.
Neither algorithm is suitable for: floating-point numbers without special handling, strings of variable length without padding, or any data without a natural numeric mapping. For those, stick to comparison sorts. The rule of thumb: if you can describe your key space as 'integers between X and Y', you have a candidate for linear sorting.
C++ and Python Implementations: Language-Specific Idioms and Performance
While Java's verbosity makes the algorithm explicit, C++ and Python offer different trade-offs. C++ allows in-place arrays and high-performance bit manipulation, while Python's list comprehensions and built-in sorting can make implementations concise but slower due to interpreter overhead.
Below are stable Counting Sort and LSD Radix Sort implementations in both languages. Notice how C++ uses std::vector and manual loops, while Python leverages list operations and the // integer division operator. The core logic (prefix-sum, right-to-left placement) remains identical across languages.
np.sort() or C-extensions. The Python implementation above is clean but not optimised — expect 10-100x slowdown compared to C++ or Java for arrays over 100k elements.Digit-by-Digit Walkthrough: Radix Sort on [329, 457, 657, 839, 436, 720, 355]
Let's trace Radix Sort on a small array of three-digit numbers. The algorithm processes units, then tens, then hundreds.
Initial array: [329, 457, 657, 839, 436, 720, 355]
Pass 1 (units digit): - Group by last digit: 9→[329,839], 7→[457,657], 6→[436], 0→[720], 5→[355]. - After stable counting sort: [720, 329, 839, 436, 355, 457, 657] (units sorted ascending).
Pass 2 (tens digit): - Group by middle digit: 2→[720, 329, 839], 3→[436, 355], 5→[457, 657]. - After stable sort: [720, 329, 839, 436, 355, 457, 657] (tens sorted; notice 720 and 329 both have tens=2 but original order preserved).
Pass 3 (hundreds digit): - Group by first digit: 7→[720], 3→[329, 355], 8→[839], 4→[436, 457], 6→[657]. - After stable sort: [329, 355, 436, 457, 657, 720, 839].
The array is now fully sorted. Each pass refines the order without ever comparing two numbers directly.
DEBUG flag that logs the array after each pass. It helps catch stability bugs early (if equal digits are reordered, the final sort will be corrupt).Advantages and Disadvantages of Counting Sort and Radix Sort
Both algorithms trade generality for speed. Here's a succinct comparison:
| Algorithm | Advantages | Disadvantages |
|---|---|---|
| Counting Sort | – O(n + k) time (linear when k small) | |
| Radix Sort | – O(d(n + b)) time (linear for fixed-width) |
Both are unsuitable for strings of varying length or floating-point data without transformation.
LSD vs MSD Radix Sort: When to Use Each Variant
LSD (Least Significant Digit) Radix Sort processes digits from rightmost to leftmost. It is the simpler, iterative approach: exactly d passes for d-digit numbers. It works only for fixed-width keys and relies on the sub-sort being stable. Because it never creates recursive branches, it's cache-friendly and easy to parallelise per pass.
MSD (Most Significant Digit) Radix Sort sorts by the most significant digit first, then recursively sorts each bucket. It can handle variable-length keys (e.g., strings) because you can stop recursion when a bucket has size 1. However, recursion overhead is heavy, and it is generally slower for fixed-width numeric keys. MSD is often used in suffix array construction and string sorting where keys have different lengths.
Rule of thumb: Use LSD for fixed-width integers; use MSD for variable-length strings or when early termination can prune large subproblems.
Real-World Applications: Suffix Arrays, String Sorting, Packet Routing
Radix Sort appears in three critical domains:
- Suffix arrays – Genome sequencing and text indexing construct suffix arrays by sorting all suffixes of a string. Since suffixes are strings of equal length (up to n), LSD Radix Sort on the last character, then second-last, etc., builds the suffix array in O(n log n) but often faster in practice due to cache efficiency. This is the basis of tools like
samtoolsandBowtie. - String sorting – When sorting fixed-length strings (e.g., licence plate numbers, telephone area codes), Radix Sort outperforms comparison sorts by a wide margin. Some databases use it internally for columnar storage indexes.
- Packet routing – Network routers classify packets by IP prefix (32-bit or 128-bit). Radix Sort on the binary representation allows O(n) sorting of routing table entries, enabling fast longest-prefix-match lookups. Cisco's IOS has used radix-like algorithms for decades.
These applications exploit the same core idea: transform the problem into sorting integers, then let Radix Sort's speed do the heavy lifting.
Complexity Analysis — Why Radix Sort Beats O(n log n) but Comes With Strings Attached
Radix sort's time complexity isn't just one number — it depends on the number of digits in your keys. For an array of n integers, each with d digits, and a base k (typically 10), the complexity is O(d * (n + k)). Counting sort runs in O(n + k) per digit pass, and we run d passes.
Here's the thing: if d is small and fixed (like 32-bit integers, d = 10 in base 10), then Radix sort is effectively O(n). That's why it wrecks comparison sorts in benchmarks on large integer datasets. But the catch is space: counting sort allocates an auxiliary array of size k for each pass, so total space is O(n + k). For base 10, that's trivial. If you naively use base 256 to reduce passes, your aux array grows to 256 — still fine. But if your keys are 64-bit floats or variable-length strings, d can blow up, and suddenly those n log n comparison sorts start looking good again.
Best case: O(n) with fixed-width keys. Worst case: O(n * d) where d grows — like sorting UUIDs as strings. Then you're paying for the digits, not the array size.
Pseudocode — The Algorithm Without the Language-Specific Cruft
Before you write a single line of C++ or Java, you need the skeleton. Radix sort is deceptively simple: it's just Counting Sort wrapped in a loop over digits. The base determines how many buckets Counting Sort uses. LSD Radix sort iterates from least significant digit to most — that's the stable version that works on integers.
Here's the pseudocode that survives code reviews: find max value to determine number of digits, then for each digit position (starting from 1s place), call Counting Sort on that digit. Counting Sort must be stable — meaning it preserves the relative order of elements with equal digits. That stability is what makes Radix sort work: each pass builds on the previous one.
The loop invariant? After sorting by the i-th least significant digit, the array is sorted by those i digits. Think of it like sorting dates: first by day, then month, then year — each pass refines the order.
Radix Sort Breaks on Negative Numbers in Production Log Analyzer
getDigit(-10, 0) returns 0 instead of 0, but the issue is that the digit extraction algorithm (number / 10^k) % 10 on negative numbers can produce negative digits, which index into the count array with base 10 causing out-of-bounds.abs(min) before sorting and subtract back after sorting. This ensures all values are non-negative during Radix Sort passes.- Non-comparison sorts assume input domain is non-negative integers. Always validate or normalise input.
- Add a precondition check:
if (any value < 0) throw new IllegalArgumentExceptionor apply the shift transform. - Unit tests must include negative numbers and edge cases like Integer.MIN_VALUE.
Arrays.parallelSort() if comparison sort is acceptable.| File | Command / Code | Purpose |
|---|---|---|
| WhyLinearSortingMatters.java | public class WhyLinearSortingMatters { | Why Comparison-Based Sorts Hit a Wall at O(n log n) |
| StableCountingSort.java | public class StableCountingSort { | Counting Sort Deep Dive |
| RadixSortPhoneNumbers.java | public class RadixSortPhoneNumbers { | Radix Sort |
| SortDecisionExample.java | public class SortDecisionExample { | Choosing Between Counting Sort, Radix Sort, and Comparison S |
| RadixSortIPAddresses.java | /** | Gotchas, Real-World Use Cases and When NOT to Use These Sort |
| counting_radix_py.py | def counting_sort(arr, max_val): | C++ and Python Implementations |
| RadixTimeAnalysis.java | public class RadixTimeAnalysis { | Complexity Analysis |
| RadixPseudo.java | function radixSort(int[] arr): | Pseudocode |
Key takeaways
Practice These on LeetCode
Interview Questions on This Topic
Why is Counting Sort said to break the O(n log n) lower bound, and does that mean it's always faster than QuickSort? Walk me through the exact conditions where Counting Sort wins and where it loses.
Frequently Asked Questions
20+ years shipping performance-critical code where algorithms decide the bill. Drawn from code that ran under real load.
That's Sorting. Mark it forged?
8 min read · try the examples if you haven't