⚙️AlgorithmIntermediate

Sorting Algorithms

Key Points

  • Sorting arranges items into a chosen order so that searching, grouping, and further algorithms become faster and simpler.
  • Comparison-based sorts like merge sort, quicksort, and heapsort have a theoretical lower bound of O(n n) comparisons.
  • Merge sort is stable and predictable O(n n) time but needs O(n) extra memory for merging.
  • Quicksort is in-place and usually fastest in practice with O(n n) average time, but it can degrade to O() without care.
  • Heapsort is in-place with O(n n) worst-case time but is not stable and often slower in practice than quicksort.
  • Counting, radix, and bucket sorts can achieve O(n) time when keys have a restricted form (small range, fixed digits, or uniform distribution).
  • C++ std::sort uses introsort, a hybrid that combines quicksort, heapsort, and insertion sort to guarantee O(n n) worst-case time.
  • Choosing the right sort depends on stability needs, memory limits, key properties, and input patterns (e.g., many duplicates, nearly sorted).

Prerequisites

  • Asymptotic notation (Big-O, Θ, Ω)To analyze and compare time and space costs of different sorting algorithms.
  • Recursion and divide-and-conquerMerge sort and quicksort are naturally expressed recursively and analyzed via recurrences.
  • Arrays, vectors, and memory layoutMost sorting implementations manipulate contiguous arrays and benefit from understanding cache effects and indexing.
  • Heaps and priority queuesHeapsort and partial sorting operations rely on heap properties and operations.
  • Randomization basicsRandomized pivot selection in quicksort provides average-case guarantees and avoids adversarial inputs.
  • Counting and discrete mathDecision-tree lower bounds and counting sort rely on combinatorics and frequency counting.

Detailed Explanation

Tap terms for definitions

01Overview

Sorting is the task of arranging a collection of items (numbers, strings, records) into a desired order such as ascending or descending. This is a foundational operation because many other tasks—like binary search, deduplication, grouping, building frequency tables, computing order statistics, or running two-pointer techniques—become easier and faster on sorted data. There are two broad families of sorting algorithms. Comparison-based sorts (like merge sort, quicksort, heapsort) use only element comparisons to decide order; they have a proven lower bound of O(n \log n) comparisons in the worst case. Non-comparison sorts (like counting, radix, and bucket sort) assume structure about the keys (small integer ranges, fixed-length digits, or distributions) and can run in linear time under those assumptions. Practical libraries combine ideas from multiple algorithms: for example, C++ std::sort uses introsort, switching strategies to avoid worst-case slowdowns and to leverage small-input optimizations. Different algorithms trade off stability (keeping equal-key items in their original order), memory usage, predictability, and constant factors. Understanding these trade-offs helps you pick the right tool for constraints on time, memory, and data characteristics.

02Intuition & Analogies

Imagine organizing books on a shelf. Merge sort is like having two helpers who each produce a neatly sorted pile; you then zip those two piles together by always taking the smaller top book—steady and predictable, but you need space on the table for the piles. Quicksort is like picking one book as a pivot, tossing lighter books to the left and heavier to the right, then repeating on each side—fast if your pivot choice splits the shelf well, but slow if you always pick the worst pivot. Heapsort is like maintaining a big-to-small pile where the largest item can always be pulled from the top; you repeatedly take the largest and place it at the end of the shelf—space-efficient but with more per-move work. For non-comparison methods, think of counting sort as sorting exam scores when scores range from 0 to 100: you simply count how many of each score there are and then write them back in order—no need to compare individual scores. Radix sort is like sorting words by letters from right to left: first sort by the last letter, then the second-to-last, and so on, using a stable method each time so earlier sorts are preserved. Bucket sort is like placing uniformly random marbles into labeled bins by range (0–9, 10–19, ...), lightly sorting within each bin, then concatenating bins—great when marbles are spread evenly. In real programs, we often want a mix: fast on average, safe in the worst case, and gentle on memory, which is why hybrids like introsort are the go-to choice in C++.

03Formal Definition

Given a sequence A of n elements with a total order relation \(\), sorting produces a permutation \(\) of indices so that \( \). A sorting algorithm is stable if for any equal keys \(x = y\) with original indices \(i < j\), the permutation preserves order: \((i) < (j)\). Comparison-based algorithms determine order solely via pairwise comparisons between elements, which can be modeled as a decision tree with at most \(n!\) possible leaves (one per permutation). This yields a worst-case lower bound of \((n n)\) comparisons. Specific classic algorithms include: merge sort (divide-and-conquer with a linear-time merge), quicksort (partition around a pivot then recurse), and heapsort (heap-based selection). Non-comparison sorts exploit additional structure: counting sort assumes keys lie in a small integer range; radix sort treats keys as sequences of digits and sorts digit-by-digit stably; bucket sort assumes a known distribution and uses bins before small in-bin sorts. Practical library sorts like C++ introsort combine methods (quicksort, heapsort, insertion sort) with strategy switches to achieve both average efficiency and worst-case guarantees.

04When to Use

  • Use merge sort when you need stability and predictable O(n \log n) time, and you can afford O(n) extra memory. It’s excellent for linked lists and external sorting (files) where merging is I/O-friendly.
  • Use quicksort when you prefer in-place sorting and average-case speed; randomization or median-of-three pivoting helps avoid O(n^2) cases. It is often fastest on typical RAM-resident arrays due to good cache behavior.
  • Use heapsort when you must guarantee O(n \log n) time with O(1) extra memory and stability is not required. It’s also useful when building a priority queue or repeatedly extracting extrema.
  • Use counting sort when keys are integers within a small known range K; it runs in O(n + K) and can be made stable, which is also a building block for radix sort.
  • Use radix sort when keys have fixed length or bounded digits (e.g., 32-bit integers, zero-padded strings) and when you can apply a stable bucketed pass per digit; it can achieve near-linear time with appropriate base.
  • Use bucket sort when inputs are roughly uniform over a range and simple in-bin sorts are cheap; expected linear time follows from balanced buckets.
  • In competitive programming, default to std::sort for general arrays (fast hybrid introsort) and std::stable_sort when you must preserve equal-key order (e.g., multi-key sorts).

⚠️Common Mistakes

  • Ignoring stability when it matters: using an unstable sort before a secondary-key sort breaks multi-key ordering. Prefer stable_sort or implement stable algorithms (merge sort, stable counting/radix passes) when needed.
  • Poor pivot selection in quicksort: always choosing the first/last element on already sorted arrays causes O(n^2). Use random or median-of-three pivots and consider three-way partitioning to handle many duplicates.
  • Off-by-one and partition bugs: incorrect loop bounds or swaps can drop or duplicate elements. Thoroughly test with tiny arrays (sizes 0–5), arrays with duplicates, and already sorted/reversed arrays.
  • Memory misuse in merge sort: forgetting to allocate one shared auxiliary buffer or repeatedly allocating per merge harms performance. Preallocate a single temp array and reuse it.
  • Misusing counting sort on large or unknown ranges: K too large causes excessive memory and time; also, forgetting to handle negative integers via an offset leads to out-of-bounds access.
  • Breaking comparator rules: std::sort requires a strict weak ordering; inconsistent comparators (e.g., equal elements sometimes compared as less) cause undefined behavior.
  • Radix/bucket pitfalls: using an unstable per-digit pass breaks radix correctness; for bucket sort, assuming uniform distribution when it isn’t leads to degraded performance.

Key Formulas

Comparison Sort Lower Bound

Explanation: Any comparison-based sort must distinguish among n! permutations using a binary decision tree. The height of such a tree is at least log2(n!), yielding a worst-case lower bound.

Stirling Link

Explanation: Using Stirling’s approximation, log n! grows like n log n. This connects the decision-tree bound to the familiar O(n log n) lower bound for comparison sorting.

Merge Sort Recurrence

Explanation: Merge sort splits the array in half, recursively sorts each half, then merges in linear time. The Master Theorem yields O(n log n) time.

Quicksort Average Time

Explanation: Random pivots make the expected split balanced. The expected number of comparisons is proportional to n log n.

Quicksort Worst Case

Explanation: Consistently poor pivots (e.g., always smallest or largest) lead to highly unbalanced partitions and quadratic time.

Heapsort Complexity

Explanation: Building a heap is linear; then n extract-max operations each cost O(log n). Total time is O(n log n).

Counting Sort Complexity

Explanation: Counting sort scans the input and builds counts over K possible keys, then reconstructs output. It is linear in n plus the key range K.

Radix Sort Complexity

Explanation: With d digits and a stable base-K pass (often counting sort), total time is d times the per-pass cost. Choose K to balance passes and counting overhead.

Bucket Sort Expected Time

Explanation: If inputs are uniformly distributed so buckets are balanced, distributing and sorting small buckets takes linear expected time.

Triangular Number

Explanation: Appears in analyzing naive O() sorts like insertion or selection sort when summing pass lengths. It shows quadratic growth with n.

Merge Sort Space

Explanation: Merging two arrays requires temporary space proportional to the input to hold intermediate results. On arrays this is typically an extra n elements.

Quicksort Depth

Explanation: Randomized pivots yield logarithmic recursion depth with high probability, keeping stack space small.

Complexity Analysis

For comparison sorts, the decision-tree model proves any algorithm needs Ω(n log n) comparisons in the worst case. Merge sort meets this bound with deterministic O(n log n) time and requires O(n) auxiliary memory for the merge step. It has excellent predictability and stability but weaker cache behavior due to extra buffers and non-in-place merges. Quicksort on average performs O(n log n) comparisons and moves, with typically superior cache locality thanks to partition scans over contiguous memory; however, it can degrade to O() in the worst case if pivots are poorly chosen. Randomization or median-of-three heuristics and three-way partitioning mitigate this risk and reduce recursion depth to O(log n) on average. Heapsort guarantees O(n log n) time deterministically and O(1) auxiliary space but tends to have larger constants and more cache misses because of non-sequential heap navigation. Non-comparison sorts avoid the Ω(n log n) barrier by leveraging structure. Counting sort runs in O(n + K) time and O(n + K) space, where K is the key range; this is attractive only when K is reasonably small relative to n. Radix sort processes d digits with a stable pass of cost O(n + K), yielding O(d(n + K)) time and typically O(n + K) space; it is practical for fixed-width integers or padded strings with an appropriate base (e.g., 256). Bucket sort achieves expected O(n + K) time under uniform distributions by distributing values to buckets and sorting each bucket (often with insertion sort). In practice, C++ std::sort (introsort) gives near-quicksort speed with a fall-back to heapsort to ensure O(n log n) worst-case time and switches to insertion sort for tiny subarrays, balancing constants, recursion depth, and memory usage.

Code Examples

Stable Merge Sort on Arrays (Top-Down)
1#include <bits/stdc++.h>
2using namespace std;
3
4// Merge two sorted ranges [l, m) and [m, r) into tmp, then copy back to a
5static void merge_ranges(vector<int>& a, vector<int>& tmp, int l, int m, int r) {
6 int i = l, j = m, k = l;
7 while (i < m && j < r) {
8 // Stable: when a[i] == a[j], take from left first
9 if (a[i] <= a[j]) tmp[k++] = a[i++];
10 else tmp[k++] = a[j++];
11 }
12 while (i < m) tmp[k++] = a[i++];
13 while (j < r) tmp[k++] = a[j++];
14 // Copy merged result back to a
15 for (int t = l; t < r; ++t) a[t] = tmp[t];
16}
17
18static void merge_sort_rec(vector<int>& a, vector<int>& tmp, int l, int r) {
19 if (r - l <= 1) return; // 0 or 1 element is already sorted
20 int m = l + (r - l) / 2;
21 merge_sort_rec(a, tmp, l, m);
22 merge_sort_rec(a, tmp, m, r);
23 // Optimization: if already ordered, skip merge
24 if (a[m - 1] <= a[m]) return;
25 merge_ranges(a, tmp, l, m, r);
26}
27
28void merge_sort(vector<int>& a) {
29 vector<int> tmp(a.size()); // O(n) auxiliary buffer reused across merges
30 merge_sort_rec(a, tmp, 0, (int)a.size());
31}
32
33int main() {
34 ios::sync_with_stdio(false);
35 cin.tie(nullptr);
36
37 vector<int> a = {5, 1, 4, 2, 8, 5, 3, 5};
38 merge_sort(a);
39 for (int x : a) cout << x << ' ';
40 cout << "\n";
41 return 0;
42}
43

This is a classic top-down merge sort. It recursively splits the array, sorts halves, and merges them using a single shared temporary buffer to achieve stability and O(n) extra memory. A small optimization skips merging when the boundary is already in order.

Time: O(n log n)Space: O(n)
Randomized Three-Way Quicksort with Tail-Recursion Elimination
1#include <bits/stdc++.h>
2using namespace std;
3
4static std::mt19937 rng((uint32_t)chrono::steady_clock::now().time_since_epoch().count());
5
6// Partition a[l..r] into < pivot, == pivot, > pivot. Returns pair {lt_end, gt_begin}.
7static pair<int,int> partition3(vector<int>& a, int l, int r, int pivot) {
8 int i = l, lt = l, gt = r;
9 while (i <= gt) {
10 if (a[i] < pivot) swap(a[i++], a[lt++]);
11 else if (a[i] > pivot) swap(a[i], a[gt--]);
12 else i++;
13 }
14 return {lt, gt + 1}; // [l, lt) < pivot, [lt, gt+1) == pivot, [gt+1, r+1) > pivot
15}
16
17void quicksort(vector<int>& a, int l, int r) {
18 while (l < r) {
19 // Switch to insertion sort for tiny segments to improve constants
20 if (r - l + 1 <= 16) {
21 for (int i = l + 1; i <= r; ++i) {
22 int x = a[i], j = i - 1;
23 while (j >= l && a[j] > x) { a[j + 1] = a[j]; --j; }
24 a[j + 1] = x;
25 }
26 return;
27 }
28 uniform_int_distribution<int> dist(l, r);
29 int pivot = a[dist(rng)];
30 auto [lt, gt] = partition3(a, l, r, pivot);
31 // Recurse on smaller side to bound recursion depth, loop on larger side
32 if (lt - l < r - gt + 1) {
33 if (l < lt - 1) quicksort(a, l, lt - 1);
34 l = gt; // tail-recurse on the larger part via loop
35 } else {
36 if (gt < r) quicksort(a, gt, r);
37 r = lt - 1; // continue on the other side
38 }
39 }
40}
41
42int main() {
43 ios::sync_with_stdio(false);
44 cin.tie(nullptr);
45
46 vector<int> a = {9, 3, 5, 3, 3, 8, 2, 7, 6, 3, 1, 0, 3};
47 quicksort(a, 0, (int)a.size() - 1);
48 for (int x : a) cout << x << ' ';
49 cout << '\n';
50 return 0;
51}
52

This quicksort uses random pivots and three-way partitioning to handle many duplicates efficiently. It switches to insertion sort on tiny segments and eliminates tail recursion by always recursing on the smaller side, yielding O(log n) stack space on average and strong practical performance.

Time: Average O(n log n), worst-case O(n^2)Space: O(log n) average stack, O(1) auxiliary
Stable Counting Sort for Small Integer Ranges (Handles Negatives)
1#include <bits/stdc++.h>
2using namespace std;
3
4void counting_sort(vector<int>& a) {
5 if (a.empty()) return;
6 int mn = *min_element(a.begin(), a.end());
7 int mx = *max_element(a.begin(), a.end());
8 long long K = 1LL * mx - mn + 1; // key range size
9 if (K > 2e7) {
10 // Guard: range too large for counting sort memory in typical settings
11 throw runtime_error("Range too large for counting sort");
12 }
13 vector<int> cnt((size_t)K, 0);
14 for (int x : a) cnt[x - mn]++;
15 // Prefix sums to get positions (stable placement)
16 for (size_t i = 1; i < cnt.size(); ++i) cnt[i] += cnt[i - 1];
17 vector<int> out(a.size());
18 // Traverse from right to left for stability
19 for (int i = (int)a.size() - 1; i >= 0; --i) {
20 int key = a[i] - mn;
21 out[--cnt[key]] = a[i];
22 }
23 a.swap(out);
24}
25
26int main() {
27 ios::sync_with_stdio(false);
28 cin.tie(nullptr);
29
30 vector<int> a = {5, -2, 7, 5, 0, -2, 3, 5};
31 try {
32 counting_sort(a);
33 for (int x : a) cout << x << ' ';
34 cout << '\n';
35 } catch (const exception& e) {
36 cerr << "Error: " << e.what() << '\n';
37 }
38 return 0;
39}
40

This is a stable counting sort that supports negative integers by shifting keys with an offset. It builds a frequency array, computes prefix sums to determine positions, and writes output in reverse to preserve input order among equal keys. A guard prevents excessive memory use when the key range is too large.

Time: O(n + K) where K = max - min + 1Space: O(n + K)
std::sort (Introsort) vs std::stable_sort (Stability Demo)
1#include <bits/stdc++.h>
2using namespace std;
3
4struct Item {
5 int key; // primary key to sort by
6 int id; // original position (to observe stability)
7};
8
9int main() {
10 ios::sync_with_stdio(false);
11 cin.tie(nullptr);
12
13 vector<Item> v = {{5,0},{3,1},{5,2},{2,3},{5,4},{3,5}};
14
15 auto cmp_key = [](const Item& a, const Item& b){ return a.key < b.key; };
16
17 // Copy for two different sorts
18 auto a = v, b = v;
19
20 // std::sort is a fast hybrid (introsort) but not stable
21 sort(a.begin(), a.end(), cmp_key);
22
23 // std::stable_sort preserves the relative order of equal keys
24 stable_sort(b.begin(), b.end(), cmp_key);
25
26 cout << "std::sort order (key,id):\n";
27 for (auto &x : a) cout << '(' << x.key << ',' << x.id << ") ";
28 cout << "\n\nstd::stable_sort order (key,id):\n";
29 for (auto &x : b) cout << '(' << x.key << ',' << x.id << ") ";
30 cout << '\n';
31 return 0;
32}
33

This program compares std::sort (introsort) with std::stable_sort. When multiple items share the same key, stable_sort maintains their original order (observed via id), while std::sort may reorder them. In competitive programming, prefer std::sort by default and switch to std::stable_sort when stable behavior is required.

Time: O(n log n) for bothSpace: std::sort: O(log n) stack; std::stable_sort: O(n) auxiliary