Sorting Algorithms
Key Points
- •Sorting arranges items into a chosen order so that searching, grouping, and further algorithms become faster and simpler.
- •Comparison-based sorts like merge sort, quicksort, and heapsort have a theoretical lower bound of O(n n) comparisons.
- •Merge sort is stable and predictable O(n n) time but needs O(n) extra memory for merging.
- •Quicksort is in-place and usually fastest in practice with O(n n) average time, but it can degrade to O() without care.
- •Heapsort is in-place with O(n n) worst-case time but is not stable and often slower in practice than quicksort.
- •Counting, radix, and bucket sorts can achieve O(n) time when keys have a restricted form (small range, fixed digits, or uniform distribution).
- •C++ std::sort uses introsort, a hybrid that combines quicksort, heapsort, and insertion sort to guarantee O(n n) worst-case time.
- •Choosing the right sort depends on stability needs, memory limits, key properties, and input patterns (e.g., many duplicates, nearly sorted).
Prerequisites
- →Asymptotic notation (Big-O, Θ, Ω) — To analyze and compare time and space costs of different sorting algorithms.
- →Recursion and divide-and-conquer — Merge sort and quicksort are naturally expressed recursively and analyzed via recurrences.
- →Arrays, vectors, and memory layout — Most sorting implementations manipulate contiguous arrays and benefit from understanding cache effects and indexing.
- →Heaps and priority queues — Heapsort and partial sorting operations rely on heap properties and operations.
- →Randomization basics — Randomized pivot selection in quicksort provides average-case guarantees and avoids adversarial inputs.
- →Counting and discrete math — Decision-tree lower bounds and counting sort rely on combinatorics and frequency counting.
Detailed Explanation
Tap terms for definitions01Overview
Sorting is the task of arranging a collection of items (numbers, strings, records) into a desired order such as ascending or descending. This is a foundational operation because many other tasks—like binary search, deduplication, grouping, building frequency tables, computing order statistics, or running two-pointer techniques—become easier and faster on sorted data. There are two broad families of sorting algorithms. Comparison-based sorts (like merge sort, quicksort, heapsort) use only element comparisons to decide order; they have a proven lower bound of O(n \log n) comparisons in the worst case. Non-comparison sorts (like counting, radix, and bucket sort) assume structure about the keys (small integer ranges, fixed-length digits, or distributions) and can run in linear time under those assumptions. Practical libraries combine ideas from multiple algorithms: for example, C++ std::sort uses introsort, switching strategies to avoid worst-case slowdowns and to leverage small-input optimizations. Different algorithms trade off stability (keeping equal-key items in their original order), memory usage, predictability, and constant factors. Understanding these trade-offs helps you pick the right tool for constraints on time, memory, and data characteristics.
02Intuition & Analogies
Imagine organizing books on a shelf. Merge sort is like having two helpers who each produce a neatly sorted pile; you then zip those two piles together by always taking the smaller top book—steady and predictable, but you need space on the table for the piles. Quicksort is like picking one book as a pivot, tossing lighter books to the left and heavier to the right, then repeating on each side—fast if your pivot choice splits the shelf well, but slow if you always pick the worst pivot. Heapsort is like maintaining a big-to-small pile where the largest item can always be pulled from the top; you repeatedly take the largest and place it at the end of the shelf—space-efficient but with more per-move work. For non-comparison methods, think of counting sort as sorting exam scores when scores range from 0 to 100: you simply count how many of each score there are and then write them back in order—no need to compare individual scores. Radix sort is like sorting words by letters from right to left: first sort by the last letter, then the second-to-last, and so on, using a stable method each time so earlier sorts are preserved. Bucket sort is like placing uniformly random marbles into labeled bins by range (0–9, 10–19, ...), lightly sorting within each bin, then concatenating bins—great when marbles are spread evenly. In real programs, we often want a mix: fast on average, safe in the worst case, and gentle on memory, which is why hybrids like introsort are the go-to choice in C++.
03Formal Definition
04When to Use
- Use merge sort when you need stability and predictable O(n \log n) time, and you can afford O(n) extra memory. It’s excellent for linked lists and external sorting (files) where merging is I/O-friendly.
- Use quicksort when you prefer in-place sorting and average-case speed; randomization or median-of-three pivoting helps avoid O(n^2) cases. It is often fastest on typical RAM-resident arrays due to good cache behavior.
- Use heapsort when you must guarantee O(n \log n) time with O(1) extra memory and stability is not required. It’s also useful when building a priority queue or repeatedly extracting extrema.
- Use counting sort when keys are integers within a small known range K; it runs in O(n + K) and can be made stable, which is also a building block for radix sort.
- Use radix sort when keys have fixed length or bounded digits (e.g., 32-bit integers, zero-padded strings) and when you can apply a stable bucketed pass per digit; it can achieve near-linear time with appropriate base.
- Use bucket sort when inputs are roughly uniform over a range and simple in-bin sorts are cheap; expected linear time follows from balanced buckets.
- In competitive programming, default to std::sort for general arrays (fast hybrid introsort) and std::stable_sort when you must preserve equal-key order (e.g., multi-key sorts).
⚠️Common Mistakes
- Ignoring stability when it matters: using an unstable sort before a secondary-key sort breaks multi-key ordering. Prefer stable_sort or implement stable algorithms (merge sort, stable counting/radix passes) when needed.
- Poor pivot selection in quicksort: always choosing the first/last element on already sorted arrays causes O(n^2). Use random or median-of-three pivots and consider three-way partitioning to handle many duplicates.
- Off-by-one and partition bugs: incorrect loop bounds or swaps can drop or duplicate elements. Thoroughly test with tiny arrays (sizes 0–5), arrays with duplicates, and already sorted/reversed arrays.
- Memory misuse in merge sort: forgetting to allocate one shared auxiliary buffer or repeatedly allocating per merge harms performance. Preallocate a single temp array and reuse it.
- Misusing counting sort on large or unknown ranges: K too large causes excessive memory and time; also, forgetting to handle negative integers via an offset leads to out-of-bounds access.
- Breaking comparator rules: std::sort requires a strict weak ordering; inconsistent comparators (e.g., equal elements sometimes compared as less) cause undefined behavior.
- Radix/bucket pitfalls: using an unstable per-digit pass breaks radix correctness; for bucket sort, assuming uniform distribution when it isn’t leads to degraded performance.
Key Formulas
Comparison Sort Lower Bound
Explanation: Any comparison-based sort must distinguish among n! permutations using a binary decision tree. The height of such a tree is at least log2(n!), yielding a worst-case lower bound.
Stirling Link
Explanation: Using Stirling’s approximation, log n! grows like n log n. This connects the decision-tree bound to the familiar O(n log n) lower bound for comparison sorting.
Merge Sort Recurrence
Explanation: Merge sort splits the array in half, recursively sorts each half, then merges in linear time. The Master Theorem yields O(n log n) time.
Quicksort Average Time
Explanation: Random pivots make the expected split balanced. The expected number of comparisons is proportional to n log n.
Quicksort Worst Case
Explanation: Consistently poor pivots (e.g., always smallest or largest) lead to highly unbalanced partitions and quadratic time.
Heapsort Complexity
Explanation: Building a heap is linear; then n extract-max operations each cost O(log n). Total time is O(n log n).
Counting Sort Complexity
Explanation: Counting sort scans the input and builds counts over K possible keys, then reconstructs output. It is linear in n plus the key range K.
Radix Sort Complexity
Explanation: With d digits and a stable base-K pass (often counting sort), total time is d times the per-pass cost. Choose K to balance passes and counting overhead.
Bucket Sort Expected Time
Explanation: If inputs are uniformly distributed so buckets are balanced, distributing and sorting small buckets takes linear expected time.
Triangular Number
Explanation: Appears in analyzing naive O() sorts like insertion or selection sort when summing pass lengths. It shows quadratic growth with n.
Merge Sort Space
Explanation: Merging two arrays requires temporary space proportional to the input to hold intermediate results. On arrays this is typically an extra n elements.
Quicksort Depth
Explanation: Randomized pivots yield logarithmic recursion depth with high probability, keeping stack space small.
Complexity Analysis
Code Examples
1 #include <bits/stdc++.h> 2 using namespace std; 3 4 // Merge two sorted ranges [l, m) and [m, r) into tmp, then copy back to a 5 static void merge_ranges(vector<int>& a, vector<int>& tmp, int l, int m, int r) { 6 int i = l, j = m, k = l; 7 while (i < m && j < r) { 8 // Stable: when a[i] == a[j], take from left first 9 if (a[i] <= a[j]) tmp[k++] = a[i++]; 10 else tmp[k++] = a[j++]; 11 } 12 while (i < m) tmp[k++] = a[i++]; 13 while (j < r) tmp[k++] = a[j++]; 14 // Copy merged result back to a 15 for (int t = l; t < r; ++t) a[t] = tmp[t]; 16 } 17 18 static void merge_sort_rec(vector<int>& a, vector<int>& tmp, int l, int r) { 19 if (r - l <= 1) return; // 0 or 1 element is already sorted 20 int m = l + (r - l) / 2; 21 merge_sort_rec(a, tmp, l, m); 22 merge_sort_rec(a, tmp, m, r); 23 // Optimization: if already ordered, skip merge 24 if (a[m - 1] <= a[m]) return; 25 merge_ranges(a, tmp, l, m, r); 26 } 27 28 void merge_sort(vector<int>& a) { 29 vector<int> tmp(a.size()); // O(n) auxiliary buffer reused across merges 30 merge_sort_rec(a, tmp, 0, (int)a.size()); 31 } 32 33 int main() { 34 ios::sync_with_stdio(false); 35 cin.tie(nullptr); 36 37 vector<int> a = {5, 1, 4, 2, 8, 5, 3, 5}; 38 merge_sort(a); 39 for (int x : a) cout << x << ' '; 40 cout << "\n"; 41 return 0; 42 } 43
This is a classic top-down merge sort. It recursively splits the array, sorts halves, and merges them using a single shared temporary buffer to achieve stability and O(n) extra memory. A small optimization skips merging when the boundary is already in order.
1 #include <bits/stdc++.h> 2 using namespace std; 3 4 static std::mt19937 rng((uint32_t)chrono::steady_clock::now().time_since_epoch().count()); 5 6 // Partition a[l..r] into < pivot, == pivot, > pivot. Returns pair {lt_end, gt_begin}. 7 static pair<int,int> partition3(vector<int>& a, int l, int r, int pivot) { 8 int i = l, lt = l, gt = r; 9 while (i <= gt) { 10 if (a[i] < pivot) swap(a[i++], a[lt++]); 11 else if (a[i] > pivot) swap(a[i], a[gt--]); 12 else i++; 13 } 14 return {lt, gt + 1}; // [l, lt) < pivot, [lt, gt+1) == pivot, [gt+1, r+1) > pivot 15 } 16 17 void quicksort(vector<int>& a, int l, int r) { 18 while (l < r) { 19 // Switch to insertion sort for tiny segments to improve constants 20 if (r - l + 1 <= 16) { 21 for (int i = l + 1; i <= r; ++i) { 22 int x = a[i], j = i - 1; 23 while (j >= l && a[j] > x) { a[j + 1] = a[j]; --j; } 24 a[j + 1] = x; 25 } 26 return; 27 } 28 uniform_int_distribution<int> dist(l, r); 29 int pivot = a[dist(rng)]; 30 auto [lt, gt] = partition3(a, l, r, pivot); 31 // Recurse on smaller side to bound recursion depth, loop on larger side 32 if (lt - l < r - gt + 1) { 33 if (l < lt - 1) quicksort(a, l, lt - 1); 34 l = gt; // tail-recurse on the larger part via loop 35 } else { 36 if (gt < r) quicksort(a, gt, r); 37 r = lt - 1; // continue on the other side 38 } 39 } 40 } 41 42 int main() { 43 ios::sync_with_stdio(false); 44 cin.tie(nullptr); 45 46 vector<int> a = {9, 3, 5, 3, 3, 8, 2, 7, 6, 3, 1, 0, 3}; 47 quicksort(a, 0, (int)a.size() - 1); 48 for (int x : a) cout << x << ' '; 49 cout << '\n'; 50 return 0; 51 } 52
This quicksort uses random pivots and three-way partitioning to handle many duplicates efficiently. It switches to insertion sort on tiny segments and eliminates tail recursion by always recursing on the smaller side, yielding O(log n) stack space on average and strong practical performance.
1 #include <bits/stdc++.h> 2 using namespace std; 3 4 void counting_sort(vector<int>& a) { 5 if (a.empty()) return; 6 int mn = *min_element(a.begin(), a.end()); 7 int mx = *max_element(a.begin(), a.end()); 8 long long K = 1LL * mx - mn + 1; // key range size 9 if (K > 2e7) { 10 // Guard: range too large for counting sort memory in typical settings 11 throw runtime_error("Range too large for counting sort"); 12 } 13 vector<int> cnt((size_t)K, 0); 14 for (int x : a) cnt[x - mn]++; 15 // Prefix sums to get positions (stable placement) 16 for (size_t i = 1; i < cnt.size(); ++i) cnt[i] += cnt[i - 1]; 17 vector<int> out(a.size()); 18 // Traverse from right to left for stability 19 for (int i = (int)a.size() - 1; i >= 0; --i) { 20 int key = a[i] - mn; 21 out[--cnt[key]] = a[i]; 22 } 23 a.swap(out); 24 } 25 26 int main() { 27 ios::sync_with_stdio(false); 28 cin.tie(nullptr); 29 30 vector<int> a = {5, -2, 7, 5, 0, -2, 3, 5}; 31 try { 32 counting_sort(a); 33 for (int x : a) cout << x << ' '; 34 cout << '\n'; 35 } catch (const exception& e) { 36 cerr << "Error: " << e.what() << '\n'; 37 } 38 return 0; 39 } 40
This is a stable counting sort that supports negative integers by shifting keys with an offset. It builds a frequency array, computes prefix sums to determine positions, and writes output in reverse to preserve input order among equal keys. A guard prevents excessive memory use when the key range is too large.
1 #include <bits/stdc++.h> 2 using namespace std; 3 4 struct Item { 5 int key; // primary key to sort by 6 int id; // original position (to observe stability) 7 }; 8 9 int main() { 10 ios::sync_with_stdio(false); 11 cin.tie(nullptr); 12 13 vector<Item> v = {{5,0},{3,1},{5,2},{2,3},{5,4},{3,5}}; 14 15 auto cmp_key = [](const Item& a, const Item& b){ return a.key < b.key; }; 16 17 // Copy for two different sorts 18 auto a = v, b = v; 19 20 // std::sort is a fast hybrid (introsort) but not stable 21 sort(a.begin(), a.end(), cmp_key); 22 23 // std::stable_sort preserves the relative order of equal keys 24 stable_sort(b.begin(), b.end(), cmp_key); 25 26 cout << "std::sort order (key,id):\n"; 27 for (auto &x : a) cout << '(' << x.key << ',' << x.id << ") "; 28 cout << "\n\nstd::stable_sort order (key,id):\n"; 29 for (auto &x : b) cout << '(' << x.key << ',' << x.id << ") "; 30 cout << '\n'; 31 return 0; 32 } 33
This program compares std::sort (introsort) with std::stable_sort. When multiple items share the same key, stable_sort maintains their original order (observed via id), while std::sort may reorder them. In competitive programming, prefer std::sort by default and switch to std::stable_sort when stable behavior is required.