Hash Tables, Universal Hashing & Perfect Hashing (FKS)

Agenda

1. The Dictionary Problem & Dimensionality Reduction
2. The Adversarial Threat: Why Determinism Fails (HashDoS)
3. Universal Hashing (Carter & Wegman): Randomized Families & O(1+\alpha) Expected Search
4. Constructing Universal Families: The Modular Linear Family \mathcal{H}_{p,m}
5. The Balls & Bins Bottleneck: Why Single Tables Fail Worst-Case O(1)
6. Perfect Hashing (FKS 2-Level Scheme): Worst-Case O(1) Search & O(n) Space

What is Hashing? (Dimensionality Reduction)

Hashing in Data Structures (Dynamic Set/Dictionary)

• The Mission: Ultra-fast key-value retrieval (Insert, Lookup, Delete) via direct O(1) Word RAM addressing.
• Massive Compression: Maps a huge 2^{64} universe into a compact table m = \Theta(n).
• Mathematical Lens (Balls & Bins): Expected load is \alpha = 1, but max chain is \mathbf{\Theta\left(\frac{\ln n}{\ln \ln n}\right)} whp.
• Adversary Defense: Drawing h \sim \mathcal{H} randomly from a universal family blinds attackers (HashDoS).

Hashing in Cryptography (Integrity & Security)

• The Mission: Digital signatures, passwords, and tamper-evident ledgers (SHA-256).
• Avalanche Digest: Flipping a single input bit completely scrambles the output hash.
• One-Way Trapdoor: Computationally infeasible to invert (pre-image resistance).
• Mathematical Lens (Birthday Paradox): Finding any collision takes \mathbf{\Theta(2^{b/2})} ops \implies 2^{128} barrier for b=256.

The Core Problem: The Dynamic Dictionary ADT

The Dictionary Problem maintains a dynamic set S \subseteq U of n active elements (n \ll |U|):

• Operations: Insert(key, value), Delete(key), Lookup(key) -> value.
• Ubiquitous in Systems:
  • Language Runtimes: Python dict, JS objects, and JVM symbol tables.
  • High-Throughput Caches: Redis & Memcached (millions of ops/sec).
  • Hardware Routers: Network switch forwarding tables at 100 Gbps+.
• The Goal: Can we achieve instantaneous O(1) lookup time independent of dataset size n?

Breaking O(\log n): The Word RAM Model

How does Hashing achieve O(1) operations instead of O(\log n) tree traversals?

• The Word RAM Model: Keys are numerical bit-vectors in machine words (k \in \{0, 1\}^w), enabling O(1) native ALU arithmetic.
• Direct Address Calculation: Target RAM address is computed via an arithmetic formula: h(k) \in \{0, 1, \dots, m-1\}.
• Direct Memory Jump (O(1) Time): CPU jumps directly to array location T[h(k)] in a single memory dereference—no comparisons needed!

The Fingerprinting Principle: Freivalds \to Hashing

• Randomized Fingerprinting: A compact random projection of high-dimensional data that preserves equality testing with high probability.
• Recall Lecture 6 (Freivalds’ Verification):
  • Vector d \in \{0, 1\}^n \to 1-bit fingerprint h_r(d) = d \cdot r \pmod 2.
  • If d \neq 0, collision probability is \le 1/2.
• Hashing as Key Fingerprinting:
  • Compresses 64-bit key x \in U \to \log_2 m-bit slot index h(x) \in [m].
  • For distinct x \neq y, collision probability is strictly \le \frac{1}{m}.

The Massive Universe Dilemma

Why can’t we simply use a direct array indexed by key value?

• 64-bit Key Universe: |U| = 2^{64} \approx 1.84 \times 10^{19} possible keys.
  • Direct array would require over 10^7 Petabytes of RAM just to allocate!
• Sparse Active Set: In practice, we only store n \ll |U| keys (e.g., n = 10^6 active items).
• Hashing Compression:
  • Maps universe U down to a compact table of size m = \Theta(n) (O(n) space, \sim 8 MB).
  • Pigeonhole Principle: Since |U| \gg m, collisions are mathematically inevitable.

Collision Resolution: Chaining

When multiple keys hash to the same slot, we store them in a linked list:

• Insert(x): Prepend to head of list T[h(x)] in O(1) worst-case time.
• Lookup(x) / Delete(x): Time proportional to chain length at T[h(x)].

The Adversarial Threat: HashDoS Attack

Why is any fixed deterministic hash (e.g., h(k) = k \bmod m) fatal?

• Pigeonhole Principle: At least |U|/m keys map to the exact same slot.
• Adversary Exploit: Attacker sends S = \{m, 2m, \dots, nm\} \implies \text{all } h(k) = 0.
• Catastrophic Collapse: All n keys land in one bucket \implies \mathbf{\Theta(n)} worst-case search!
• The Shield: Choose h randomly at runtime from a universal family \mathcal{H}.

Universal Hashing (Carter & Wegman, 1979)

• Randomness Source: Probability is over the algorithm’s internal choice h \in \mathcal{H}, not input data.
• Adversary Blinded: Guarantee holds for any worst-case dataset, since h is chosen secretly at runtime.

Search Cost Analysis: Pairwise Linearity

• Collision Indicator: I_{x,y} = 1 if h(x) = h(y), and 0 otherwise.
• Chain Length: Slot h(x) has chain length C_x = \sum_{y \neq x} I_{x,y}.
• Pairwise Linearity: \mathbb{E}[C_x] = \sum_{y \neq x} \Pr[h(x)=h(y)] \le \frac{n-1}{m} < \alpha.
• Takeaway: Pairwise independence guarantees O(1) expected search!

Constructing a Universal Family

Choose prime p \ge |U|. The Carter–Wegman linear family: \mathcal{H}_{p,m} = \left\{\, h_{a,b}(x) = ((ax + b) \bmod p) \bmod m \;\mid\; a \in \mathbb{Z}_p^*,\, b \in \mathbb{Z}_p \,\right\}

• Compact Storage: Only 2 integers stored: a, b (2 \times 64 bits).
• Bijection: For x \neq y, (a,b) \mapsto (ax+b \bmod p, \, ay+b \bmod p) is bijective in \mathbb{Z}_p^* \times \mathbb{Z}_p.
• Collision Bound: r \equiv s \pmod m holds for at most (p-1)/m values.
• Universality: \Pr[h(x)=h(y)] \le \frac{(p-1)/m}{p-1} = \frac{1}{m}. \quad\blacksquare

The Balls & Bins Bottleneck

How does single-hash chaining behave in the worst case?

• Average Case: For any bucket i, expected load is \mathbb{E}[L_i] = \frac{n}{m} = 1 \implies O(1) search.
• Worst-Case Chain (Maximum Load):
  • In Lecture 5, we proved via Chernoff Bounds: \text{Max Bucket Size} = \mathbf{\Theta\left(\frac{\ln n}{\ln \ln n}\right)} \quad \text{with high probability}
• The Bottleneck: Even with ideal hashing, single-table chaining cannot achieve worst-case O(1) lookup—unlucky keys suffer \Theta(\frac{\ln n}{\ln \ln n}) latency.

Static Sets: FKS 2-Level Architecture

For static sets (S fixed, no inserts/deletes), can we achieve O(1) worst-case lookup in O(n) space?

• Fredman, Komlós, & Szemerédi (1984) Architecture:
  • Level 1: Hash n keys into m = n primary buckets using universal hash h.
  • Level 2: For each bucket i, allocate secondary table S_i of size m_i = n_i^2 with universal hash h_i.

FKS Analysis: Zero Secondary Collisions

Why does secondary table size m_i = n_i^2 guarantee zero collisions?

• Expected Collisions at Level 2 (Birthday Paradox): \mathbb{E}[\text{collisions in } S_i] = \binom{n_i}{2} \frac{1}{m_i} = \frac{n_i(n_i-1)}{2 n_i^2} < \frac{1}{2}

• Applying Markov’s Inequality: \Pr[\text{collisions} \ge 1] \le \frac{\mathbb{E}[\text{collisions}]}{1} < \frac{1}{2}

• Result: With probability \ge 1/2, S_i has zero collisions. A collision-free h_i is found in \le 2 draws on average!

• Worst-Case Search Time: Compute i = h(x), then look up S_i[h_i(x)] \implies strictly 2 memory probes (O(1) worst-case)! \quad\blacksquare

FKS Analysis: Linear Total Space O(n)

Does allocating quadratic secondary tables m_i = n_i^2 blow up total space?

• Total Space Formula: \text{Space} = \sum_{i=1}^n n_i^2 = n + 2 \sum_{x < y} I_{x,y}.

• Expected Space: By universality of primary hash function h: \mathbb{E}\left[\sum_{i=1}^n n_i^2\right] = n + 2\binom{n}{2}\frac{1}{n} < \mathbf{2n}

• Markov Space Guarantee: \Pr\left[\sum n_i^2 \ge 4n\right] \le \frac{2n}{4n} = \frac{1}{2}.

• Conclusion: Total space across all secondary tables is strictly < 2n = \mathbf{O(n)}. \quad\blacksquare

The Dynamic Dilemma: Why FKS Fails Dynamically

• The FKS Bottleneck: Inserting into bucket i forces secondary reallocation (n_i^2 \to (n_i+1)^2) \implies \Omega(n_i^2) rebuild per insert.
• Takeaway: FKS solves the dictionary problem with O(1) worst-case lookup in O(n) space, but is strictly for static sets.

Concept Check & Socratic Prompts

• Q1: Why is pairwise independence sufficient for universal chaining?
  Search time is the sum of pairwise collision indicators \sum_{y \neq x} I_{x,y}. Linearity of expectation requires only that each pair collides with \Pr \le 1/m.

• Q2: Why does FKS Level 2 (m_i = n_i^2) achieve zero collisions?
  By the Birthday Paradox, \mathbb{E}[\text{coll}] < 1/2. By Markov’s inequality, \Pr[\ge 1 \text{ coll}] < 1/2. A fresh universal draw succeeds in \le 2 tries.

• Q3: Why can’t FKS be used efficiently for dynamic sets?
  Adding 1 key forces quadratic table reallocation (n_i+1)^2, requiring expensive \Omega(n_i^2) rehashes on updates.

Next Frontier

Randomized & Algebraic Shortest Paths (Lecture 14):

• Algebraic Graph Algorithms: Moving beyond standard combinatorial traversals into algebraic matrix operations.
• The (\min, +) Tropical Semiring: Why standard fast matrix multiplication fails over shortest path semirings.
• Seidel’s Algorithm (1995): Solving unweighted undirected All-Pairs Shortest Paths in \tilde{O}(n^\omega) time using boolean matrix multiplication over rings!