Information Structure: Perfect (all history known) vs. Imperfect (hidden state).
Fundamental Tension: No player controls their destiny alone! Your payoff depends on what others do (s_{-i}).
Our Ultimate Goal: View Algorithm Design as a Game between an Algorithm Designer (minimizing runtime/queries) and an Adversary (maximizing bad inputs).
Simple Examples of Everyday Games
Games appear everywhere across decisions, markets, and competition:
Rock-Paper-Scissors (Zero-Sum, Simultaneous):
You and your friend throw hands simultaneously. Pure conflict: +1, -1, 0.
Penalty Shootout in Soccer (Zero-Sum, Asymmetric):
Choosing a Meeting Location / Technology Standard (Coordination Game):
You and a friend want to meet up: Café vs. Library (both prefer being together over ending up alone at different places).
Or two cars approaching each other in a narrow alley: swerving to the same relative side allows safe passage; swerving into each other causes a crash!
Tic-Tac-Toe & Chess (Sequential Tree Games):
Alternating turns on a board with full visibility, no randomness, and no hidden information.
Mechanism Design: “Reverse Game Theory”
If Game Theory asks “Given the rules and payoffs, how will agents behave?”…
Definition: Mechanism Design
Mechanism Design asks the reverse question: “Given desired social goals, how do we design the rules, payment schemes, and payoff functions so that self-interested agents naturally achieve our goal?”
Also known as “Inverse Game Theory” or “Incentive Engineering”:
The designer cannot force players to act against their self-interest.
Instead, the designer engineers the payoff matrix / utility function: u_i(s) = v_i(s) - p_i(s) \quad (\text{valuation minus designed payment } p_i)
Famous Classic Examples:
Vickrey (Second-Price) Auction: Designing the payment rule such that bidding your true valuation is a dominant strategy!
Kidney Exchange & Stable Matching: Designing matching algorithms so participants have no incentive to lie about their preferences (Strategyproofness).
Takeaway: Payoffs aren’t always given by nature—often, we get to design them!
Game Representations: Tree Games (Extensive Form)
When decisions happen sequentially in time, we represent the game as a directed tree.
Backward Induction: Solve from leaves backwards to the root.
P2 chooses the branch maximizing their payoff at each subgame:
At left node: \max(1, 0) = 1 \implies chooses Left \implies payoff (3, 1).
At right node: \max(2, 4) = 4 \implies chooses Right \implies payoff (2, 4).
Expected Payoff:\mathbb{E}[u_1] = x^T A y (for zero-sum games, Column pays x^T A y).
The Indifference Principle:
A rational player randomizes across multiple pure actions if and only if all those actions yield the exact same expected payoff against the opponent’s mixed strategy.
Nash’s Theorem (1950): Every finite game has at least one mixed-strategy Nash equilibrium.
For general games, computing equilibria is PPAD-complete; for zero-sum games, it is in P via Linear Programming!
Two-Player Zero-Sum Games
In a zero-sum game, B = -A. One player’s gain is identically the other’s loss. Let A \in \mathbb{R}^{m \times n} be the payoff matrix for Row (Maximizer).
Row Player (Max): Aims to maximize their worst-case guarantee: \max_{x \in \Delta_m} \min_{y \in \Delta_n} x^T A y
Column Player (Min): Aims to minimize their worst-case payout: \min_{y \in \Delta_n} \max_{x \in \Delta_m} x^T A y
The Core Question: Does the order of commitment matter? Is \max \min = \min \max?
The Weak Minimax Inequality
For pure actions or general functions, committing first is always a disadvantage:
For every row i and column j: \min_{j'} A_{i, j'} \le A_{ij} \le \max_{i'} A_{i', j}.
Thus g(i) \le h(j) for all i, j \implies \max_i g(i) \le \min_j h(j). \quad\blacksquare
The Pure Duality Gap in Matching Pennies:\max_i \min_j A_{ij} = -1 \quad < \quad +1 = \min_j \max_i A_{ij}
A strict duality gap of size 2!
von Neumann’s Minimax Theorem (1928)
What happens to this gap when players can randomize over mixed strategies?
Theorem (John von Neumann, 1928)
For any two-player zero-sum game with m \times n payoff matrix A: \max_{x \in \Delta_m} \min_{y \in \Delta_n} x^T A y = \min_{y \in \Delta_n} \max_{x \in \Delta_m} x^T A y = v^* where v^* \in \mathbb{R} is the unique Value of the Game.
The Power of Convexity:
Randomization completely collapses the duality gap to ZERO!
It makes no difference who announces their mixed strategy first.
Row can guarantee expected gain \ge v^*; Column can guarantee payout \le v^*.
Geometric View: 2 \times n Games
Suppose Row has 2 actions (m=2), so x = (p, 1-p)^T with p \in [0, 1]:
Each column action j produces a linear payoff: f_j(p) = p A_{1j} + (1-p) A_{2j}.
Against Row’s p, Column chooses the best response: \min_j f_j(p) (lower envelope).
Row maximizes this minimum \implies optimal p^* is the highest peak of the lower envelope!
Interpretation: Column Player chooses distribution y \in \Delta_n to minimize their upper-bound payout u across all pure Row actions!
Key Fact: This is mechanically the exact Linear Programming Dual of Row’s Primal LP!
Proving Minimax from Strong LP Duality
Strong Duality Theorem of Linear Programming
If a primal LP is feasible and bounded, its dual LP is also feasible and bounded, and their optimal values are strictly equal: v^* = u^*.
Feasibility Check:
Setting x_i = 1/m and v = \min_j (x^T A)_j is always primal feasible.
Setting y_j = 1/n and u = \max_i (A y)_i is always dual feasible.
Since both LPs are non-empty and bounded, Strong Duality applies!
The Minimax Conclusion:\max_{x \in \Delta_m} \min_{y \in \Delta_n} x^T A y = v^* = u^* = \min_{y \in \Delta_n} \max_{x \in \Delta_m} x^T A y
von Neumann’s Minimax Theorem is an exact algebraic consequence of Strong LP Duality!\quad\blacksquare
Three Algorithmic Paradigms
Before bridging to game theory, let us formalize what an algorithm means in different settings:
1. Deterministic Algorithm (A)
Behavior: Fixed, reproducible execution on any given input x.
Correctness:Never errors. For every valid input x \in \mathcal{I}, A(x) = f(x).
Running Time: If the algorithm has running time T(n), then A terminates in at most T(n) steps on all inputs of size n.
2. Randomized Algorithm (Monte Carlo \mathcal{R})
Formal View: A probability distribution \mathcal{D} over a family of deterministic algorithms \{A_1, A_2, \dots, A_m\}.
Running Time: All algorithms in the support \text{supp}(\mathcal{D}) terminate in at most T(n) steps on all inputs.
Error Bound: For each fixed input x, at most an \varepsilon fraction of the algorithms err: \Pr_{A \sim \mathcal{D}}[A(x) \ne f(x)] \le \varepsilon \quad \forall x \in \mathcal{I}
Distributional (Average-Case) Algorithms
What if the randomness resides in the input, rather than the algorithm?
3. Distributional Algorithm (A_\mu)
Setting: Nature draws inputs from a fixed distribution \mu over the input domain \mathcal{I}.
The Algorithm: The algorithm A itself is purely deterministic.
Running Time:A terminates in at most T(n) steps on all inputsx \in \mathcal{I}.
Error Bound: The \mu-mass of the set of inputs where A makes an error is at most \varepsilon: \Pr_{X \sim \mu}[A(X) \ne f(X)] = \sum_{x \in \mathcal{I} : A(x) \ne f(x)} \mu(x) \le \varepsilon
Key Distinction:
In a Randomized Algorithm, the algorithm flips internal coins; the guarantee holds for every worst-case input.
In a Distributional Algorithm, the input is random; the algorithm is fixed and deterministic, and may completely fail on some inputs as long as their total probability mass is \le \varepsilon.
The Algorithm-Input Matrix & Double Counting
Consider a boolean matrix M where rows are deterministic algorithms and columns are inputs:
Define M(A_i, x_j) = \mathbf{1}[A_i(x_j) = f(x_j)] (1 if correct, 0 if error).
Suppose a randomized algorithm \mathcal{D} has error \le \varepsilon on every input x_j:
In column j, the fraction of zeros (weighted by \mathcal{D}) is \le \varepsilon \implies fraction of ones is \ge 1 - \varepsilon.
Averaging Principle: There must exist at least one specific deterministic algorithm A^* in the support whose average correctness over uniform inputs is \ge 1 - \varepsilon!
This double-counting argument generalizes to any input distribution \mu:
Theorem (Easy Direction via Double Counting)
If there exists a randomized algorithm \mathcal{R} with running time T(n) and worst-case error \le \varepsilon, then for EVERY input distribution \mu, there exists a deterministic algorithm A_\mu with running time T(n) whose distributional error under \mu is at most \varepsilon.
The Power of the Contrapositive:\text{Randomized Alg exists} \implies \forall \mu, \; \text{Distributional Alg exists}\mathbf{\exists \mu \text{ where NO deterministic algorithm achieves error } \le \varepsilon \implies \text{NO randomized algorithm can achieve error } \le \varepsilon!}
Why is this profound?
Proving a lower bound against randomized algorithms seems hard (infinite coin flip sequences).
But to rule out all randomized algorithms, it suffices to construct a single “hard” input distribution \mu and prove that deterministic algorithms fail on it!
Beyond Error: What about Expected Running Time?
When costs are continuous runtimes or query counts rather than binary errors, von Neumann’s Minimax Theorem / Yao’s Minimax Principle gives the exact, quantitative duality!
Algorithm Design as a Zero-Sum Game
Now we import this minimax machinery into theoretical computer science:
Fix an input size n. Consider an adversarial game between two players:
The Complexity Game: Algorithm Designer vs. Adversary
Player 1 (Algorithm Designer — Minimizer):
Chooses an algorithm to solve the computational problem.
Pure actions: The finite set of all deterministic algorithms\mathcal{A} = \{A_1, A_2, \dots, A_m\}.
Player 2 (Nature / Adversary — Maximizer):
Chooses an input instance of size n.
Pure actions: The finite set of all valid inputs\mathcal{I} = \{I_1, I_2, \dots, I_k\}.
Cost Matrix C(A_i, I_j):
Running time, number of comparisons, or query complexity of A_i on input I_j.
Mixed Strategies in the Complexity Game
What do probability distributions mean in this Algorithm vs. Adversary game?
Mixed Strategy for the Algorithm Designer:
A probability distribution p \in \Delta_{\mathcal{A}} over deterministic algorithms.
Choosing a deterministic algorithm from distribution p before running it is precisely a Randomized Algorithm!
Mixed Strategy for the Adversary:
A probability distribution q \in \Delta_{\mathcal{I}} over input instances.
Nature choosing inputs according to q defines a Randomized Input Distribution (average-case complexity)!
Yao’s Minimax Principle (1977)
Apply von Neumann’s Minimax Theorem to the finite cost matrix C:
Left-Hand Side: The worst-case expected cost of the optimal randomized algorithm.
Right-Hand Side: The expected cost of the best deterministic algorithm against the hardest input distributionq^*.
The Equality: Both quantities are identically equal!
The “Hard Distribution” Recipe for Lower Bounds
To prove a lower bound against all randomized algorithms, we only need Weak Duality:
The Core Inequality (Universal)
For any probability distribution q over inputs: \min_{p \in \Delta_{\mathcal{A}}} \max_{I \in \mathcal{I}} \mathbb{E}[C(A, I)] \;\ge\; \min_{A \in \mathcal{A}} \mathbb{E}_{I \sim q}[C(A, I)]
The 3-Step Lower Bound Recipe:
Pick a Clever Distribution q: Choose a tricky distribution over inputs (“Nature’s trap”).
Analyze Arbitrary Deterministic Algorithms: Prove that no deterministic algorithm can achieve expected cost < L on inputs drawn from q.
Apply Yao’s Principle: Deduce that no randomized algorithm can achieve worst-case expected cost < L on all inputs!
Case Study 1: Finding an Item (The Upper Bound)
Problem: An array X[1 \dots n] contains n-1 zeros and exactly one 1. Find the index of the 1 using probing queries.
Randomized Upper Bound: Random Permutation Algorithm
Pick a permutation \pi \in S_nuniformly at random.
Probe indices in order \pi(1), \pi(2), \dots, \pi(n) until finding the 1.
Analysis on ANY Fixed Worst-Case Input:
Let the single 1 be at fixed index k^*.
Since \pi is uniform, k^* appears at position t \in \{1, 2, \dots, n\} with probability exactly 1/n.
Expected Number of Probing Queries:\mathbb{E}[Q] = \sum_{t=1}^n t \cdot \Pr[k^* \text{ at position } t] = \sum_{t=1}^n t \cdot \frac{1}{n} = \frac{n(n+1)}{2n} = \mathbf{\frac{n+1}{2}}
Thus, randomized worst-case expected query complexity is at most \frac{n+1}{2} = O(n).
Case Study 1: Proving the Matching \Omega(n) via Yao
Can any clever randomized algorithm do better than \frac{n+1}{2} queries in the worst case?
Applying the “Hard Distribution” Recipe
Nature’s Hard Distribution q: Place the 1 uniformly at random at index K \in \{1, \dots, n\} with probability 1/n.
Arbitrary Deterministic Algorithm A: Probes a fixed sequence of distinct indices i_1, i_2, \dots, i_n.
Average Cost for ANY Deterministic Algorithm under q:\mathbb{E}_{K \sim q}[Q] = \sum_{t=1}^n t \cdot \Pr[K = i_t] = \sum_{t=1}^n t \cdot \frac{1}{n} = \mathbf{\frac{n+1}{2}}
The bound is exact and tight: \frac{n+1}{2} = \mathbf{\Theta(n)}. Randomization cannot beat \frac{n+1}{2}!
Case Study 2: Game-Tree Evaluation
Evaluate a complete binary NAND Tree of depth d with n = 2^d boolean leaf inputs.
Deterministic Barrier: An adversary can force any deterministic algorithm to read all n = 2^d leaves (\Theta(n)).
Randomized Upper Bound: Snir (1985) gave a randomized algorithm evaluating the tree in O\left(n^{\log_2\left(\frac{1+\sqrt{33}}{4}\right)}\right) \approx O(n^{0.753}) expected queries!
Can any randomized algorithm do better?
NAND Trees: Setting Up the Recurrence
To apply Yao’s Principle, we design a Hard Distribution q and analyze deterministic algorithms:
Step 1: The Golden Ratio Hard Distribution q
Assign each leaf independently to 1 with probability p.
A NAND gate with independent inputs outputs 1 \iff inputs are not(1, 1): \Pr[\text{output} = 1] = 1 - p^2
We set p = 1 - p^2 \implies p^2 + p - 1 = 0 \implies \mathbf{p = \frac{\sqrt{5}-1}{2} \approx 0.618} (or \frac{3-\sqrt{5}}{2} depending on 0/1 convention).
Invariant: Every internal node at every level outputs 1 with this exact same probability p!
Step 2: How Must a Deterministic Algorithm Evaluate a Gate?
Let T(h) be the minimum expected queries needed to evaluate a node of height h.
A deterministic algorithm must pick one child first (say, Left), which costs T(h-1).
If Left child evaluates to 0: The NAND is guaranteed to be 1. We stop! (No need to evaluate Right).
If Left child evaluates to 1: We cannot determine the NAND output yet! We must evaluate the Right child too, incurring an extra T(h-1) cost.
Solving the Recurrence & Yao’s Lower Bound
The Recurrence Relation for Height h
\mathbb{E}[\text{queries at height } h] = T(h-1) + \Pr[\text{first child} = 1] \cdot T(h-1) Because the first child outputs 1 with probability p \approx 0.618: T(h) \ge (1 + p) \cdot T(h-1) = \left(1 + \frac{\sqrt{5}-1}{2}\right) T(h-1) = \left(\frac{1 + \sqrt{5}}{2}\right) T(h-1) For a two-level (depth-2) block, the optimal adversary distribution gives: T(h) \ge \left(\frac{1 + \sqrt{33}}{4}\right) T(h-1) \approx 1.686 \cdot T(h-1)
Unrolling the Recurrence down to the Leaves (h = d = \log_2 n):T(d) \ge \left(\frac{1 + \sqrt{33}}{4}\right)^d = \left(2^{\log_2\left(\frac{1+\sqrt{33}}{4}\right)}\right)^d = \left(2^d\right)^{\log_2\left(\frac{1+\sqrt{33}}{4}\right)} = \mathbf{n^{\log_2(1.686)} \approx \Omega(n^{0.753})}
Step 3: Yao’s Conclusion:
Since this holds for every deterministic algorithm on distribution q: \min_{p \in \Delta_{\mathcal{A}}} \max_{I} \mathbb{E}[C(A, I)] \;\ge\; \min_{A \in \mathcal{A}} \mathbb{E}_{I \sim q}[C(A, I)] = \mathbf{\Omega(n^{0.753})}
No randomized algorithm can ever beat \Omega(n^{0.753}) queries!\quad\blacksquare
Concept Check
Answer each question based on today’s concepts:
To prove a randomized lower bound using Yao’s Minimax Principle, do you need to find the worst-case distribution q^*, or does any distribution q suffice?
Why is von Neumann’s Minimax Theorem equivalent to Strong LP Duality?
Any distribution q suffices! By weak duality, \min_A \mathbb{E}_{I \sim q}[C(A, I)] is a valid lower bound for any choice of q. A cleverer distribution simply gives a tighter bound!
Because Row’s maximum guarantee problem can be written as a Primal LP, whose algebraic dual is identically Column’s minimum payout problem. Strong LP Duality guarantees v^* = u^* without any duality gap!
