BITS Pilani
BITS Logo
CS G526: Advanced Algorithms and Complexity

Applications of Tail Bounds

Lecture 4 |2026-08-10
Tulasimohan Molli
BITS Pilani, Hyderabad Campus

Agenda

  1. Balls into Bins — max load via Chernoff.
  2. Birthday Paradox — collision probabilities.
  3. Coupon Collector — expectation and concentration.
  4. Random Sampling — estimating a mean with additive error.
  5. Which Bound, When — the decision table.
  6. Concept Check — apply the right tool.

Quick Recap: Tail Bounds

We use tail bounds to bound the probability that a random variable X deviates from its expectation \mu = \mathbb{E}[X].

  • Markov’s Inequality: For any non-negative R.V. X and a > 0: \Pr(X \ge a) \le \frac{\mathbb{E}[X]}{a}
  • Chebyshev’s Inequality: For any R.V. X with variance \sigma^2 and k > 0: \Pr(|X - \mu| \ge k) \le \frac{\sigma^2}{k^2}
  • Chernoff Bounds (Upper): For X = \sum X_i sum of independent Bernoulli trials: \Pr(X \ge (1+\delta)\mu) \le \left(\frac{e^\delta}{(1+\delta)^{1+\delta}}\right)^\mu

Balls into Bins: The Experiment

Throw n balls into n bins independently and uniformly at random (i.i.d). Let L_i be the load of bin i.

  • Expected load per bin: \mu = \mathbb{E}[L_i] = 1 \sum_{i=1}^n \mathbb{E}[L_i] = \mathbb{E}\left[\sum_{i=1}^n L_i\right] = \mathbb{E}[n] = n
  • The Question: What is the maximum load of any bin?
  • Deterministic worst case: n (all balls in one bin).
  • Randomized reality: Concentrates heavily around the mean.

Balls into Bins: Chernoff Bound

For the ith bin , L_i = \sum_{j=1}^n X_j is a sum of independent r.v.s.

  • Chernoff Bound Theorem: For a sum of independent Bernoulli trials with expectation \mu, and any \delta > 0: \Pr(X \ge (1+\delta)\mu) \le \left(\frac{e^\delta}{(1+\delta)^{1+\delta}}\right)^\mu
  • Substituting 1+\delta = \lambda (since \mu = \mathbb{E}[L_i] = 1), we get: \Pr(L_i \ge \lambda) \le \left(\frac{e^{\lambda - 1}}{\lambda^\lambda}\right)^\mu = \frac{e^{\lambda - 1}}{\lambda^\lambda} \le \left(\frac{e}{\lambda}\right)^\lambda
  • We want to find a threshold \lambda such that this probability is very small.

Balls into Bins: Setting the Threshold

Let’s choose \lambda = \frac{3 \ln n}{\ln \ln n}.

  • Taking the natural log of our bound: \ln \Pr(L_i \ge \lambda) \le \lambda (1 - \ln \lambda)
  • Substituting \lambda: \ln \Pr(L_i \ge \lambda) \approx -\frac{3 \ln n}{\ln \ln n} (\ln \ln n - \ln 3) \le -2 \ln n
  • This yields: \Pr(L_i \ge \lambda) \le n^{-2}

Balls into Bins: Union Bound

We want to bound the probability that any bin exceeds our threshold \lambda.

  • Applying the Union Bound over all n bins: \Pr(\max_i L_i \ge \lambda) = \Pr\left(\bigcup_{i=1}^n \{L_i \ge \lambda\}\right) \le \sum_{i=1}^n \Pr(L_i \ge \lambda)
  • Substituting our per-bin bound: \Pr(\max_i L_i \ge \lambda) \le n \cdot n^{-2} = \frac{1}{n}
  • Conclusion: With high probability (1 - 1/n), the max load is O\big(\frac{\log n}{\log \log n}\mathrm{\big)}.

Birthday Paradox: The Setting

Throw m balls into n bins (e.g., n = 365 days in a year).

  • The Question: How large does m need to be so that the probability of at least one collision (a bin with load \ge 2) is \ge 1/2?
  • Pigeon hole principle says that if a class has 366 people, two people always have the same birthday.
  • The Reality: We only need m = 23 people!
  • Let’s analyze the math behind this collision rate.

Birthday Paradox: The Analysis

What is the probability of having no collisions (all balls land in distinct bins)?

  • We place the balls one by one. The probability that the i-th ball does not collide with any of the previous i-1 balls is 1 - \frac{i-1}{n}.
  • Therefore, the probability of no collisions after placing m balls is: \Pr(\text{no collision}) = \prod_{i=1}^{m} \left(1 - \frac{i-1}{n}\right) = \prod_{i=1}^{m-1} \left(1 - \frac{i}{n}\right)
  • Using the inequality 1 - x \le e^{-x} for all x: \Pr(\text{no collision}) \le \prod_{i=1}^{m-1} e^{-i/n} = e^{-\frac{1}{n}\sum_{i=1}^{m-1}i} = e^{-\frac{m(m-1)}{2n}}

Birthday Paradox: The Threshold

We want to find m such that the collision probability is at least 1/2 (\Pr(\text{no collision}) \le 1/2).

  • Using our exponential bound: e^{-\frac{m(m-1)}{2n}} \le \frac{1}{2} \iff \frac{m(m-1)}{2n} \ge \ln 2 \iff m(m-1) \ge 2n \ln 2
  • Approximating m(m-1) \approx m^2, this holds when: m \ge \sqrt{2n \ln 2}
  • For n = 365: m \ge \sqrt{730 \ln 2} \approx 22.49 \implies m = 23
  • Application: Hash collision boundaries (e.g., why b-bit hashes collide after 2^{b/2} items).

Coupon Collector: Concentration

Recall the classic Coupon Collector problem: we want to collect n unique coupons.

  • Expected draws to collect all n coupons: nH_n = \Theta(n \log n).
  • The Question: How likely are we to deviate significantly from this expectation?
  • We will show that the probability of needing more draws decays rapidly.

Concept Check

After throwing n balls into n bins, bound the probability that some bin has load > 6 \ln n / \ln\ln n.

Hint: per-bin Chernoff gives \Pr(L_i \ge \lambda) \le n^{-2} for \lambda = 6\ln n/\ln\ln n; union bound over n bins gives \le 1/n.

Next Lecture

Coupon Collector & Random Sampling — concentration details, sample complexity, and comparing bounds.