BITS Pilani
BITS Logo
CS G526: Advanced Algorithms and Complexity

Applications of Tail Bounds II

Lecture 5 |2026-08-11
Tulasimohan Molli
BITS Pilani, Hyderabad Campus

Agenda

  1. Coupon Collector — expectation and concentration.
  2. Random Sampling — estimating a mean.
  3. Which Bound, When — comparing tail bounds.

Coupon Collector: Problem Setup

The Experiment: We draw from n distinct coupon types uniformly at random with replacement.

  • The Random Variable: Let T be the total number of trials needed to collect at least one coupon of each of the n types.
  • Recall the Expectation: We showed that \mathbb{E}[T] = n H_n \approx n \ln n.
  • The Goal (Concentration): How likely is T to exceed its expectation significantly?
    • Specifically, we want to bound the tail probability: \Pr(T > t) for t = c n \ln n (where c > 1).

Coupon Collector: Analyzing the Tail

To bound \Pr(T > c n \ln n), let’s examine what must happen after t = c n \ln n trials.

  • Key Equivalence: The event \{T > t\} occurs if and only if at least one coupon type is still missing after t trials.

  • Let M_i be the event that coupon type i is missing after t trials: \{T > t\} = \bigcup_{i=1}^n M_i

  • Probability of a Single Miss: In each trial, the probability of not picking type i is 1 - 1/n. By independence of trials: \Pr(M_i) = \left(1 - \frac{1}{n}\right)^{t} = \left(1 - \frac{1}{n}\right)^{c n \ln n} \le e^{-\frac{c n \ln n}{n}} = n^{-c}

Coupon Collector: Union Bound

Now, we apply the Union Bound over all n possible missing coupons to bound the total failure probability:

  • \Pr(T > c n \ln n) = \Pr\left(\bigcup_{i=1}^n M_i\right) \le \sum_{i=1}^n \Pr(M_i) \le n \cdot n^{-c} = n^{1-c}

  • Example (c=2): The probability of needing more than 2n\ln n trials is: \Pr(T > 2n\ln n) \le \frac{1}{n} = o(1)

  • Conclusion: The total trials T is highly concentrated around its expectation n\ln n.

Random Sampling: Estimating a Mean

We want to estimate an unknown parameter p \in [0, 1] (e.g., fraction of biased coins) within error \varepsilon with failure probability \le \delta.

  • We draw m i.i.d. samples X_1, \dots, X_m \sim \text{Bernoulli}(p).
  • Compute the sample mean \hat{p} = \frac{1}{m}\sum_{j=1}^m X_j.
  • Let’s compare Chebyshev and Chernoff bounds for this estimation.

Mean Estimation: Real-World Applications

Why do we care about estimating p using random samples?

  • Opinion Polls & Surveys:
    • Estimating the fraction p of the population supporting a policy/candidate.
    • Example: Setting \varepsilon = 0.03 (3% margin of error) and \delta = 0.05 (95% confidence).
  • Randomized Audits & Quality Control:
    • Inspecting a small random sample of factory goods to bound the defect rate.
  • Randomized Algorithms (Volume Estimation)

Sampling: Chebyshev vs Chernoff Bounds

Chebyshev Bound

  • Uses variance: \mathrm{Var}(\hat{p}) \le \frac{1}{4m}.
  • By Chebyshev: \Pr(|\hat{p} - p| \ge \varepsilon) \le \frac{1}{4m\varepsilon^2}
  • Setting failure probability \le \delta: m \ge \frac{1}{4\varepsilon^2 \delta}

Chernoff Bound

  • Uses Hoeffding’s inequality: \Pr(|\hat{p} - p| \ge \varepsilon) \le 2e^{-2m\varepsilon^2}
  • Setting failure probability \le \delta: m \ge \frac{1}{2\varepsilon^2} \ln \left(\frac{2}{\delta}\right)

Sampling: Sample Complexity Comparison

Why is the difference between Chebyshev and Chernoff bounds so critical?

  • Chebyshev: m \ge \Omega\big(\frac{1}{\varepsilon^2 \delta}\mathrm{\big)} — sample size grows linearly with confidence 1/\delta.
  • Chernoff: m \ge \Omega\big(\frac{\log(1/\delta)}{\varepsilon^2}\mathrm{\big)} — sample size grows only logarithmically with confidence!
  • Takeaway: For high confidence (e.g., \delta = 10^{-6}), Chernoff requires orders of magnitude fewer samples.

Which Bound, When?

Situation Tool
Any non-negative RV, need a quick bound Markov
Any RV, deviation from mean, know variance Chebyshev
Sum of independent trials, want exponential tail Chernoff
Complicated union of many bad events Union bound on top of any of these
  • Worked example: max load of n balls into n bins — union bound + Chernoff on each bin.

Next Lecture

Las Vegas vs Monte Carlo — the two flavors of randomized algorithms, and amplification by repetition.