Quick Quiz: Do you know these?

Before we dive in, let’s test your memory of these key terms:

• Random Experiment
• Sample Space
• Outcome
• Event
• Random Variable

Sample Space and Outcomes

• Sample Space (\Omega): The set of all possible results of a random experiment.

Random Variables

• A Random Variable (RV) is a function X: \Omega \to \mathbb{R} that assigns a real number to each outcome.

• Examples:

  • Number of heads in 3 coin tosses: X \in \{0, 1, 2, 3\}
  • Value shown on a die roll: X \in \{1, 2, 3, 4, 5, 6\}
  • In Quicksort: “number of comparisons between elements i and j”

• Random variables can be discrete (countable values) or continuous.

Expectation

The expected value (or mean) of a discrete random variable X is: E[X] = \sum_{x} x \cdot \Pr(X = x)

Example: Expected value of a fair die roll: E[X] = 1 \cdot \frac{1}{6} + 2 \cdot \frac{1}{6} + 3 \cdot \frac{1}{6} + 4 \cdot \frac{1}{6} + 5 \cdot \frac{1}{6} + 6 \cdot \frac{1}{6} = 3.5

Linearity of Expectation

For any random variables X and Y: E[X + Y] = E[X] + E[Y]

This holds even if X and Y are dependent!

Example: Expected sum of two dice rolls: - E[X_1 + X_2] = E[X_1] + E[X_2] = 3.5 + 3.5 = 7

This property is what makes analyzing randomized algorithms tractable.

Indicator Random Variables

An Indicator Random Variable I_A for an event A is defined as: I_A = \begin{cases} 1 & \text{if event } A \text{ occurs} \\ 0 & \text{otherwise} \end{cases}

The expected value of an indicator variable is simply the probability of the event: E[I_A] = 1 \cdot \Pr(A) + 0 \cdot \Pr(\text{not } A) = \Pr(A)

Classic Example: Expected number of heads in n coin flips:

• Let I_k be indicator that k^{th} flip is heads.
• E[I_k] = \Pr(\text{heads}) = 1/2.
• E[\text{total heads}] = E[\sum I_k] = \sum E[I_k] = n \cdot 1/2 = n/2.

We’ll use this to count the expected number of comparisons in Quicksort!

Randomized Quicksort

• The pivot is chosen uniformly at random from the array.
• This makes the algorithm’s performance independent of the input order.

Key Idea of Analysis

• Focus on the number of comparisons made during partitioning.
• Each pair of elements is compared at most once.
• Let’s use Indicator Random Variables.

Let X_{ij} be the random variable: X_{ij} = \begin{cases} 1, & \text{if elements } i \text{ and } j \text{ are compared} \\ 0, & \text{otherwise} \end{cases}

Total comparisons: X = \sum_{i < j} X_{ij}

Probability of Comparison

Assume the elements are sorted: z_1 < z_2 < \cdots < z_n.

Elements z_i and z_j (where i < j) are compared only if:

• One of them is chosen as the pivot first among \{z_i, z_{i+1}, \dots, z_j\}.
• If any element between them is chosen first, they are never compared!

The number of elements in this set is j - i + 1. Since each is equally likely: \Pr(X_{ij} = 1) = \frac{2}{j - i + 1}

Expected Number of Comparisons

By Linearity of Expectation: \mathbb{E}[X] = \sum_{i < j} \mathbb{E}[X_{ij}] = \sum_{i < j} \Pr(X_{ij} = 1) \mathbb{E}[X] = \sum_{i=1}^{n-1} \sum_{j=i+1}^{n} \frac{2}{j - i + 1}

Expected Number of Comparisons (Continued)

Let k = j - i: \mathbb{E}[X] = \sum_{i=1}^{n-1} \sum_{k=1}^{n-i} \frac{2}{k + 1}

Now swap the order of summation: \mathbb{E}[X] = \sum_{k=1}^{n-1} \sum_{i=1}^{n-k} \frac{2}{k + 1}

Expected Number of Comparisons (Continued)

Simplifying: \mathbb{E}[X] = \sum_{k=1}^{n-1} \frac{2(n-k)}{k + 1}

Bounding: \mathbb{E}[X] \leq 2n \sum_{k=1}^{n-1} \frac{1}{k + 1}

Final Bound

Using the bound for the Harmonic series H_n = \sum_{k=1}^n \frac{1}{k} = O(\log n): \mathbb{E}[X] = O(n \log n)

Expected running time of Randomized Quicksort: \boxed{O(n \log n)}

Exercise: Show that \mathbb{E}[X] = \Theta(n \log n).

Conclusion

• Randomization eliminates bad input cases.
• Expected performance is O(n \log n), which is asymptotically optimal for comparison-based sorting.
• The analysis uses indicator random variables and linearity of expectation — powerful tools for analyzing randomized algorithms.
• Randomized Quicksort is widely used in practice due to its simplicity, in-place nature, and excellent practical performance.