Recurrences

• We will be learning two techniques for solving recurrences of the form T(n) = aT(n/b)+f(n).

  • Substitution method

  • Recursion Tree method

Substitution method

In the substitution method, we

• Guess the solution

• Use mathematical induction to find the constants and show that the solution works.

Example-1

Recurrence: T(1)=1 and T(n) = 2T(\lfloor\frac{n}{2}\rfloor)+n for n>1.

• We guess that T(n) = O(n\log{n}).

• Now we need to prove that T(n)\leq cn\log{n} for some constant c.

Proof by induction

• (in class)

Points to remember

• Warning-1: It is easier to prove a weaker bound than the one you are supposed to prove. For instance, if the runtime is O(n), you might still be able to substitute cn^2 into the recurrence and prove that the bound is O(n^2). Although it is technically correct, don’t let it mislead you into thinking it’s the best bound!

• Warning-2: You must prove the exact form of the induction hypothesis. For example, in the recurrence T(n) = 2T(\lfloor\frac{n}{2}\rfloor)+n, we could falsely prove T(n) = O(n) by guessing T(n)\leq cn and then arguing T(n)\leq 2(c\lfloor\frac{n}{2}\rfloor)+n\leq cn+n = O(n). Here we need to prove T(n)\leq cn, not T(n)\leq (c+1)n.

Recursion Tree for solving recurrences:

• A recursion tree is a tree where each node represents the cost of certain recursive subproblem. Then we sum up the numbers in each node to get the cost.

• We will use a recursion tree for to generate possible guesses for the runtime, and then use the substitution method to prove them.

Recursion Tree

Consider the following recurrence:

Let us drop the floors and write the recurrence:

Recursion Tree

image

Recursion Tree

• Suppose the number of levels in this tree is l.

• \frac{n}{4^l} = 1. This implies l = \log_4{n}.

• Number of leaf nodes: 3^{\log_4{n}} = n^{\log_4{3}}.

\begin{aligned} T(n) &= cn^2+\frac{3}{16}cn^2+(\frac{3}{16})^2cn^2+\cdots+(\frac{3}{16})^{\log_4{n}-1}cn^2 + \Theta(n^{\log_4{3}}) \\ &= \displaystyle\sum_{i=0}^{\log_4{n}-1}(\frac{3}{16})^icn^2 + \Theta(n^{\log_4{3}}) \\ &\leq \displaystyle\sum_{i=0}^{\infty}(\frac{3}{16})^icn^2 + \Theta(n^{\log_4{3}}) \\ &= (1-3/16)^{-1}cn^2 + \Theta(n^{\log_4{3}}) \end{aligned}

Since the functions in \Theta(n^{\log_4{3}}) are also in O(n^2), This whole expression is O(n^2). Therefore we can guess that T(n) = O(n^2).

Back to the substitution to prove the bound

$$