Fibonacci Numbers: A Recursive Approach

The Fibonacci sequence is defined by the recurrence: F_n = \begin{cases} 0 & \text{if } n=0\\ 1 & \text{if } n=1\\ F_{n-1} + F_{n-2} & \text{otherwise} \end{cases}

A direct recursive translation to compute the n^{\text{th}} Fibonacci number:

function rFib(n) {
    if (n <= 1) return n;
    return rFib(n - 1) + rFib(n - 2);
}

Complexity of Recursion

Let T(n) be the number of operations to compute F_n: T(n) = T(n-1) + T(n-2) + 1 T(0) = T(1) = 1

The solution to this recurrence is: T(n) = O(\phi^n) \quad \text{where } \phi = \frac{\sqrt{5}+1}{2} \approx 1.618

• The algorithm runs in exponential time!
• There is a massive amount of redundant computation.

Recursion Tree

Fibonacci Numbers: An Iterative Approach

Instead of top-down recursion, we can compute solutions bottom-up and store them in an array:

function iterativeFib(n) {
    let F = new Array(n + 1);
    F[0] = 0;
    F[1] = 1;
    for (let i = 2; i <= n; i++) {
        F[i] = F[i - 1] + F[i - 2];
    }
    return F[n];
}

• Time Complexity: O(n) – linear time!
• Space Complexity: O(n) (which can be optimized to O(1)).

The Core Philosophy & Properties of Dynamic Programming

The Core Philosophy: Solve each subproblem exactly once and store the result to avoid redundant calculations—trading space for time.

Two Essential Properties

1. Optimal Substructure: An optimal solution to the problem can be constructed efficiently from the optimal solutions of its subproblems.

2. Overlapping Subproblems: The recursive search space contains the same subproblems repeatedly, rather than generating new independent subproblems at each step.

Text Similarity

• Biological applications often need to compare the DNA of two (or more) different organisms.
• Eg: The DNA of an organism may be S_1 = \text{ACCGGTCGAGTGCGCGGAAGCCGGCCGAA}, and the DNA of another organism may be S_2 = \text{GTCGTTCGGAATGCCGTTGCTCTGTAAA}.

Subsequence

Given a sequence X = \langle x_1, x_2, \ldots, x_m \rangle, another sequence Z = \langle z_1, z_2, \ldots, z_k \rangle is a subsequence of X if there exists a strictly increasing sequence \langle i_1, i_2, \ldots, i_k \rangle of indices of X such that for all j = 1, 2, \ldots, k, we have x_{i_j} = z_j.

Common Subsequence

Given two sequences X and Y, we say that Z is a common subsequence of X and Y if Z is a subsequence of both X and Y.

Longest Common Subsequence Problem

Given two sequences X = \langle x_1, x_2, \ldots, x_m \rangle and Y = \langle y_1, y_2, \ldots, y_n \rangle, the goal is to find a maximum length common subsequence of X and Y.

DP Formulation for LCS

• The DP formulation is based on pairs of prefixes.
• A prefix of X = \langle x_1, \ldots, x_n \rangle is X_i = \langle x_1, \ldots, x_i \rangle. X_0 is the empty sequence.
• Let \text{lcs}(i, j) denote the length of the longest common subsequence of X_i and Y_j.

Observations – Case 1

If x_i = y_j, then \text{lcs}(i, j) = \text{lcs}(i-1, j-1) + 1.

Observations – Case 2

If x_i \neq y_j, then \text{lcs}(i, j) = \max(\text{lcs}(i-1, j), \text{lcs}(i, j-1)).

DP Recurrence

\text{lcs}(i, j) = \begin{cases} 0 & \text{if } i = 0 \text{ or } j = 0 \\ \text{lcs}(i-1, j-1) + 1 & \text{if } x_i = y_j \\ \max(\text{lcs}(i-1, j), \text{lcs}(i, j-1)) & \text{if } x_i \neq y_j \end{cases}

DP Algorithm – Top-Down Memoized Version

DP Algorithm – Bottom-Up Tabulation Version

Extracting the LCS

The algorithms above compute only the length of the LCS. We save extra direction flags (hints) in a table b[1..m, 1..n] to recover the actual sequence.

Three flags maintained during the LCS algorithm:

1. \nwarrow : Add x_i (= y_j) to the LCS and continue with \text{lcs}(i-1, j-1).
2. \uparrow : Skip x_i and continue with \text{lcs}(i-1, j).
3. \leftarrow : Skip y_j and continue with \text{lcs}(i, j-1).

Reconstructing the LCS (Backtracking)

We backtrack from the bottom-right corner b[m, n] of the table to the start (i=0 or j=0):

function printLCS(b, X, i, j) {
    if (i === 0 || j === 0) return;
    
    if (b[i][j] === "↖") {
        printLCS(b, X, i - 1, j - 1);
        print(X[i]); // Output character (part of LCS)
    } else if (b[i][j] === "↑") {
        printLCS(b, X, i - 1, j);
    } else {
        printLCS(b, X, i, j - 1);
    }
}

• Time Complexity: O(m + n) – we decrement at least one index (i or j) in each step.

Example

References

• Michael T. Goodrich and Roberto Tamassia, Algorithm Design: Foundations, Analysis, and Internet Examples, Wiley Student Edition.
• Jon Kleinberg and Eva Tardos, Algorithm Design, Pearson Publishers.
• Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein, Introduction to Algorithms, MIT Press.
• Sanjoy Dasgupta, Christos Papadimitriou, and Umesh Vazirani, Algorithms, McGraw-Hill.