Recap: Dynamic Programming so far

Lecture 10 introduced DP — two key ideas:

Both rely on optimal substructure and overlapping subproblems — we’ll see the same pattern in today’s new problems.

Where does the name come from?

• In the 1950s, Richard Bellman was developing a theory for multistage decision processes at the RAND Corporation.

• The Secretary of Defense, Charles Wilson, had “a pathological fear and hatred of the word, research” — he would turn red and get violent if anyone used it in his presence.

• RAND was employed by the Air Force, which Wilson headed. Bellman needed a name that described time-varying, multistage optimization — but wouldn’t trigger Washington.

Matrix Multiplication Review

Matrix: An p\times q array of numbers with p rows and q columns.

Product: C = A_{p\times q} \times B_{q\times r} produces a p\times r matrix where: c[i,j] = \sum_{k=1}^{q} a[i,k]\cdot b[k,j]

Cost: Each of the pr entries needs \Theta(q) operations → \Theta(pqr) total.

Parenthesization Matters

Matrix multiplication is associative: (AB)C = A(BC) — the result is the same regardless of grouping.

However, the cost is not the same!

Matrix Chain Multiplication — Problem

Given: n matrices A_1,\dots,A_n with dimensions p_{i-1}\times p_i.

Goal: Find the parenthesization of A_1A_2\cdots A_n that minimizes the number of scalar multiplications.

Key insight: Parenthesization matters — costs can differ by 2\times or more!

Why Exhaustive Search Fails

The number of distinct parenthesizations of n matrices is the Catalan number: C_{n-1} = \frac{1}{n}\binom{2n-2}{n-1} \;\sim\; \frac{4^{n-1}}{(n-1)^{3/2}\sqrt{\pi}}

• n=10: 4,862 — slow but manageable
• n=20: 1.8 \times 10^9 — brute-force infeasible
• n=30: 3.8 \times 10^{15} — exceeds CPU speeds

Recurrence: P(1)=1, P(n)=\sum_{k=1}^{n-1} P(k)\,P(n-k)

DP approach: O(n^3) — polynomial instead of exponential!

DP: Optimal Substructure

Key idea: In the optimal parenthesization of A_i A_{i+1}\cdots A_j, the last multiplication splits at some k: A_{i\dots j} = A_{i\dots k}\; A_{k+1\dots j},\qquad \text{each optimally parenthesized}

• The sub-chains A_{i\dots k} and A_{k+1\dots j} must themselves be optimally parenthesized (cut-and-paste argument).
• Result A_{i\dots j} has dimension p_{i-1}\times p_j.

DP: The Recurrence

Let m[i,j] = minimum cost to compute A_{i\dots j}. We try all possible split points k:

m[i,j] = \begin{cases} 0 & i=j,\\[6pt] \min\limits_{i\le k<j}\big(m[i,k] + m[k+1,j] + p_{i-1}p_kp_j\big) & i<j \end{cases}

DP: Table Structure

We fill m[i,j] for increasing chain length l = 2, 3, \dots, n:

l=2:   m[1,2]  m[2,3]  m[3,4]  ...
l=3:   m[1,3]  m[2,4]  m[3,5]  ...
l=4:   m[1,4]  m[2,5]  m[3,6]  ...
...
l=n:   m[1,n]

Complexity Analysis

DP vs. Exhaustive Search:

Why O(n^3)?

1. Outer loop: chain length l = 2 to n — O(n)
2. Middle loop: start index i — O(n)
3. Inner loop: split point k — O(n)
4. Constant work per (i,k) pair

Key takeaway: Exponential \to polynomial — DP turns an infeasible problem into a practical one.

DP Complexity Progression

MCM introduces a fundamentally more complex DP structure:

Interval DP problem: subproblems are contiguous intervals [i,j], and we try all split points k. The extra loop over k adds a third dimension to the computation — this is a powerful pattern for many problems (Optimal BST, polygon triangulation, etc.).

Reconstructing Parenthesization

We store s[i,j] = the optimal split k for A_{i\dots j}. Recurse to print:

Print-Parens(s, i, j):
    if i == j: print "A" + i
    else:
        print "("
        Print-Parens(s, i, s[i,j])
        Print-Parens(s, s[i,j]+1, j)
        print ")"