CS G526: Advanced Algorithms and Complexity

Complexity: Introduction to Turing Machines & Diagonalisation

Lectures 23–24 |2026-10-12

Tulasimohan Molli

BITS Pilani, Hyderabad Campus

Agenda

Why Study Complexity? — Mathematical limits, lower bounds & cryptography.
Turing Machines & Encodings — Programs as bit strings \langle M \rangle.
The Universal Turing Machine — Software as data & simulation overhead.
Diagonalization & Time Hierarchy — Proving \mathbf{P} \subsetneq \mathbf{EXP}.

Why Study Computational Complexity?

1. Intrinsic Hardness

Understanding the limits of efficient computation.

Beyond clever algorithm design.
Provable mathematical barriers (Lower Bounds).
Knowing when to stop searching for exact algorithms.

2. Hardness as a Shield

Modern Cryptography relies on hard problems!

Encryption, digital signatures, Zero-Knowledge.
If all problems were in \mathbf{P}, cryptography collapses.
Hardness is a feature, not a bug.

3. Coping with Limits

Mapping the landscape reveals alternatives:

Approximation algorithms.
Heuristics & average-case analysis.
Randomized algorithms.

What is an Algorithm?

The Classical Turing Machine

Infinite memory tape with discrete cells.
Read/write head moving \text{L} or \text{R}.
Finite state control: \delta: Q \times \Gamma \to Q \times \Gamma \times \{\text{L}, \text{R}\}.

The Church–Turing Thesis

Every physically realizable computation can be simulated by a standard Turing machine.

Invariance of the model:
- Multi-tape TMs \longleftrightarrow Single-tape TMs.
- RAM models \longleftrightarrow Multi-tape TMs.
- All simulation slowdowns are polynomial (O(T^2) or O(T \log T)).
The polynomial time scale (\mathbf{P}) is mathematically universal.

Turing Machines as Data

Crucial insight: Code is just data.

Machine Descriptions \langle M \rangle

Every Turing machine M = \langle Q, \Sigma, \Gamma, \delta, q_0, q_{acc}, q_{rej} \rangle is a finite mathematical object.
We can encode its states, alphabet, and transition table \delta as a finite binary string: \langle M \rangle \in \{0, 1\}^*

Effective Enumeration

We can sort all binary strings lexicographically: M_1, M_2, M_3, M_4, \dots
Every string encodes some TM. (Invalid syntax maps to a dummy machine that rejects immediately.)
Every program is an integer; every integer is a program.

The Universal Turing Machine

Can one machine execute any algorithm?

                  ┌──────────────────────────────┐
  <M>, x  ───────►│  Universal Turing Machine U  ├───────► M(x)
                  │                              │
                  │   Tape 1: Code <M>           │
                  │   Tape 2: Simulated Tape     │
                  │   Tape 3: State of M         │
                  └──────────────────────────────┘

Theorem (Turing 1936)

There exists a fixed Universal Turing Machine \mathcal{U} such that for every machine M and input x: \mathcal{U}(\langle M \rangle, x) = M(x) If M(x) halts, \mathcal{U}(\langle M \rangle, x) halts with the identical output. If M(x) loops, \mathcal{U} loops.

Universal Simulation Overhead

How much does universal emulation cost?

Naive Simulation

Scanning transition table in \langle M \rangle on every step: T \text{ steps of } M \implies O(T^2) \text{ steps on } \mathcal{U}

Efficient Simulation (Hennie–Stearns)

Using two tapes and binary search addressing:

Hennie–Stearns Simulation (1966)

If M halts in T steps on input x, a 2-tape Universal TM simulates it in: O(T \log T) \text{ steps}

Takeaway for Complexity

Universal simulation incurs only an almost linear time overhead.
A polynomial runtime remains a polynomial runtime!

Cantor’s Legacy: Diagonalization

How do we prove that certain problems are impossible for any machine?

The Execution Matrix

Consider an infinite grid where:

Rows = All possible Turing machines M_i.
Columns = All binary inputs x_j.
Cell (i, j) = M_i(x_j) (1 if accepts, 0 if rejects/loops).

Machine	x_1	x_2	x_3	x_4	\dots
M_1	0	1	0	0	\dots
M_2	1	1	1	0	\dots
M_3	0	0	0	1	\dots
M_4	1	0	1	1	\dots

Constructing the Anti-Diagonal

Define a new language D: On input x_i, flip the diagonal element! x_i \in D \iff M_i(x_i) \text{ does not accept}

The Diagonal Language Cannot Be Computed

Proof by Contradiction

Suppose some Turing machine M_d decides D.

Because M_d is a Turing machine, it must appear as some row d in our enumeration.
Now, evaluate M_d on its own description x_d = \langle M_d \rangle:

x_d \in D \iff M_d(x_d) \text{ accepts} \quad (\text{by definition of } M_d \text{ deciding } D)

\text{Yet by definition of } D: \quad x_d \in D \iff M_d(x_d) \text{ does not accept!}

\text{Accepts} \iff \text{Does not accept} \quad \implies \mathbf{Contradiction!}

D cannot have any row in the table. D is uncomputable.

From Undecidability to Complexity

Diagonalization proves uncomputability (\text{HALT} \notin \mathbf{R}).
Can we use it inside complexity to show more time buys more computational power?

The Goal

Does granting n^{100} time allow us to solve strictly more problems than n^2 time?

\mathbf{DTIME}(n^2) \stackrel{?}{\subsetneq} \mathbf{DTIME}(n^{100})

The Challenge

A machine with more time must simulate machines with less time, but it must abort them before they exceed their clock!

Clocked Universal Simulation

To diagonalize within time bounds, we introduce a clocked simulation:

Input: <M>, x
  │
  ├─► Start clock for t(|x|) steps
  │
  ├─► Universal Simulator U runs M(x)
  │
  ├─── If M(x) halts within t(|x|) steps:
  │      Output OPPOSITE of M(x)
  │
  └─── If clock expires before M halts:
         REJECT and abort

Because the clock enforces a hard deadline, the diagonalizer always halts.
Therefore, the language it decides is guaranteed to be in a slightly larger time class!

The Time Hierarchy Theorem

Time Hierarchy Theorem (Hartmanis & Stearns 1965)

For any time-constructible functions t_1(n) and t_2(n) satisfying: t_1(n) \log(t_1(n)) = o(t_2(n)) we have: \mathbf{DTIME}(t_1(n)) \subsetneq \mathbf{DTIME}(t_2(n))

Concrete Separations

\mathbf{DTIME}(n) \subsetneq \mathbf{DTIME}(n^2)
\mathbf{DTIME}(n^2) \subsetneq \mathbf{DTIME}(n^3)
\mathbf{DTIME}(n^k) \subsetneq \mathbf{DTIME}(n^{k+1}) for all integers k \ge 1.

Landmark Consequence: P vs. EXP

What does the Time Hierarchy imply about polynomial time vs. exponential time?

\mathbf{P} = \bigcup_{k \ge 1} \mathbf{DTIME}(n^k) \qquad \text{vs.} \qquad \mathbf{EXP} = \bigcup_{c \ge 1} \mathbf{DTIME}(2^{n^c})

Corollary: P is Strictly Contained in EXP

\mathbf{P} \subsetneq \mathbf{EXP}

Proof Intuition

Pick t_1(n) = n^{\log n} or 2^n.
Every polynomial n^k satisfies n^k \log(n^k) = o(2^n).
By the Time Hierarchy Theorem, there exist problems decidable in O(2^n) time that no polynomial time algorithm can ever solve.

The Power and Limits of Diagonalization

Diagonalization is a profound tool, but it has a fundamental boundary:

Why Diagonalization Works

Relies only on two basic properties:
1. Programs can be encoded as strings (\langle M \rangle).
2. Universal simulation with bounded overhead (\mathcal{U}).
Works identically with any external oracle: \mathbf{DTIME}^A(t_1) \subsetneq \mathbf{DTIME}^A(t_2)

Why It Cannot Resolve P vs NP

The Relativization Barrier (Baker–Gill–Solovay 1975):
- There exists oracle A where \mathbf{P}^A = \mathbf{NP}^A.
- There exists oracle B where \mathbf{P}^B \ne \mathbf{NP}^B.
Any pure diagonalization proof relativizes!

Summary & Next Steps

Today’s Takeaways

Turing machines are finite strings; UTMs execute software with O(T \log T) overhead.
Diagonalization flips the execution matrix to produce uncomputable languages.
Adding a clock yields the Time Hierarchy Theorem: time strictly buys power.
\mathbf{P} \subsetneq \mathbf{EXP} is proven unconditionally!

Coming Up Next (Lecture 24)

What happens when machines can make non-deterministic guesses?
Certificates vs. Non-Deterministic Turing Machines.
Polynomial-time verifiers and the formal definitions of \mathbf{P}, \mathbf{NP}, and reductions.