Lectures

Following is an outline of lectures given along with references and links to additional reading.

\( \def\PAL{\rm PAL} \def\HP{\rm HP} \def\MP{\rm MP} \def\EQ{\rm EQ} \def\USELESS{\rm USELESS} \def\EMPTY{\rm EMPTY} \def\DTIME{\sf DTIME} \def\NTIME{\sf NTIME} \def\DSPACE{\sf DSPACE} \def\NSPACE{\sf NSPACE} \def\P{\sf P} \def\co{\sf co} \def\NP{\sf NP} \def\LOG{\sf LOG} \def\NLOG{\sf NLOG} \def\PSPACE{\sf PSPACE} \def\NPSPACE{\sf NPSPACE} \def\EXP{\sf EXP} \)

Lec 1 (31 Jul, Wed) : (10-10.50 AM) Administrative information. A broad definition of computation. Examples. Role of theory of computing in sciences. Motivation for a study via two questions - (1) performing multiplication via table look up efficiently and (2) Ruler and Compass construction - Constructible numbers.

^{Reading : A detailed account of the connections between Theory of Computation to mathematics and other sciences is beautifully described in this book. Highly recommended !}

Lec 2 (31 Jul, Wed) : (2-2.50 PM) Answering which numbers are constructible via ruler and compass constructions - details of performing arithmetic operations and computing square roots. Impossibility of computing cube roots (statement only). Historical context behind the study – contributions from Hilbert, Godel and Turing in formalizing the notion of effective computation.

^{Reading : Lec 1 [Kozen].}

Theme: Finite automata and Regular Languages

Lec 3 (01 Aug, Thu) : Recap, course overview and course objectives. Describing decision problems. Formal way of capturing decision problems via languages - Alphabets, Strings. Equivalence of decision problems and languages.

^{Reading : Lec 2,3 [Kozen].}

Lec 4 (02 Aug, Fri) : Informal introduction to finite automata via modelling the functionality of an air conditioner. Sample problems ((1) checking if string is of even length, (2) has even number of ones, (3) only Boolean alphabets) and expressing them formally as languages and obtaining finite automata for them. Discussion on common issues/mistakes while describing finite automata for languages.

^{Reading : Lec 3 [Kozen].}

Lec 5 (07 Aug, Wed) : (10-10.50 AM) Multiples of $3$ - over unary and binary alphabets. Representation as two languages along with description of finite automata for the two languages. Need for formal description and proof of correctness for the binary case.

^{Reading : Lec 3,4 [Kozen]}

^{Additional References : Chap 2, Section 2.1 [Martin]}

Lec 6 (07 Aug, Wed) : (2-2.50 PM) Formal definition of deterministic finite automata (DFA) as $5$-tuple. Meaning of determinism. Transition function and extended transition function. Defining the language accepted by a deterministic finite automata. Proof of correctness for DFA accepting strings over unary whose length is a multiples of $3$.

^{Reading : Lec 3,4 [Kozen]}

^{Additional References : Chap 2, Section 2.1 [Martin]}

Lec 7 (08 Aug, Thu) : (Non-evaluated) Tutorial session 1

Lec 8 (09 Aug, Fri) : Completed the proof of correctness of DFA accepting binary strings whose decimal value is a multiple of $3$. Product construction - an example and formal description. Using product construction for union and intersection of languages accepted by DFAs. More operations – Concatenation and reversal. Discussed the mind blocks in designing DFAs for concatenation of two languages and reversal of a language.

^{Reading : Lec 4 [Kozen]}

Lec 9 (14 Aug, Wed) : Choice automata - introduction and worked out example - binary string whole third symbol from the right is a $1$. Working of a choice automata via choice trees. Condition for a choice automata to accept and reject. Informal introduction to non-determinism.

^{Reading : Lec 5 [Kozen]}

Lec 10 (14 Aug, Wed) : Choice automata for strings of the form $wc$ where $w \in \Sigma^*$ and $c \in \Sigma$ with $\Sigma = \{0,1,2\}$. Informal argument for correctness. Formal definition of Non-deterministic Finite automata (NFA).

^{Reading : Lec 5 [Kozen]}

Lec () (15 Aug, Thu) : Holiday due to Independence Day.

Lec 11 (16 Aug, Fri) : Two more examples of NFA construction. Arguing correctness of NFA accepting a language. Non-determinism in computation and its difference from randomness, parallelism and brute-force search. Illustrating non-determinism via run of a Coffee machine.

^{Reading : Lec 5 [Kozen]}

^{Additional References : Chap 3, Section 3.2 [Martin]}

Lec 12 (21 Aug, Wed) : Formal description of NFA for concatenation, reversal and argued correctness. Making non-deterministic finite automata deterministic - the subset construction. Idea behind the construction using pebbles.

^{Reading : Lec 5, 6 [Kozen]}

^{Additional References : Chap 3, Section 3.3 [Martin]}

Lec 13 (21 Aug, Wed) : Details of the Subset construction and a worked out example. Star closure of a language. NFA for constructing star closure. Introduction to regular languages.

^{Reading : Lec 6 [Kozen]}

^{Additional References : Chap 3, Section 3.3 [Martin]}

Lec 14 (22 Aug, Thu) : Tutorial 1 (Lectures 1 to 11)

Lec 15 (23 Aug, Fri) : Subset construction for handling $\epsilon$ transitions in NFA and taking epsilon-closure. Regular languages - definition and worked out examples. Regular expressions and Kleene Algebra.

^{Reading : Lec 7, 8 [Kozen]}

Lec 16 (28 Aug, Wed) : (10-10.50 AM) Rules of Simplification. Using the rules to show that for any regular expression $\alpha$, $\alpha^*\alpha^*\equiv \alpha^*$. Arden’s Theorem (statement only). Using Arden’s theorem to obtain a regular expression from DFA. Worked out example for obtaining regular expression for equal number of $10$ and $01$.

^{Reading : Lec 9 [Kozen]}

Lec 17 (28 Aug, Wed) : (2-2.50 PM) Obtaining NFA from a regular expression. Regular languages are precisely the languages accepted by DFAs. Worked out examples. Designing NFAs with lesser states - worked out examples. Writing regular expression directly for complement of languages - two worked out examples.

^{Reading : Lec 9 [Kozen]}

Lec 18 (29 Aug, Thu) : Tutorial 2 (Lectures 12 to 15)

Lec 19 (30 Aug, Fri) : Kleene’s method of obtaining a regular expression from an NFA. A worked out example. Connection to transitive closure. Summary of closure properties of DFA, NFA and regular expression for operations - union, intersection, complementation, concatenation, star closure and reversal.

^{Reading : Lec 9 [Kozen]}

Lec 20 (04 Sep, Wed) : Test 1 (8 – 8.50 AM)

^{Reading :}

Lec 21 (04 Sep, Wed) : Limitations of finite automata. Pumping Lemma – statement and proof overview. Statement of Pumping lemma in the contra-positive form.

^{Reading : Lec 11 [Kozen]}

Lec 22 (05 Sep, Thu) : Pumping lemma as a game against an adversary. Using game formulation of pumping lemma to show non-regularity of languages. Two worked out examples.

^{Reading : Lec 12 [Kozen]}

Lec 23 (06 Sep, Fri) : The set $\{a^{n!} \mid n \ge 0\}$ is not regular. Using closure properties to show non-regularity of languages. Regularity as well as non-regularity need not hold under subset inclusion.

^{Reading : Lec 12 [Kozen]}

Lec 24 (11 Sep, Wed) : (10-10.50 AM) Minimizing the number of states of a DFA. DFA minimization algorithm. Criteria for collapse of states. The collapsing algorithm, runtime and correctness. A worked out example. Minimal automata for a regular language and its uniqueness. Towards Myhill-Nerode relations and connection to minimization.

^{Reading : Lec 13, 14 [Kozen]}

Lec 25 (11 Sep, Wed) : (2-2.50 PM) Criteria for collapse of states is an equivalence relation. Partitions and Equivalence relations - every equivalence relation induces a partition and every partition gives rise to an equivalence relation.

^{Reading : Lec 13, 14 [Kozen]}

Lec 26 (12 Sep, Thu) : Obtaining an partition from a DFA. Defining equivalence relations based on the partition. Motivation for the definition of refinement and right congruence. Definition of a Myhill-Nerode relation. Describing a Myhill-Nerode relation from a DFA with no inaccessible states.

^{Reading : Lec 15 [Kozen]}

Lec 27 (13 Sep, Fri) : Discussion on Test 1.

^{Reading :}

Lec 28 (18 Sep, Wed) : (10-10.50 AM) Formal description of Myhill-Nerode relation from DFA (with no inaccessible states). Index of a Myhill-Nerode relation. The Myhill-Nerode relation from DFA is of finite index. Converse holds - from a Myhill-Nerode relation of finite index to a DFA A worked out example and a formal description. Comparing Myhill-Nerode relations.

^{Reading : Lec 15, 16 [Kozen]}

Lec 29 (18 Sep, Wed) : (2-2.50 PM) Comparing Myhill-Nerode relations - examples. Definition of the coarsest Myhill-Nerode relation $\equiv_L$ for a language $L$. Proof that $\equiv_L$ is indeed the coarsest for any language $L$. Coarsest Myhill-Nerode relation for $L$ and minimal sized DFA for $L$.

^{Reading : Lec 15, 16 [Kozen]}

Lec 30 (19 Sep, Thu) : Tutorial 3 (Lectures 21 to 26)

Lec () (20 Sep, Fri) : Class cancelled due to IAC 6.0. To be compensated.

Lec 31 (25 Sep, Wed) : (10-10.50 AM) Finding all the equivalence classes corresponding to $\equiv_L$ (coarsest Myhill-Nerode relation) for $L = (0+1)^*01$.

^{Reading : Class notes}

Lec 32 (25 Sep, Wed) : (2-2.50 PM) Uses of coarsest Myhill-Nerode relation. (1) Proving non-regularity of languages using by showing that the coarsest Myhill-Nerode relation is not finite index. Two worked out examples. (2) Proving non-regularity of languages (3) Coarsest Myhill-Nerode relation $\equiv_L$ is same as the Myhill-Nerode relation that is obtained from a minimal DFA for a regular $L$. Completed proof of uniqueness of minimal DFA for a regular language. (4) Using coarsest Myhill-Nerode relation for $\equiv_L$ to show lower bounds on the size of any DFA for a regular $L$.

^{Reading : Lec 16 [Kozen]}

Lec 33 (26 Sep, Thu) : Non-evaluated Tutorial session 2. Also discussed optimality of product construction and subset construction by exhibiting languages and showing lower bounds the number of equivalence classes for the coarsest Myhill-Nerode relation for those languages.

Theme: Computability

Lec 34 (27 Sep, Fri) : Review of models of computation. Introduction to Turing machines. Informal definition of Turing machines. A brief History - Hilbert’s formalist program and the Entscheidungsproblem. Justification for the model and a formal definition.

^{Reading : Lec 28 [Kozen], Turing’s original paper is very well written and worth a read ! An intuitive justification for the Turing model appears in Section 9 of the paper. For an inspiring talk on Alan Turing and his works, check this video.}

Lec 35 (02 Oct, Wed) : (Compensatory lecture for September 20) Turing machines and need for the model. Encoding a Turing machine as string, Configurations, run of a Turing machine - accepting and rejecting configurations. Language accepted by a Turing machine.

^{Reading : Lec 28, 29 [Kozen]}

Lec 36 (03 Oct, Thu) : Tutorial 4 (Lecture 26 to 33)

Lec 37 (04 Oct, Fri) : Total and partial Turing machines. Decidable and Semi-decidable languages. Examples of decidable problems. Turing machines as strings. Turing machines as input to Turing machines. Universal Turing machine. Undecidable problems. Halting problem ($\HP$) and Membership problem ($\MP$) are semi-decidable.

^{Reading : Lec 30, 31 [Kozen]}

Lec 38 (09 Oct, Wed) : (10-10.50 AM) Proving undecidability of Halting problem via diagonalization. Intuition behind the proof. Description of machine $N$ and the final contradiction argument.

^{Reading : Lec 31 [Kozen]}

Lec 39 (09 Oct, Wed) : (2-2.50 PM) Closure properties of decidable languages under union, intersection and complementation. Closure properties of semi-decidable languages under union and intersection. Language is decidable if and only if the language and its complement are semi-decidable. The complement of $\HP$ is not even semi-decidable. Towards answering decidability/undecidability of the two problems - (1) checking if two programs behave the same way ($\EQ$) and (2) does a program have inaccessible code ($\USELESS$).

^{Reading : Lec 32 [Kozen], Section 9.1 [Martin]}

Lec 40 (10 Oct, Thu) : Reductions. A toy problem to understand reductions - $\HP_{42}$. Using reduction to show undecidability of $\HP_{42}$.

^{Reading : Lec 33 [Kozen]}

Lec () (11 Oct, Fri) : Holiday due to Dussehra.

Lec 41 (16 Oct, Wed) : (10-10.50 AM) Completed the proof of undecidability of $\HP_{42}$. Reductions - formal definition and notation $\le_m$. Undecidability and implication of reduction between languages.

^{Reading : Lec 33 [Kozen], Section 9.2 [Martin]}

Lec 42 (16 Oct, Wed) : (2-2.50 PM) Argued $\HP \le_m \EQ$ and $\overline{\HP} \le_m \USELESS$. Showed $\HP$ and $\USELESS$ are undecidable. Checking emptiness of the language accepted by a Turing machine is undecidable.

^{Reading : Lec 33 [Kozen], Section 9.3 [Martin]}

Lec 43 (17 Oct, Thu) : Argued undecidability of $\EMPTY$. Defined classes $SD$ (semi-decidable languages), $coSD$ (complement of languages that are semi-decidable) and $D$ (decidable languages). Argued that $\EMPTY$ is in $coSD$. Showed that $D = SD \cap coSD$. Notion of hardness and completeness for $SD$.

^{Reading : Lec 33, Supplementary lecture J (page 278) [Kozen], Notes by Watrous.}

Lec 44 (18 Oct, Fri) : Property of machines and Property of semi-decidable languages. Examples. Trivial and non-trivial properties. Rice’s Theorem - statement and proof idea.

^{Reading : Lec 34 [Kozen], Section 9.3 [Martin]}

Lec 45 (23 Oct, Wed) : Completed the proof of Rice’s Theorem. Decidability of problems on Turing machines.

^{Reading : Lec 34 [Kozen]}

Lec 46 (23 Oct, Wed) : Variants of Turing machines - multi-tape Turing machines, two way infinite tape Turing machines, two dimensional Turing machines and their equivalence to one-tape deterministic Turing machines. Introduction to non-deterministic Turing machines. A motivating example via closure properties of decidable languages under concatenation.

^{Reading : Lec 30 [Kozen], Notes by Watrous.}

Lec 47 (24 Oct, Thu) : Formal definition of a non-deterministic Turing machine. Examples. Visualizing the run of a non-deterministic Turing machine via choice trees. Configuration of a non-deterministic Turing machine on an input. Configuration graphs. Simulating a non-deterministic Turing machine deterministically - proof idea.

^{Reading : See the notes by Watrous.}

Theme: Resource Bounded Computation

Lec 48 (25 Oct, Fri) : Details of deterministic simulation of a non-deterministic Turing machine. Revisit completeness - showed that $\HP$ is $SD$-complete. Introduction to resource bounded computation. Definition of a resource - Blum’s axioms. Two resources - Time and Space.

^{Reading : See class notes.}

Lec 49 (29 Oct, Tue) : Tutorial 5 (Lecture 39 to 44)

Lec 50 (30 Oct, Wed) : Argument that time and space are resources according to Blum’s axioms. Space bounded machines and bound on the number of configurations. Definition of $\DTIME$ and $\DSPACE$. Examples of solvable in $\DTIME(n)$ and $\DTIME(n^2)$ where $n$ is the input length. Argued $DTIME(t) \subseteq \DSPACE(t)$.

^{Reading : Class notes and the notes by Watrous (you may skip section 19.2 from the pdf link)}

Lec 51 (30 Oct, Wed) : For $s(n) \ge \log n$ argued that $\DSPACE(s(n)) \subseteq \DTIME(2^{O(s(n))})$. Defined complexity classes $\LOG$, $\P$, $\PSPACE$, $\EXP$. Showed that $\LOG \subseteq \P \subseteq \PSPACE \subseteq \EXP$. Examples problems - multiplication (observed that high school multiplication is efficient). Also, checking non-primality and graph isomorphism problems are in $\EXP$ but can be solved in polynomial time with non-determinism.

^{Reading : Class notes and the notes by Watrous.}

Lec () (31 Oct, Thu) : Holiday due to Diwali.

Lec () (01 Nov, Fri) : Non-instructional day.

Lec 52 (06 Nov, Wed) : (10-10.50 AM) Showed that ANTAKSHARI is in $\PSPACE$. Definition of time and space when computation is non-deterministic - $\NTIME$ and $\NSPACE$. Containments - $\DTIME(t)$ is contained in $\NTIME(t)$ and $\DSPACE(s)$ is contained in $\NSPACE(s)$. Argued that $\NSPACE(s)$ is contained in $\DTIME(2^{O(s)})$ - problems solvable in non-deterministic $s$ space are solvable in deterministic $2^{O(s)}$ time.

^{Reading : Class notes and the notes by Watrous.}

Lec 53 (06 Nov, Wed) : (2-2.50 PM) Examples of problems - non-deterministic polynomial time algorithms for non-primality and graph isomorphism and argued their correctness. Non-determinism does not mean trying all possibilities. Instead it corresponds to guess + verify. Non-deterministic polynomial time class - $\NP$. Definition of non-deterministic space classes $\NLOG$ and $\NPSPACE$. Showed that directed graph reachability problem is in $\NLOG$. Argued $\LOG \subseteq \NLOG \subseteq \P \subseteq \NP \subseteq \PSPACE \subseteq \NPSPACE \subseteq \EXP$.

^{Reading : Class notes and the notes by Watrous.}

Lec 54 (07 Nov, Thu) : Discussed complement classes. Defined $\co\NTIME$ and $\co\NP$ class. Showed that $P \subseteq \NP \cap \co\NP$. Argued that satisfiability checking problem $SAT \in \NP$ and tautology checking problem - $TAUT \in \co\NP$. Defined $\NP$-completeness and showed that Bounded Membership Problem (BMP) is $\NP$-complete. SAT is $\NP$-complete (statement only). Concluded the discussion on resource bounded computation.

^{Reading : Class notes and the notes by Watrous.}

Theme: Context-free Languages

Lec 55 (08 Nov, Fri) : Introduction to context-free languages. Formal definition. Grammars with worked out examples. Parse trees and ambiguity.

^{Reading : Class notes and Lec 19 [Kozen]}

Lec 56 (12 Nov, Tue) : Tutorial 6 (Lectures 45 to 54)

Lec 57 (13 Nov, Wed) : (10-10.50 AM) Formal definition of a language of a grammar. Two languages $\{a^nb^n \mid n \ge 0\}$ and set of all palindromes over ${a,b}$ and their grammars along with proof of correctness of the grammars using induction arguments.

^{Reading : Lec 19 [Kozen]}

Lec 58 (13 Nov, Wed) : (2-2.50 PM) Grammar for the set of all strings with equal number of $a$’s and $b$’s. Proof of correctness of the grammar using induction.

^{Reading : Class notes and Lec 20 [Kozen]}

Lec 59 (14 Nov, Thu) : Closure properties of context-free languages. Normal forms. Expressing grammars in Chomsky Normal form - a worked out example. Reason for expressing grammars in Chomsky normal form - a string of length $n$ can be derived in $2n-1$ steps for a grammar expressed in Chomsky normal form.

^{Reading : Lec 21 [Kozen]}

Lec () (15 Nov, Fri) : Holiday due to Guru Nank Jayanti.