Lectures

Following is an outline of lectures given along with references and links to additional reading. For the abbreviations (ECT), (Hill), (Roth), (Lint), please check the references page.

\( \def\PAL{\rm PAL} \def\HP{\rm HP} \def\MP{\rm MP} \def\EQ{\rm EQ} \def\USELESS{\rm USELESS} \def\EMPTY{\rm EMPTY} \def\DTIME{\sf DTIME} \def\NTIME{\sf NTIME} \def\DSPACE{\sf DSPACE} \def\NSPACE{\sf NSPACE} \def\P{\sf P} \def\co{\sf co} \def\NP{\sf NP} \def\LOG{\sf LOG} \def\NLOG{\sf NLOG} \def\PSPACE{\sf PSPACE} \def\NPSPACE{\sf NPSPACE} \def\EXP{\sf EXP} \def\F{\mathbb{F}} \)

Lec 1 (30 Jul, Thu) : Course plan. Basic goals in the design of error correcting codes. Shannon’s model of communication. Examples of simple codes - repetition codes, parity check codes. On error detection and correction of codes.

^{Reading : (ECT) Section 1.1; (Hill) Chapter 1}

^{Person of the day : Claude E. Shannon - Father of information theory, AI agent of same first name, Mathematical Theory of Communication, Creative thinking.}

Lec 2 (01 Aug, Sat) : (9.30 - 10.45 AM) Hamming code- description and codewords. Parity check criteria for error detection. Error correction in Hamming codes as guided exercise.

^{Reading : (ECT) Section 1.2 to 1.5}

^{Person of the day : Richard W. Hamming - Famous for Hamming codes, Hamming metric, Hamming ball and Hamming bound, ACM Turing award laudation ‘Purpose of computing is insight, not numbers’, You and your research.}

^{Exercises : (ECT) Chap 1, Problems 1.1, 1.3}

Lec 3 (01 Aug, Sat) : (11.00 - 12.30 PM) Hat check puzzle - two approaches using codes. Minimum distance and its role in error detection and unique decoding.

^{Reading : (ECT) Section 1.2}

^{Exercises : (ECT) Chap 1, Problems 1.5, 1.10, 1.11}

Lec 4 (04 Aug, Tue) : Code with distance $d \ge 2$ $\iff$, it can detect $d-1$ errors, correct $\lfloor (d-1)/2 \rfloor$ errors. Errors and erasures. BSC and BEC channel models.

^{Reading : (ECT) Chap Section 1.2, 1.3, 1.4, (Hill) Chap 2}

^{Person of the day : Robert Fano - Famous for Fano’s inequality, Developed one of the first timer-sharing computers. One of the world’s first open-source advocates.}

^{Exercises : (ECT) Chap 1, Problem 1.6, 1.8}

Lec 5 (06 Aug, Thu) : Linear binary codes. Review of basic linear algebra. Linear codes. Characterization using generator and parity check matrix. Computing minimum distance from parity check matrix.

^{Reading : (ECT) Chap 2, Section 2.2, 2.3, 2.4, (Hill) Chap 5}

^{Person of the day : Irving S. Reed - Famous for Reed Solomon codes, Design team of MADDIDA computer, Developed the register transfer language.}

^{Exercises : (Hill) Chap 5, Ex. 5.1, 5.3}

Lec () (11 Aug, Tue) : Instructor out of town (compensated on Aug 01)

Lec () (13 Aug, Thu) : Instructor out of town (compensated on Aug 01)

Lec () (18 Aug, Tue) : Instructor out of town (to be compensated)

Lec () (20 Aug, Thu) : Instructor out of town (to be compensated)

Lec 6 (25 Aug, Tue) : Recap. Equivalences of linear codes and equivalence in terms on operations on generator matrix. Systematic code and systematic generator matrix. Relation to parity check matrix. Decoding - MLD algorithm.

^{Reading : (Hill) Chap 5, (ECT) Chap 2, Section 2.3, 2.4, 2.5}

^{Person of the day : Gustave Solomon - Famous for Reed Solomon codes, Founder of algebraic theory of error correcting codes.}

^{Exercises : (ECT) Ex. 1.8}

Lec 7 (27 Aug, Thu) : Characterization of distance of code using linear (in)dependence of columns of parity check matrix. Decoding algorithm for linear codes. Syndrome decoding. Hardness of linear code decoding problem (statement only). Generalized Hamming codes and an efficient decoding algorithm. Limits on the relation between rate and distance - the Hamming bound. Hamming bound is tight for Generalized Hamming codes. Perfect codes.

^{Reading : (Hill) Chap 5, (ECT) Chap 2, Section 2.3, 2.4, 2.5}

^{Person of the day : Marcel J. E. Golay - Discovered binary and ternary Golay codes - perfect codes. Invented Golay Detector used to detect aircraft by emitting IR. Introduced complementary sequences which is used in Wifi and 3G standards}

^{Exercises : (ECT) Ex. 2.7, 2.10, 2.14}

Lec 8 (01 Sep, Tue) : Linear algebra fails over $\mathbb{Z}_m$ for composite $m$ - failure of existence of inverse of elements. Way to overcome this, a motivating example - constructing $\mathbb{C}$ from $\mathbb{R}$. Constructing fields that are finite. The three views of this construction - generator, polynomial, vector space views. Irreducible polynomials. Enumeration of irreducible polynomials over $\mathbb{F}_2$ of degree $2$, $3$ and $4$. Using them to construct finite fields of size 4 and 8.

^{Reading : (Roth) Chapter 3. (Hill) Chapter 3,4}

^{Person of the day : Robert J. McEliece - Showed that the decoding problem is NP-hard. Known for McEliece’s Theorem. Also McEliece cryptosystem based on hardness of decoding}

^{Exercises : (Hill) Ex. 4.1, 4.3, 4.4}

Lec 9 (03 Sep, Thu) : Description of distance $5$ code using $GF(16)$ and a decoding algorithm. Basics of algebra - well ordering principle, division algorithm, GCD and its properties for integers. polynomial division and polynomial GCD computation.

^{Reading : (Roth) Chapter 3, Class notes}

^{Person of the day : Selmer M. Johnson - Famous for Johnson bound, Johnson graphs, Johnson schemes.}

^{Exercises : Work out more questions from the sage demo done in class.}

Lec 10 (08 Sep, Tue) : Polynomial division and polynomial GCD algorithm. Irreducibility and divisibility of polynomials. Reviewed proof of unique factorization of integers (fundamental theorem of arithmetic). Argued polynomials in $\mathbb{F}[x]$ are uniquely factorizable.

^{Reading : (Roth) Chapter 3, Class notes}

^{Person of the day : Elwyn R. Berlekamp - Noted algebraic coding theorist and entrepreneur. Famous for algorithms to factor polynomials over finite fields and integers, known for two algorithms for decoding Reed-Solomon codes.}

Lec 11 (10 Sep, Thu) : Basics of groups, cosets. Properties of cosets. Used them to derive Lagrange’s theorem. Extension fields and why study them. Towards proving cyclic nature of finite groups.

^{Reading : (Roth) Chapter 3, Class notes}

^{Person of the day : Neil A. Sloane - Creator of an enormous encyclopedia for interesting integer sequences oeis.org launched in 1996. Has contributions in combinatorics, coding theory and sphere packing.}

Lec 12 (15 Sep, Tue) : Roots and multiplicity. Proved that any non-zero polynomial $p$ in $\mathbb{F}[x]$ has at most $deg(p)$ many roots in any extension of $\mathbb{F}$. Order of an element in a group and three of its properties. Towards arguing that $\mathbb{F}^*$ is cyclic using the results seen so far.

^{Reading : (Roth) Chapter 3, Class notes}

^{Person of the day : Michael Tanner - Founder of the field of “codes on graphs”. Creator of Tanner code. These codes are used in cellular, wifi and deep-space communication systems.}

Lec 13 (17 Sep, Thu) : Recap of discussion so far. Relating order of an element to the order of its power. Order of product of two elements whose orders are relativly prime is exactly the product of the orders. Used these results to complete the proof that $\mathbb{F}^*$ is cyclic.

^{Reading : (Roth) Chapter 3, Class notes}

^{Person of the day : Jessie C. MacWilliams - She is known for the MacWilliams identity. Author of the earliest comprehensive text “The Theory of Error-Correcting Codes” with Neil A. Sloane. This book was of interest for mathematicians as well as engineers. First woman to publish in the area of coding theory.}

Lec 14 (22 Sep, Tue) : Characteristic of a finite field. Characteristic of every finite field must be prime. Every finite field contains a subfield whose size is a prime number. Vector space axioms. Every finite field can be viewed as a vector space over the underlying field of prime size.

^{Reading : (Roth) Chapter 3, Class notes}

^{Person of the day : Priti Shankar - Credited with generalizing BCH codes to finite (Galois) rings. Editorial member of Resonance magazine - an Indian science education journal.}

Lec 15 (24 Sep, Thu) : Existence of minimal polynomial of a field element. Every finite field can be viewed as $\mathbb{F}_p[x]/q$ for some prime $p$ and irreducible polynomial $q \in \mathbb{F}[x]$. Existence of extension field where all the roots of non-zero polynomial appears - splitting fields. All splitting fields of a given size are isomorphic (statement only). For every prime $p$ and integer $k$, there is a finite field of size $q=p^k$.

^{Reading : (Roth) Chapter 3, Class notes.}

^{Person of the day : David A. Huffman - Known for Huffman coding for lossless data compression; studied hazards in digitial logic and ways to design circuits avoiding them.}

Lec 16 (29 Sep, Tue) : Recap of discussion so far. Completed the proof of existence of splitting field. Argued how dimension of finite fields viewed as vector space over subfields are related. Towards arguing existence of a finite field of order $p^n$ for every prime $p$ and $n \ge 1$ - structure of extension fields, minimal polynomials and conjugacy classes.

^{Reading : (Roth) Chapter 3, Class notes.}

^{Person of the day : Vera Pless - Known for code constructions over $\mathbb{Z}_4$ and for proving uniqueness of Golay codes.}

Lec 17 (01 Oct, Thu) : Conjugacy classes of elements in extension field and their minimal polynomials. Worked out examples for fields of size $8$ and $16$. Minimal polynomial of an element from its conjugacy class. Properties of minimal polynomials and irreducibility.

^{Reading : (Roth) Chapter 7, Class notes.}

^{Person of the day : J. H. van Lint - Known for several influential textbooks in Combinatorics, Coding theory and Algebraic Geometry. Also known for studying strongly regular graphs that arise form structured codes (particularly Golay code).}

Lec 18 (06 Oct, Tue) : Closed form expression for minimal polynomial of an element in terms of its conjugacy class. For a field $\mathbb{F}_{q^n}$, factorization of $x^{q^n}-x$ to minimal polynomials of elements from the field. Connection to irreducible polynomials over $\mathbb{F}_q$ whose degree divides $n$. Completed the proof of existence of irreducible polynomials over every $q$ and degree.

^{Reading : (Roth) Chapter 7, Class notes.}

^{Person of the day : David E. Muller - Discovered the Reed-Muller codes. Irving Reed, proposed the first efficient decoding algorithm. Also known for Muller automata on infinite words.}

Lec 19 (08 Oct, Thu) : Introduction to Reed-Solomon codes- definition and distance. Generator matrix view. Key goals for the remainder of this course - (1) what are the best parameters that can be achieved for rate and distance of a code, (2) are there codes that achieve them and can they be efficiently decoded. Limitations - Singleton-Joshi-Komamiya bound. Three proofs. Reed-Solomon codes achieve Singleton bound. Sphere-packing bound. Rate of growth of Hamming ball in terms of the binary (and $q$-ary) entropy functions.

^{Reading : (Roth) Chapter 4}

^{Person of the day : Raj Chandra Bose (R. C. Bose) - Known for works in coding theory, esp. BCH codes. Also known for Bose-Mesner algebra and for being one of the Euler’s spoiler. In memory of Bose.}

Lec 20 (13 Oct, Tue) : Relative rate and distance. Asymptotic variants of the Sphere packing and Singleton bounds and plots showing what relative rates and distances are not possible. What is possible ? Answered by Gilbert-Varshamov bound showing existence of linear codes. A naive proof attempt via a greedy packing argument and a mind block.

^{Reading : (Roth) Chapter 4, class notes.}

^{Person of the day : Dijen K. Ray-Chaudhuri - Known for BCH codes (jointly with R. C. Bose). Also known for giving a solution to the general case of Kirkman’s schoolgirl problem.}

Lec 21 (15 Oct, Thu) : Completing the proof of GV bound. Asymptotic variant of GV bound and plot. Towards showing a random linear code achieves GV bound. A refresher on probability basics.

^{Reading : (Roth) Chapter 4, class notes.}

^{Person of the day : Richard M. Wilson - Known for contributions to combinatorics and design theory. Known for inequalities on the family of sets of restricted pairwise intersection. Also known for a textbook - “A course on combinatorics” written with van Lint.}

Lec () (20 Oct, Tue) : Holiday due to Dussehra

Lec 22 (22 Oct, Thu) : Completed the proof that random linear code achieves GV bound. Plotkin bound and its proof and geometric packing argument for alphabet set of size $2$.

^{Reading : (Roth) Chapter 4, class notes.}

^{Person of the day : Katalin Marton - Marton is known for several contributions in information theory (blow-up lemma) and coding theory (capacity of two-receiver broadcast channel) with wide applications in combinatorics (polynomial Freiman–Ruzsa conjecture, now a theorem) and probability theory (Talagrand inequality).}

Lec 23 (27 Oct, Tue) : Elias bound. Bounding the number of codewords in a Hamming ball. Proof of Elias bound for alphabet size $2$ - viewing bounding the number of codewords in a Hamming ball as an optimization problem and finding the minimizer.

^{Reading : (Roth) Chapter 4, class notes.}

Lec 24 (29 Oct, Thu) : Elias bound for general alphabet of size $q \ge 2$. Completed the proof details. Connection to list decoding problem.

^{Reading : (Roth) Chapter 4, class notes.}

Lec 25 (03 Nov, Tue) : Decoding error probability for a code. Worked out examples for repetition and Hamming codes for binary symmetric channel. Relation between rate and decoding error. An impossibility result - Shannon’s converse coding theorem. Statement and proof details discussed.

^{Reading : (Roth) Chapter 4, class notes.}

Lec 26 (05 Nov, Thu) : Decoding of Reed-Solomon codes - Berlekamp-Welsch algorithm. Simple decoding algorithm assuming that the error locations are known, along with correctness and runtime. Decoding when the error locations are unknown - the error locator polynomial. Details of the algorithm and correctness argument.

^{Reading : (ECT) Section 5.2, class notes.}