This course deals with fundamental limits on communication, including the following topics: entropy, relative entropy and mutual information: entropy rates for stochastic processes differential entropy data compression the Kraft inequality Shannon-Fano codes Huffman codes arithmetic coding channel capacity discrete channels the random coding bound and its converse the capacity of Gaussian channels the sphere-packing bound coloured Gaussian noise and water-filling rate-distortion theory the rate-distortion function multiuser information theory. Topics include algebraic coding theory: finite fields, linear codes, cyclic codes, BCH codes and decoding, Reed-Solomon codes iterative decoding: codes defined on graphs, the sum-product algorithm, low-density parity-check codes, turbo codes. This course provides an introduction to error control techniques, with emphasis on decoding algorithms. Topics include random vectors, random convergence, random processes, specifying random processes, Poisson and Gaussian processes, stationarity, mean square derivatives and integrals, ergodicity, power spectrum, linear systems with stochastic input, mean square estimation, Markov chains, recurrence, absorption, limiting and steady-state distributions, time reversibility, and balance equations. Introduction to the principles and properties of random processes, with applications to communications, control systems, and computer science. Note: The course catalogues, the SGS Calendar, and ACORN list all graduate courses associated with ECE – please note that not all courses will be offered every year.

Entanglement entropy