(SEM IV) THEORY EXAMINATION 2018-19 INFORMATION THEORY AND CODING

INFORMATION THEORY AND CODING – B.Tech (Semester IV), REC-406

This is a 3-hour, 70-mark theory exam designed to evaluate students’ understanding of information theory fundamentals, entropy, source coding, channel capacity, coding theorems, error-correcting codes, and decoding concepts.
The paper contains three structured sections, testing basic definitions, derivations, numerical problems, and advanced coding concepts.
 

SECTION A — Short Answer Questions (2 × 7 = 14 Marks)

This section has 7 brief questions targeting conceptual clarity.

Topics include:

Channel capacity                                                          Information rate

Relation between information & probability               Why logarithms are used to measure information

Extension of discrete memoryless source                    Properties of entropy

Source coding theorem

These short questions test basic definitions, formulas, and theoretical understanding.
 

SECTION B — Descriptive Questions (7 × 3 = 21 Marks)

Attempt any 3 out of 5.
This section assesses reasoning, derivations, coding logic, and theory explanation.

Major topics include:

Mutual information & capacity of Binary Erasure Channel (BEC)

Properties of a typical set (AEP – Asymptotic Equipartition Property)

Channel coding theorem preview & channel capacity properties

Proof that average code length ≥ entropy (L ≥ H)

Physical significance of different entropies

This section focuses on analytical and mathematical reasoning.
 

SECTION C — Long Answer Questions (7 marks each)

Each major question contains two choices (a or b) requiring detailed proofs, numerical solutions, coding procedures, or diagrams.

Q3 – Coding Inequalities & DSP Concepts

Extended Kraft Inequality + proving codeword lengths

Log-Sum Inequality & Data-Processing Inequality

Q4 – Communication System Probability Questions

Includes error probabilities using Bayes’ theorem:

Calculating p(r0), p(r1)

Probability of correct/incorrect detection

Receiver error probability

or

Entropy, code efficiency & redundancy using Huffman coding for 8 symbols

Q5 – Convolution Codes / Channel Capacity

Encoding convolution codes using shift registers, code tree & state diagram

Deriving capacity of infinite bandwidth channel

Q6 – Linear Block & Cyclic Codes

Standard arrays

Generator matrix for (7,4) cyclic code & systematic coding

Q7 – Trellis & Viterbi Decoding / Shortened Codes

Trellis diagram + Viterbi decoding (error detection & correction)

Concept and application of shortened codes

This section tests deep understanding of coding theory, error correction, probability, and decoding algorithms.
 

OVERALL PURPOSE OF THE EXAM

The question paper aims to evaluate whether the student can:

Understand entropy, mutual information & channel capacity

Apply information theory laws to practical communication channels

Construct & analyze various source and channel coding schemes

Perform Huffman, convolutional & cyclic coding

Use decoding techniques such as Viterbi and Standard Array

Apply mathematical inequalities (Kraft, Log-Sum, Data-Processing)

Compute error probabilities in noisy communication systems

This paper ensures students gain strong foundations in digital communication and coding theory, essential for networking, telecommunications, and data compression applications.