(SEM-VII) THEORY EXAMINATION 2018-19 INFORMATION THEORY AND CODING

SECTION – A

(Attempt all questions in brief – 2 × 10 = 20 marks)

(a) Information and Its Measure

Information is the reduction of uncertainty when a message is received. The amount of information depends on the probability of occurrence of the message. Rare events carry more information than common ones. Shannon defined information in terms of probability using logarithmic measures.

(b) Entropy

Entropy is a measure of average information content or uncertainty of a source. It represents the minimum number of bits required to encode source symbols without loss. Higher entropy means more uncertainty in the source.

(c) Self-Information

Self-information is the amount of information associated with a single event. It depends only on the probability of that event. Less probable events carry more self-information.

(d) Redundancy

Redundancy is the difference between maximum possible entropy and actual entropy of a source. It represents unnecessary or repeated information and helps in error detection and correction.

(e) Channel Capacity

Channel capacity is the maximum rate at which information can be transmitted over a communication channel with arbitrarily small error. It depends on bandwidth and noise conditions.

(f) Noise

Noise is any unwanted disturbance that affects signal transmission. It reduces the reliability of communication and increases the probability of errors.

(g) Source Coding

Source coding is the process of representing information efficiently by removing redundancy. Its aim is data compression without losing information.

(h) Channel Coding

Channel coding adds controlled redundancy to the transmitted data to detect and correct errors caused by noise in the channel.

(i) Binary Symmetric Channel

A binary symmetric channel is a communication channel where bits may flip with equal probability. It is widely used as a basic model in information theory.

(j) Coding Efficiency

Coding efficiency is the ratio of entropy to average code length. Higher efficiency means better utilization of the coding scheme.

SECTION – B

(Attempt any ONE – 10 marks)

(a) Shannon’s Source Coding Theorem

Shannon’s source coding theorem states that it is possible to encode information from a source with an average code length close to the entropy of the source, but not less than it. This theorem establishes the theoretical limit for lossless data compression. It proves that efficient coding schemes can be designed to minimize redundancy while preserving information.

(b) Types of Noise in Communication Systems

Noise affects the accuracy of information transmission. Thermal noise arises due to random motion of electrons. Intermodulation noise is caused by nonlinear devices. Crosstalk occurs due to signal leakage between channels. Impulse noise appears as sudden spikes and is harmful to digital communication. Understanding noise helps in designing error-resistant systems.

SECTION – C

(Attempt any ONE – 10 marks)

(a) Convolutional Encoder Explanation

A convolutional encoder generates coded output by passing input bits through shift registers and modulo-2 adders. In this question, the encoder has constraint length K = 3 with two generator sequences. One generator produces output (1,1,1) and the other produces (1,0,1).

The encoder diagram consists of a shift register with two memory elements and two adders. The output sequences depend on current and previous input bits. When an input sequence like 10011 is applied, the top and bottom output sequences are obtained by convolving the input with the generator sequences.

Convolutional coding improves error correction capability and is widely used in wireless and satellite communication.

(b) Impulse Response of Convolutional Encoder

The impulse response of a convolutional encoder represents the output when a single ‘1’ followed by zeros is applied at the input. It shows how the encoder responds over time. The impulse response is directly related to the generator sequences and helps in analyzing code performance.