(SEM VII) THEORY EXAMINATION 2024-25 DEEP LEARNING

SECTION A — Very Short Answers (2 Marks Each)

a. Loss Function

A loss function measures the difference between predicted output and true output and guides weight updates during training.

b. Stochastic Gradient Descent (SGD)

An optimization algorithm that updates model parameters using one or a small batch of training samples at a time.

c. AI Winter

A period of reduced funding and interest in AI due to limited computational power and unmet expectations.

d. Probabilistic Theory of Deep Learning

Views deep learning models as probabilistic systems that learn data distributions and uncertainty.

e. Linear Discriminant Analysis (LDA)

A supervised dimensionality-reduction technique that maximizes class separability.

f. Distance Metrics in ML

Used to measure similarity between data points
Examples: Euclidean, Manhattan, Cosine distance.

g. Non-Convex Optimization

Optimization problems with multiple local minima and saddle points, common in deep networks.

h. Generalization

The ability of a trained model to perform well on unseen data.

i. WaveNet

A deep generative model for raw audio waveform generation, known for high-quality speech synthesis.

j. Word2Vec

A technique that learns vector representations of words capturing semantic relationships.

SECTION B — Short Descriptive Answers (10 Marks)

a. Activation Functions (ReLU & Sigmoid)

Sigmoid: Outputs values between 0 and 1, used in binary classification

ReLU: Outputs max(0, x), faster convergence and avoids vanishing gradient

b. GANs: Generator & Discriminator

Generator: Creates fake data                           Discriminator: Distinguishes real vs fake data

Both trained in an adversarial manner

c. AlexNet Significance                                      Won ImageNet 2012

Introduced ReLU, dropout, GPU training          Sparked deep learning revolution

d. Spatial Transformer Networks (STNs)           Allow networks to learn spatial transformations

Improve spatial invariance

e. Deep Learning in Bioinformatics                  Gene sequencing

Protein structure prediction                             Disease diagnosis

SECTION C — Optimization & Training (10 Marks)

a. Backpropagation                                          Computes gradients using chain rule

Updates weights to minimize loss                   Enables deep network training

b. Role of SGD in Optimization                        Faster convergence

Handles large datasets                                     Introduces noise that helps escape local minima

SECTION D — CNN & Normalization (10 Marks)

a. CNN for Image Classification                      Convolution layers extract features

Pooling reduces dimensionality                      Fully connected layers perform classification

b. Batch Normalization                                   Normalizes layer inputs

Reduces internal covariate shift                    Speeds up training and improves stability

SECTION E — Initialization & Deep Architectures (10 Marks)

a. Xavier vs He Initialization                           Xavier: Suitable for Sigmoid/Tanh

He: Suitable for ReLU                                    Prevents exploding/vanishing gradients

b. Residual Connections (ResNet)                 Skip connections allow gradient flow

Enable very deep networks                          Reduce degradation problem