(SEM VII) THEORY EXAMINATION 2024-25 DEEP LEARNING

DEEP LEARNING (KCS078)

Time: 3 Hours | Maximum Marks: 100

SECTION A (10 × 2 = 20 Marks)

(Attempt all questions in brief)

(a) Difference between perceptron and SVM in decision boundary formation

A perceptron forms a linear decision boundary and works only for linearly separable data.
An SVM forms an optimal margin-based decision boundary and can handle non-linear data using kernel functions.

(b) Effect of loss function choice on classification performance

The loss function determines how errors are penalized. For example, cross-entropy loss provides smoother gradients and better convergence than mean squared error in classification tasks.

(c) Historical development enabling deep learning success

Deep learning advanced due to large datasets, GPU computing, better activation functions (ReLU), improved optimization methods, and techniques like dropout and batch normalization.

(d) Role of weight sharing in CNNs

Weight sharing reduces the number of parameters, improves computational efficiency, prevents overfitting, and enables translation invariance in image processing.

(e) Impact of weight initialization on deep networks

Proper initialization prevents vanishing or exploding gradients and ensures faster convergence. Poor initialization may cause slow learning or training failure.

(f) Autoencoders vs traditional dimensionality reduction

Autoencoders learn non-linear representations, whereas PCA and LDA are linear. Autoencoders perform better for complex, high-dimensional data.

(g) Examples of optimization algorithms

Gradient Descent, Stochastic Gradient Descent (SGD), Momentum, RMSProp, Adam, and Adagrad.

(h) Addressing challenges of stochastic optimization

Techniques like learning rate scheduling, momentum, batch normalization, adaptive optimizers, and mini-batch training stabilize stochastic optimization.

(i) Architecture and training of WaveNet

WaveNet uses dilated causal convolutions to model raw audio waveforms. It is trained using maximum likelihood estimation to generate high-quality audio.

(j) Role of Word2Vec in NLP

Word2Vec learns dense word embeddings capturing semantic relationships using CBOW and Skip-Gram models.

SECTION B (Attempt any THREE) (3 × 10 = 30 Marks)

2(a) Gradient of logistic regression loss & effect of regularization

Logistic loss:
L = −[y log(hθ(x)) + (1 − y) log(1 − hθ(x))]

Gradient:
∂L/∂θ = (hθ(x) − y)x

Regularization adds a penalty term, reducing overfitting and improving generalization.

2(b) VC dimension and generalization in deep networks

VC dimension measures model complexity.
High VC dimension means higher capacity but risk of overfitting.
Deep networks generalize well due to implicit regularization and large datasets.

2(c) Role of manifold learning

Manifold learning assumes data lies on a lower-dimensional manifold.
Unlike PCA, it preserves local structures and captures non-linear relationships.

2(d) Generalization & role of dropout and regularization

Generalization refers to performance on unseen data.
Dropout prevents co-adaptation of neurons.
Weight regularization controls model complexity and reduces overfitting.

2(e) Scene understanding pipeline using deep learning

Pipeline includes feature extraction, object detection, segmentation, and semantic understanding.
Occlusion and lighting variations are handled using data augmentation, CNNs, and attention mechanisms.

SECTION C (Attempt any ONE) (10 Marks)

3(a) Universal Approximation Theorem

Neural networks with at least one hidden layer and non-linear activation can approximate any continuous function.

Assumptions:                                               Sufficient neurons

Non-linear activation                                    Continuous target function

3(b) Activation functions and non-linearity

Choice of activation affects expressiveness.
ReLU enables deep architectures, while sigmoid/tanh may cause vanishing gradients.