DEEP LEARNING (KCS078)
Time: 3 Hours | Maximum Marks: 100
SECTION A (10 × 2 = 20 Marks)
(Attempt all questions in brief)
(a) Difference between perceptron and SVM in decision boundary formation
A perceptron forms a linear decision boundary and works only for linearly separable data.
An SVM forms an optimal margin-based decision boundary and can handle non-linear data using kernel functions.
(b) Effect of loss function choice on classification performance
The loss function determines how errors are penalized. For example, cross-entropy loss provides smoother gradients and better convergence than mean squared error in classification tasks.
(c) Historical development enabling deep learning success
Deep learning advanced due to large datasets, GPU computing, better activation functions (ReLU), improved optimization methods, and techniques like dropout and batch normalization.
(d) Role of weight sharing in CNNs
Weight sharing reduces the number of parameters, improves computational efficiency, prevents overfitting, and enables translation invariance in image processing.
(e) Impact of weight initialization on deep networks
Proper initialization prevents vanishing or exploding gradients and ensures faster convergence. Poor initialization may cause slow learning or training failure.
(f) Autoencoders vs traditional dimensionality reduction
Autoencoders learn non-linear representations, whereas PCA and LDA are linear. Autoencoders perform better for complex, high-dimensional data.
(g) Examples of optimization algorithms
Gradient Descent, Stochastic Gradient Descent (SGD), Momentum, RMSProp, Adam, and Adagrad.
(h) Addressing challenges of stochastic optimization
Techniques like learning rate scheduling, momentum, batch normalization, adaptive optimizers, and mini-batch training stabilize stochastic optimization.
(i) Architecture and training of WaveNet
WaveNet uses dilated causal convolutions to model raw audio waveforms. It is trained using maximum likelihood estimation to generate high-quality audio.
(j) Role of Word2Vec in NLP
Word2Vec learns dense word embeddings capturing semantic relationships using CBOW and Skip-Gram models.
SECTION B (Attempt any THREE) (3 × 10 = 30 Marks)
2(a) Gradient of logistic regression loss & effect of regularization
Logistic loss:
L = −[y log(hθ(x)) + (1 − y) log(1 − hθ(x))]
Gradient:
∂L/∂θ = (hθ(x) − y)x
Regularization adds a penalty term, reducing overfitting and improving generalization.
2(b) VC dimension and generalization in deep networks
VC dimension measures model complexity.
High VC dimension means higher capacity but risk of overfitting.
Deep networks generalize well due to implicit regularization and large datasets.
2(c) Role of manifold learning
Manifold learning assumes data lies on a lower-dimensional manifold.
Unlike PCA, it preserves local structures and captures non-linear relationships.
2(d) Generalization & role of dropout and regularization
Generalization refers to performance on unseen data.
Dropout prevents co-adaptation of neurons.
Weight regularization controls model complexity and reduces overfitting.
2(e) Scene understanding pipeline using deep learning
Pipeline includes feature extraction, object detection, segmentation, and semantic understanding.
Occlusion and lighting variations are handled using data augmentation, CNNs, and attention mechanisms.
SECTION C (Attempt any ONE) (10 Marks)
3(a) Universal Approximation Theorem
Neural networks with at least one hidden layer and non-linear activation can approximate any continuous function.
Assumptions: Sufficient neurons
Non-linear activation Continuous target function
3(b) Activation functions and non-linearity
Choice of activation affects expressiveness.
ReLU enables deep architectures, while sigmoid/tanh may cause vanishing gradients.
