(SEM VII) THEORY EXAMINATION 2024-25 MACHINE LEARNING

SECTION A

(2 × 10 = 20 marks | Short Answers)

a) Well-defined vs Ill-defined learning problems

Well-defined problems: Clear input, output, and performance measure (e.g., spam classification).

Ill-defined problems: Unclear goals or incomplete data (e.g., learning human emotions).

b) Bias–variance trade-off

Bias: Error due to oversimplified assumptions.     Variance: Error due to sensitivity to training data.
A good model balances both to achieve better generalization.

c) Inductive bias in decision tree learning

Inductive bias is the assumption made by a learner to generalize beyond training data. Decision trees prefer shorter trees with fewer nodes.

d) Effect of overfitting on decision trees

Overfitting causes a decision tree to memorize training data, leading to poor performance on unseen data.

e) Hypothesis accuracy

Hypothesis accuracy is the percentage of correctly classified examples by a hypothesis on a dataset.

f) Bayes theorem

Bayes theorem relates conditional probabilities:

P(H∣D)=P(D∣H)P(H)P(D)P(H|D) = \frac{P(D|H)P(H)}{P(D)}P(H∣D)=P(D)P(D∣H)P(H)​ 

g) Sample size vs generalization error

As sample size increases, the generalization error decreases, especially for finite hypothesis spaces.

h) Mistake bound model

It is a theoretical model that bounds the maximum number of mistakes an online learning algorithm can make.

i) Hypothesis space

A hypothesis space is the set of all possible hypotheses a learning algorithm can choose from.

j) General-to-specific beam search

It starts with a general hypothesis and progressively specializes it, keeping only the best candidates at each step.

SECTION B

(Attempt any 3 | 10 marks each)

a) Concept learning for binary classification

In concept learning, examples are classified as positive or negative based on attributes.
Example: Learning whether an email is spam or not using features like keywords and sender.

b) Weight adjustment in Adaline

Adaline updates weights using the Delta rule, minimizing mean squared error through gradient descent.

c) Bayes Optimal Classifier

It assigns the class with maximum posterior probability, minimizing classification error theoretically.

d) k-NN implementation

Steps:                                                                               Choose k

Calculate distance                                                            Select nearest neighbors

Assign majority class

e) Advantages of FOIL

FOIL handles first-order logic, allowing relational learning, unlike propositional rule learners.

SECTION C

Q3 (Attempt any one)

a) Find-S algorithm

Find-S finds the most specific hypothesis consistent with all positive examples by progressively generalizing attributes.

b) List-Then-Eliminate algorithm

It lists all hypotheses and removes inconsistent ones using training examples, leaving consistent hypotheses.

Q4 (Attempt any one)

a) Perceptron training

Weights are updated as:

wnew=wold+η(y−y^)xw_{new} = w_{old} + \eta (y - \hat{y})xwnew​=wold​+η(y−y^​)x

Used for linearly separable problems.

b) Delta rule

The Delta rule adjusts weights to minimize error using gradient descent in single-layer neural networks.

Q5 (Attempt any one)

a) EM algorithm steps

E-step: Estimate missing data using current parameters

M-step: Update parameters to maximize likelihood

b) Bayesian Belief Networks

Conditional probabilities are represented using directed acyclic graphs, where nodes represent variables and edges represent dependencies.

Q6 (Attempt any one)

a) Mistake bound model with Perceptron

The perceptron makes a finite number of mistakes if data is linearly separable, bounded by margin and input magnitude.

b) Locally Weighted Regression (LWR)

LWR fits a local model around the query point using nearby data points weighted by distance.

Q7 (Attempt any one)

a) Reinforcement learning components

States: Environment situations                           Actions: Choices available

Rewards: Feedback signal

b) Learning rate & discount factor in Q-learning

Learning rate (α): Controls speed of learning

Discount factor (γ): Importance of future rewards