# Supervised Learning-Instance Based Classifiers Part-1

course: Module 2 — Machine Learning Algorithms
module: Module-2-Machine-Learning-Algorithms
type: pdf
source_url: https://personal-learn.armco.dev/files/Module-2-Machine-Learning-Algorithms/General/Supervised_Learning-Instance_Based_Classifiers_Part-1.pdf
pages: 16

---
[page 1]
IIT Roorkee – Futurense
PG Certification 
GenAI and Agentic AI for Engineers
Supervised Learning  
Dr Durga Toshniwal
Professor
Department of Computer Science & Engg.
Indian Institute of Technology Roorkee
durga.toshniwal@cs.iitr.ac.in, durgatoshniwal@gmail.com
www.durgatoshniwal.in

[page 2]
Today’s Agenda
• Instance Based Classifiers
• Rote Learner
• kNN Classifier
• Naïve Bayes’ Classifier
• K-Fold Cross Validation
GenAI and Agentic AI for Engineers

[page 3]
Classifiers
• Decision tree classifier involves a model 
construction & then its application
• They are eager learners as they are 
designed to learn the model mapping the 
attributes to class labels as soon as the 
training data set is made available
GenAI and Agentic AI for Engineers

[page 4]
Classifiers
• An opposite strategy would be to delay the 
process of learning the training data until it 
is required to classify the test examples
• Such classifiers are called lazy classifiers
GenAI and Agentic AI for Engineers

[page 5]
Instance-Based Classifiers
Atr1 ……... AtrN Class
A
B
B
C
A
C
B
Set of Stored Cases
Atr1 ……... AtrN
Unseen Case
• Store the training records 
• Use training records to 
   predict the class label of 
   unseen cases
GenAI and Agentic AI for 
Engineers

[page 6]
Instance Based Classifiers
• Examples:
– Rote-learner
•  Performs classification only if attributes of 
record match one of the training examples 
exactly
• Drawback – some records may not be 
classified as they don’t match exactly to any 
training record
• Leads to…
GenAI and Agentic AI for Engineers

[page 7]
Instance Based Classifiers
So we go for approximate match…
So all training records that are not exactly similar 
but relatively similar to the attributes of the test 
example are identified
Hence ...
– Nearest Neighbor
•  Uses k “closest” points (nearest neighbors)      
for performing classification
GenAI and Agentic AI for Engineers

[page 8]
Nearest Neighbor Classifiers
• Basic idea:
– If it looks like a duck, is part of a flock of duck, 
then it’s probably a duck
Training 
Records
Test 
Record
Compute 
Distance
Choose k of the 
“nearest” records
GenAI and Agentic AI for Engineers

[page 9]
Instance Based Classifiers
• K Nearest Neighbor (kNN)
•  Represents each record as a data point in 
a d-dimensional space where d is the 
number of attributes
• A distance measure is used to compute 
closeness of a test record to the data points 
in training set
• K closest points are identified & their class 
labels are used to find class label for the 
test record
GenAI and Agentic AI for Engineers

[page 10]
K Nearest-Neighbor (KNN)  
Classifiers
 Requires three 
things
– The set of stored 
records
– Distance Metric to 
compute distance 
between records
– The value of k, the 
number of nearest 
neighbors to 
retrieve
Unknown record
GenAI and Agentic AI for 
Engineers

[page 11]
K Nearest-Neighbor (KNN)  Classifiers
 To classify an unknown 
record:
– Compute distance to 
other training records
– Identify k nearest 
neighbors 
– Use class labels of k 
nearest neighbors to 
determine the class 
label of unknown 
record e.g., by taking 
majority vote
Unknown record
GenAI and Agentic AI for 
Engineers

[page 12]
K Nearest Neighbor (KNN) 
Classification
• Compute distance between two points:
– Euclidean distance 
• Determine the class from nearest neighbor 
list
– take the majority vote of class labels among the 
k-nearest neighbors
 −=
i ii qpqpd
2
)(),(
GenAI and Agentic AI for Engineers

[page 13]
K Nearest Neighbors
X X X
(a) 1-nearest neighbor (b) 2-nearest neighbor (c) 3-nearest neighbor
K-nearest neighbors of a record x are k data points 
closest to x
GenAI and Agentic AI for 
Engineers

[page 14]
Nearest Neighbor Classification
• Choosing the value of k:
– If k is too small, sensitive to noise points
– If k is too large, neighborhood may include points from 
other classes
X
GenAI and Agentic AI for Engineers

[page 15]
K Nearest Neighbor (KNN) 
Classification
• k-NN classifiers are lazy learners 
• Do not maintain an abstraction or model from 
training data
• Thus Make their predictions on the basis of local 
information & not global models that fit the entire 
data
• So are quite susceptible to noise
GenAI and Agentic AI for Engineers

[page 16]
Eager Learners Versus Lazy 
Learners
Eager Learners are better ? 
Lazy Learners are better ?
 
GenAI and Agentic AI for Engineers