# Session%20-23%2011%202025-%20Summary%20Slides

course: Module 1 — Foundations of AI & ML
module: Module-1-Foundations-AI-ML
type: pdf
source_url: https://personal-learn.armco.dev/files/Module-1-Foundations-AI-ML/General/PPT_Slides_22-11-2025_and_23-11-2025/Session%20-23%2011%202025-%20Summary%20Slides.pdf
pages: 17

---
[page 1]
IIT Roorkee – Futurense
PG Certification 
GenAI and Agentic AI for Engineers
Supervised Learning 
Performance Analysis 
Dr Durga Toshniwal
Professor
Department of Computer Science & Engg.
Indian Institute of Technology Roorkee
durga.toshniwal@cs.iitr.ac.in, durgatoshniwal@gmail.com
www.durgatoshniwal.in
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 2]
Today’s Agenda
• Data Types and Similarity Measures
• Model Performance Metrics 
• FPR
• FNR
• TPR
• TNR
• Precision, Recall, Sensitivity, Specificity, ROC - AUC
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 3]
Similarity and Dissimilarity
• Similarity
– Numerical measure of how alike two data objects are.
– Is higher when objects are more alike.
– Often falls in the range [0,1]
• Dissimilarity
– Numerical measure of how different two data objects are
– Lower when objects are more alike
– Minimum dissimilarity is often 0
– Upper limit varies
• Proximity refers to a similarity or dissimilarity
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 4]
Similarity Between Binary Vectors
• Common situation is that objects, p and q, 
have only binary attributes, 
• Compute similarities using the following 
quantities
 M01 = the number of attributes where p was 0 and q was 1
 M10 = the number of attributes where p was 1 and q was 0
 M00 = the number of attributes where p was 0 and q was 0
 M11 = the number of attributes where p was 1 and q was 1
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 5]
Similarity Between Binary Vectors
Similarity coefficients
• Simple Matching Coefficient (SMC)
 SMC =  number of matches / number of attributes 
           =  (M00 + M11) / (M01 + M10 + M00 + M11)
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 6]
Similarity Between Binary Vectors
Similarity coefficients
• Jaccard Coefficient 
 
J = number of 11 matches / number of not-both-zero 
attributes values
      = (M11) / (M01 + M10 + M11) 
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 7]
Cosine Similarity
Documents are often represented as vectors, where 
each attribute represents the frequency with which a 
particular term/word occurs in the document 
•   If d1 and d2 are two document vectors, then
             
     cos( d1, d2 ) =  (d1 • d2) / ||d1|| ||d2|| , 
   
    where • indicates dot product and || d || is the length of  
vector d   
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 8]
Confusion Matrix / Contingency 
Table / Error Matrix
• It is a specific table layout that allows visualization of the 
performance of an algorithm
• Each column of the matrix represents the instances in a 
predicted class while each row represents the instances in 
an actual class (or vice-versa).
• The name stems from the fact that it makes it easy to see if 
the system is confusing two classes (i.e. commonly 
mislabeling one as another).
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 9]
Confusion Matrix / Contingency 
Table / Error Matrix
• TP = True Positive
• TN = True Negative
•  FP = False Positive
•  FN = False Negative
Predicted Class
Positive Negative
Actual
class
Positive TP FN
Negative FP TN
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 10]
Confusion Matrix / Contingency 
Table / Error Matrix
Predicted Class
Positive Negative
Actual
Class
Positive TP FN
Negative FP TN
𝑷𝒓𝒆𝒄𝒊𝒔𝒊𝒐𝒏 =
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)+𝑭𝒂𝒍𝒔𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑭𝑷)  
                     
                    =   
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒐𝒕𝒂𝒍 𝑷𝒓𝒆𝒅𝒊𝒄𝒕𝒆𝒅 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔  
Precision talks about how precise/accurate our model is
Out of those predicted positive, how many of them are actual 
positive.
Sensitivity synonymous to Precision
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 11]
Confusion Matrix / Contingency 
Table / Error Matrix
Predicted Class
Positive Negative
Actual
class
Positive TP FN
Negative FP TN
𝑹𝒆𝒄𝒂𝒍𝒍 =
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)+𝑭𝒂𝒍𝒔𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑭𝑵)  
            
So Recall actually calculates how many of the Actual Positives our model 
captures 
Recall is the model metric we use to select our best model when there is a 
high cost associated with False Negative.GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 12]
Confusion Matrix / Contingency 
Table / Error Matrix
Predicted Class
Positive Negative
Actual
class
Positive TP FN
Negative FP TN
𝑷𝒓𝒆𝒄𝒊𝒔𝒊𝒐𝒏 = 𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)+𝑭𝒂𝒍𝒔𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑭𝑷)=   𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒐𝒕𝒂𝒍 𝑷𝒓𝒆𝒅𝒊𝒄𝒕𝒆𝒅 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔  
𝑹𝒆𝒄𝒂𝒍𝒍 =
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)+𝑭𝒂𝒍𝒔𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑭𝑵)  =  
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒐𝒕𝒂𝒍 𝑨𝒄𝒕𝒖𝒂𝒍 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆  
𝑭𝟏 𝑺𝒄𝒐𝒓𝒆 = 𝟐 𝒙 𝑷𝒓𝒆𝒄𝒊𝒔𝒊𝒐𝒏 ∗ 𝑹𝒆𝒄𝒂𝒍𝒍
𝑷𝒓𝒆𝒄𝒊𝒔𝒊𝒐𝒏 + 𝑹𝒆𝒄𝒂𝒍𝒍 
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 13]
Confusion Matrix / Contingency 
Table / Error Matrix
Predicted Class
Positive Negative
Actual
class
Positive TP FN
Negative FP TN
𝑨𝒄𝒄𝒖𝒓𝒂𝒄𝒚 = 𝑻𝑷 + 𝑻𝑵
𝑻𝑷 + 𝑭𝑷 + 𝑻𝑵 + 𝑭𝑵 
           =
𝑪𝒐𝒓𝒓𝒆𝒄𝒕 𝑷𝒓𝒆𝒅𝒊𝒄𝒕𝒊𝒐𝒏𝒔
𝑻𝒐𝒕𝒂𝒍 𝑷𝒓𝒆𝒅𝒊𝒄𝒕𝒊𝒐𝒏𝒔 
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 14]
Confusion Matrix / Contingency 
Table / Error Matrix
Predicted Class
Positive Negative
Actual
class
Positive TP FN
Negative FP TN
GenAI and Agentic AI - Prof Durga 
Toshniwal
𝑺𝒆𝒏𝒔𝒊𝒕𝒊𝒗𝒊𝒕𝒚 = 𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷)
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷) + 𝑭𝒂𝒍𝒔𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑭𝑵)
Sensitivity measures the proportion of positives that are correctly identified 
(e.g., the percentage of sick people who are correctly identified as having 
some illness).

[page 15]
Confusion Matrix / Contingency 
Table / Error Matrix
Predicted Class
Positive Negative
Actual
class
Positive TP FN
Negative FP TN
𝑺𝒑𝒆𝒄𝒊𝒇𝒊𝒄𝒊𝒕𝒚 = 𝑻𝒓𝒖𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑻𝑵)
𝑻𝒓𝒖𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑻𝑵) + 𝑭𝒂𝒍𝒔𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑭𝑷)
Specificity measures the proportion of negatives that are correctly 
identified (e.g., the percentage of healthy people who are correctly 
identified as not having some illness).
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 16]
Confusion Matrix / Contingency 
Table / Error Matrix
Predicted
Positive Negative
Actual
class
Positive TP FN
Negative FP TN
𝑭𝑷𝑹 = 𝑭𝒂𝒍𝒔𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑭𝑷)
𝑻𝒓𝒖𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑻𝑵) + 𝑭𝒂𝒍𝒔𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑭𝑷)
          =  
𝑭𝒂𝒍𝒔𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑭𝑷)
𝑨𝒄𝒕𝒖𝒂𝒍 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔
𝑭𝑵𝑹 
= 𝑭𝒂𝒍𝒔𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑭𝑵)
𝑻𝒓𝒖𝒆 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔(𝑻𝑷) + 𝑭𝒂𝒍𝒔𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑭𝑵)
          =  
𝑭𝒂𝒍𝒔𝒆 𝑵𝒆𝒈𝒂𝒕𝒊𝒗𝒆𝒔(𝑭𝑵)
𝑨𝒄𝒕𝒖𝒂𝒍 𝑷𝒐𝒔𝒊𝒕𝒊𝒗𝒆𝒔
GenAI and Agentic AI - Prof Durga 
Toshniwal

[page 17]
ROC Curve
(0,1) – Perfect Classification 
           
The ROC curve is created by plotting the true positive rate (TPR) against 
the false positive rate (FPR) at various threshold settings.
GenAI and Agentic AI - Prof Durga 
Toshniwal