First time studying this topic?

Launch the interactive walkthrough to discover each component, including the 3D Q-Sphere Lab & Python in Colab.

Topic 2.1

What is Machine Learning?

Delve into the core mechanics of machine learning: how statistical algorithms turn raw data into actionable predictive knowledge.

Learning Objectives

✓Define Machine Learning from statistical and algorithmic viewpoints
✓Understand how mathematical functions are approximated from observations
✓Map out the end-to-end Machine Learning lifecycle

📖Let's Begin with a Story

When Johannes Kepler analyzed the planetary observations recorded by Tycho Brahe in the early 1600s, he didn't have access to modern physics. He had massive tables of numbers: dates, angles, and celestial coordinates. For years, Kepler analyzed those numbers until he discovered the hidden mathematical relationship: planets travel in elliptical orbits with the Sun at one focus (

P^2 = a^3

). Machine learning does on a colossal, automated scale what Kepler did by hand: it sifts through millions of numbers, detects underlying geometric regularities, and expresses them as predictive mathematical equations.

Why Should You Learn What is Machine Learning??

🎯 Practical & Engineering Stakes

Why Machine Learning is essential knowledge:

1. Automating knowledge discovery: Human experts cannot inspect millions of hospital patient records or web clicks; ML models uncover patterns instantly. 2. Predictive advantage: From anticipating equipment failure in factories to forecasting customer demand, ML shifts organizations from reactive to proactive.

What Is It?

Simple Explanation

Machine Learning is a way to make computers smart without giving them step-by-step instructions. Instead, we show them hundreds or thousands of examples. The computer finds the common patterns on its own and uses those patterns to make smart guesses on new examples it has never seen before.

Technical Explanation

Machine Learning is an inductive functional approximation paradigm. Given an unknown underlying data-generating distribution

P(X, Y)

, an ML algorithm searches a parameterized hypothesis class

\mathcal{H} = \{f_\\heta : \\heta \in \Theta\}

to identify optimal parameters

\\heta^*

minimizing expected risk

\mathbb{E}_{(x,y)\sim P}[\mathcal{L}(f_\\heta(x), y)]

📚 Key Terms

Hypothesis Class

Simple: The family of possible formulas the computer is allowed to test.

Technical: The restricted functional space

\mathcal{H}

of candidate models searched during optimization.

Loss Function

Simple: A score that measures how wrong the computer’s guess is.

Technical: A mapping

\mathcal{L}: \hat{\mathcal{Y}} \times \mathcal{Y} \rightarrow \mathbb{R}^+

penalizing prediction discrepancy.

$Optimization$

Simple: Tuning the dials to make the mistakes as small as possible.

Technical: Algorithmic minimization of empirical risk via convex solvers, gradient descent, or heuristics.

📐 Mathematical Foundation

LaTeX KaTeX

\theta^* = \arg\min_{\theta} \frac{1}{N} \sum_{i=1}^N \mathcal{L}\big(f_\theta(x_i), y_i\big)

What does it mean?

Empirical Risk Minimization (ERM): Finding the exact parameter values theta* that minimize average loss across all training samples.

Understanding the Symbols

Symbol / Term	Meaning	Beginner Interpretation
$\theta^*$	Optimal parameters	The best weights for our model
$\mathcal{L}$	Loss function	Penalty for error
$N$	Number of training samples	Size of dataset

📊 Numerical Example

Finding optimal line slope m for points (1, 2) and (2, 4).

f_m(x) = m \cdot x \rightarrow \mathcal{L} = (2 - m)^2 + (4 - 2m)^2 = 0 \implies m^* = 2.0

Result: Optimal parameter is m* = 2.0 (zero loss)

⚙️ How Does It Work?

Formulate Problem

Define target label, evaluation metric, and input features.

Data Ingestion & Cleaning

Handle missing values, encode text/categories, and normalize scales.

Model Selection & Fitting

Train candidate algorithms on training split.

Validation & Deployment

Evaluate on independent test set and deploy via microservice API.

🌍 Where Is It Used?

Credit Risk Scoring

Problem: Assessing default probability for mortgage applicants.

Solution: Logistic regression and gradient boosting evaluate income, debt, and credit history.

Predictive Machinery Maintenance

Problem: Factory machines breaking down unexpectedly causing factory downtime.

Solution: Vibration and temperature sensors feed ML models that flag wear-and-tear days before failure.

🎯 Try It Yourself

In Machine Learning, what is the "Loss Function" designed to do?

Interactive activity component

🐍 Python Hands-On: A Minimal Machine Learning Loop from Scratch

Implement gradient descent to learn a relationship without any ML libraries.

Open Colab

1Install Required Packages

For Google Colab or Jupyter notebooks, run this shell command to install the required dependencies:

Terminal / Notebook Install Commandbash

!pip install numpy

Packages:

numpy

2Import Libraries

Import Statementspython

1 import numpy as np

import numpy as np → NumPy for vector operations

3Demonstration Code & Logic

Python Execution Codepython

1 # Ground truth rule: y = 3 * x + 2
2 X = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
3 y = np.array([5.0, 8.0, 11.0, 14.0, 17.0])
4  
5 w = 0.0 # Initialize weight
6 b = 0.0 # Initialize bias
7 lr = 0.01 # Learning rate
8  
9 for epoch in range(1000):
10     y_pred = w * X + b
11     error = y_pred - y
12     dw = (2/len(X)) * np.sum(error * X) # Gradient w.r.t w
13     db = (2/len(X)) * np.sum(error)     # Gradient w.r.t b
14     w -= lr * dw # Update weight
15     b -= lr * db # Update bias
16  
17 print(f"Learned equation: y = {w:.2f} * x + {b:.2f}")
18 print(f"Prediction for x=10: {w * 10 + b:.2f} (True answer: 32.0)")

Step-by-Step Code Breakdown:

# Ground truth rule: y = 3 * x + 2

Target rule

X = np.array([1.0, 2.0, 3.0, 4.0, 5.0])

Inputs

y = np.array([5.0, 8.0, 11.0, 14.0, 17.0])

Targets

w = 0.0 # Initialize weight

Initial weight

b = 0.0 # Initialize bias

Initial bias

lr = 0.01 # Learning rate

Step size

for epoch in range(1000):

Run 1000 training iterations

y_pred = w * X + b

Model forward pass

error = y_pred - y

Calculate error

dw = (2/len(X)) * np.sum(error * X) # Gradient w.r.t w

Compute gradient dw

db = (2/len(X)) * np.sum(error) # Gradient w.r.t b

Compute gradient db

w -= lr * dw # Update weight

Step down gradient

b -= lr * db # Update bias

Step down gradient

print(f"Learned equation: y = {w:.2f} * x + {b:.2f}")

Output learned parameters

print(f"Prediction for x=10: {w * 10 + b:.2f} (True answer: 32.0)")

Test extrapolation

📋 Complete Script (All-in-One)

Click "Copy for Colab", open Google Colab, paste into a single cell, and run.

Google Colab Notebook Scriptpython

1 # =========================================================
2 # A Minimal Machine Learning Loop from Scratch
3 # Prepared for Google Colab (Run with Shift + Enter)
4 # =========================================================
5  
6 # Step 1: Install required packages
7 !pip install numpy
8  
9 # Step 2: Import libraries
10 import numpy as np
11  
12 # Step 3: Run Demonstration
13 # Ground truth rule: y = 3 * x + 2
14 X = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
15 y = np.array([5.0, 8.0, 11.0, 14.0, 17.0])
16  
17 w = 0.0 # Initialize weight
18 b = 0.0 # Initialize bias
19 lr = 0.01 # Learning rate
20  
21 for epoch in range(1000):
22     y_pred = w * X + b
23     error = y_pred - y
24     dw = (2/len(X)) * np.sum(error * X) # Gradient w.r.t w
25     db = (2/len(X)) * np.sum(error)     # Gradient w.r.t b
26     w -= lr * dw # Update weight
27     b -= lr * db # Update bias
28  
29 print(f"Learned equation: y = {w:.2f} * x + {b:.2f}")
30 print(f"Prediction for x=10: {w * 10 + b:.2f} (True answer: 32.0)")

⚡ How to Run in Google Colab (3 Quick Steps)

Click "Copy for Google Colab" above.
Click Open New Google Colab Notebook (opens in a new tab).
Click on the first empty code cell, press Ctrl + V (or Cmd + V) to paste, then press Shift + Enter to execute!

Launch Blank Colab Notebook

🔄 Algorithmic Execution Flow

Forward Pass

Computes current predictions using initial parameters (w=0, b=0).

→

Gradient Calculation

Calculates partial derivatives showing which direction reduces error.

→

Parameter Update

Updates w and b; within 1000 iterations converges to w=3.00, b=2.00.

📥 Input / Initial State

Points following y = 3x + 2

📤 Expected Output

Learned equation: y = 3.00 * x + 2.00
Prediction for x=10: 32.00 (True answer: 32.0)

💡 What Does This Demonstration Show?

Through pure gradient feedback, the computer discovered the exact linear coefficients without human intervention.

Course Learning Resource • Slide DeckLesson 417 Slides

Lesson 4: Python, Data Preparation and Classical ML Foundations

NumPy arrays, pandas DataFrames, train-test splits, and scikit-learn models.

Author: Dr. Syed Muzamil Basha• FDP Approved Curriculum

Open in New Tab

📝 Learning Check

Test your understanding with these questions. Remember, this is for learning, not examination!

Question 1: Which of the following describes the purpose of Empirical Risk Minimization?

Question 2: What is the primary difference between supervised and unsupervised learning?

Question 3: Which of the following is an example of reinforcement learning?

Question 4: What does it mean for a machine learning model to "generalize"?

Question 5: Who is considered the "father of machine learning" for his work on the Arthur Samuel checkers program (1959)?

Previous: Python for AI: Essential Libraries Next: Data and Features: Representation & Engineering

1	# =========================================================
2	# A Minimal Machine Learning Loop from Scratch
3	# Prepared for Google Colab (Run with Shift + Enter)
4	# =========================================================
5
6	# Step 1: Install required packages
7	!pip install numpy
8
9	# Step 2: Import libraries
10	import numpy as np
11
12	# Step 3: Run Demonstration
13	# Ground truth rule: y = 3 * x + 2
14	X = np.array([1.0, 2.0, 3.0, 4.0, 5.0])
15	y = np.array([5.0, 8.0, 11.0, 14.0, 17.0])
16
17	w = 0.0 # Initialize weight
18	b = 0.0 # Initialize bias
19	lr = 0.01 # Learning rate
20
21	for epoch in range(1000):
22	y_pred = w * X + b
23	error = y_pred - y
24	dw = (2/len(X)) * np.sum(error * X) # Gradient w.r.t w
25	db = (2/len(X)) * np.sum(error) # Gradient w.r.t b
26	w -= lr * dw # Update weight
27	b -= lr * db # Update bias
28
29	print(f"Learned equation: y = {w:.2f} * x + {b:.2f}")
30	print(f"Prediction for x=10: {w * 10 + b:.2f} (True answer: 32.0)")