First time studying this topic?

Launch the interactive walkthrough to discover each component, including the 3D Q-Sphere Lab & Python in Colab.

Topic 1.8

Unsupervised Learning: Clustering & Dimensionality Reduction

Explore how algorithms discover hidden geometry, groups, and latent structures without human labels.

Learning Objectives

✓Understand K-Means and hierarchical clustering
✓Grasp dimensionality reduction using Principal Component Analysis (PCA)
✓Identify anomaly detection applications in cybersecurity and finance

📖Let's Begin with a Story

Astronomers surveying the night sky collect spectral radiation data from hundreds of millions of distant celestial bodies. No human has ever visited these stars, and no astronomer has pre-labeled them. Using unsupervised clustering, algorithms grouped millions of stars by their surface temperature and luminosity. The algorithm naturally discovered the famous "Hertzsprung-Russell Diagram"—separating Main Sequence stars, Red Giants, and White Dwarfs without ever knowing what a star was. Unsupervised learning reveals the innate order hidden in raw data.

Why Should You Learn Unsupervised Learning: Clustering & Dimensionality Reduction?

🎯 Practical & Engineering Stakes

Why Unsupervised Learning is crucial:

1. Labels are scarce and expensive: 99% of the world's data (satellite feeds, audio, web text) is unlabeled. 2. Exploratory Data Analysis: Discovers market niches and user behavior that human product managers never thought to look for. 3. Data Compression: Dimensionality reduction shrinks terabytes of data while preserving 95%+ of essential variance.

What Is It?

Simple Explanation

Unsupervised learning is like sorting a huge box of mystery LEGO bricks without instructions. You naturally group them by color, shape, or size. You find clusters without anyone telling you the names of the groups.

Technical Explanation

Unsupervised learning discovers latent representations

\mathbf{z}

that model the data generating distribution

p(\mathbf{x}) = \int p(\mathbf{x}|\mathbf{z})p(\mathbf{z})d\mathbf{z}

, or partitions

\mathcal{X}

into Voronoi cells

S_k

minimizing intra-cluster inertia

\sum_{k} \sum_{\mathbf{x} \in S_k} ||\mathbf{x} - \\boldsymbol{\mu}_k||^2

📚 Key Terms

$Clustering$

Simple: Grouping similar data points together based on how close they are to each other.

Technical: Unsupervised partitioning of instances into disjoint or probabilistic subsets based on metric distance functions.

Dimensionality Reduction

Simple: Compressing data with many features down to fewer features while keeping the important information.

Technical: Projecting high-dimensional data

\mathbb{R}^D

onto a lower-dimensional manifold

\mathbb{R}^d

(

d \ll D

) while maximizing preserved variance.

📐 Mathematical Foundation

LaTeX KaTeX

J = \sum_{k=1}^K \sum_{x \in S_k} \|x - \mu_k\|^2

What does it mean?

K-Means objective function: minimizes the sum of squared Euclidean distances between data points and their assigned cluster centroid.

Understanding the Symbols

Symbol / Term	Meaning	Beginner Interpretation
$K$	Number of clusters	Predefined count of groups
$\mu_k$	Centroid of cluster k	The center coordinate of cluster k
$S_k$	Set of points in cluster k	All observations assigned to group k

📊 Numerical Example

Point x=3 assigned to centroid mu=2. Squared distance is (3 - 2)^2 = 1.

\|3 - 2\|^2 = 1

Result: Distance contribution to inertia is 1

⚙️ How Does It Work?

Centroid Initialization

K random points are chosen as initial cluster centers.

Assignment Step

Every data point is assigned to its nearest centroid.

Update Step

Centroids are recalculated as the mean of all points assigned to them.

Convergence

Repeat until centroid positions stabilize.

🌍 Where Is It Used?

Genomics & DNA Clustering

Problem: Identifying disease subtypes among 20,000 gene expressions.

Solution: Hierarchical clustering identifies genetic cohorts responding to specialized therapies.

Server Intrusion Detection

Problem: Zero-day cyberattacks have never been seen or labeled before.

Solution: Density-based anomaly detection flags network packets deviating from typical baseline traffic.

🎯 Try It Yourself

In the K-Means algorithm, what does "K" represent?

Interactive activity component

🐍 Python Hands-On: K-Means Clustering with Scikit-Learn

Cluster customer spend and visit frequency without any labels.

Open Colab

1Install Required Packages

For Google Colab or Jupyter notebooks, run this shell command to install the required dependencies:

Terminal / Notebook Install Commandbash

!pip install scikit-learn numpy

Packages:

scikit-learnnumpy

2Import Libraries

Import Statementspython

1 from sklearn.cluster import KMeans
2 import numpy as np

from sklearn.cluster import KMeans → Import K-Means

import numpy as np → NumPy array support

3Demonstration Code & Logic

Python Execution Codepython

1 # Customer data: [Visits per month, Average Spend in Dollars]
2 customers = np.array([[1, 20], [2, 25], [1, 15], [15, 300], [18, 350], [14, 280]])
3 kmeans = KMeans(n_clusters=2, random_state=0).fit(customers)
4 print("Cluster Centers (Centroids):\n", kmeans.cluster_centers_)
5 new_shopper = [[16, 320]]
6 print("Assigned Cluster for [16 visits, $320]:", kmeans.predict(new_shopper)[0])

Step-by-Step Code Breakdown:

# Customer data: [Visits per month, Average Spend in Dollars]

Data setup

customers = np.array([[1, 20], [2, 25], [1, 15], [15, 300], [18, 350], [14, 280]])

Casual vs VIP customers

kmeans = KMeans(n_clusters=2, random_state=0).fit(customers)

Run K-Means with K=2

print("Cluster Centers (Centroids):\n", kmeans.cluster_centers_)

Show centers

new_shopper = [[16, 320]]

New customer

print("Assigned Cluster for [16 visits, $320]:", kmeans.predict(new_shopper)[0])

Predict cluster

📋 Complete Script (All-in-One)

Click "Copy for Colab", open Google Colab, paste into a single cell, and run.

Google Colab Notebook Scriptpython

1 # =========================================================
2 # K-Means Clustering with Scikit-Learn
3 # Prepared for Google Colab (Run with Shift + Enter)
4 # =========================================================
5  
6 # Step 1: Install required packages
7 !pip install scikit-learn numpy
8  
9 # Step 2: Import libraries
10 from sklearn.cluster import KMeans
11 import numpy as np
12  
13 # Step 3: Run Demonstration
14 # Customer data: [Visits per month, Average Spend in Dollars]
15 customers = np.array([[1, 20], [2, 25], [1, 15], [15, 300], [18, 350], [14, 280]])
16 kmeans = KMeans(n_clusters=2, random_state=0).fit(customers)
17 print("Cluster Centers (Centroids):\n", kmeans.cluster_centers_)
18 new_shopper = [[16, 320]]
19 print("Assigned Cluster for [16 visits, $320]:", kmeans.predict(new_shopper)[0])

⚡ How to Run in Google Colab (3 Quick Steps)

Click "Copy for Google Colab" above.
Click Open New Google Colab Notebook (opens in a new tab).
Click on the first empty code cell, press Ctrl + V (or Cmd + V) to paste, then press Shift + Enter to execute!

Launch Blank Colab Notebook

🔄 Algorithmic Execution Flow

Centroid Fitting

Calculates centers for casual shoppers (~$20) and VIP shoppers (~$310).

→

Inference

Assigns new shopper with 16 visits and $320 spend to the VIP cluster.

📥 Input / Initial State

Customer behavior matrix with two distinct customer cohorts

📤 Expected Output

Cluster Centers:
[[ 1.33  20.00]
 [15.67 310.00]]
Assigned Cluster for [16 visits, $320]: 1

💡 What Does This Demonstration Show?

Without any supervisory intervention, K-Means discovered the high-value VIP segment and the low-frequency casual shopper segment.

Course Learning Resource • Slide DeckLesson 211 Slides

Lesson 2: Classical Machine Learning as the Starting Point

Data, prediction, and the bridge to modern artificial intelligence.

Author: Dr. Syed Muzamil Basha• FDP Approved Curriculum

Open in New Tab

📝 Learning Check

Test your understanding with these questions. Remember, this is for learning, not examination!

Question 1: Which technique is primarily used to project high-dimensional data into a lower-dimensional space while preserving variance?

Question 2: In K-Means clustering, what happens during the "Assignment Step"?

Question 3: Astronomers used unsupervised clustering to analyze millions of stellar spectra without any pre-existing star classifications. What does this demonstrate?

Question 4: The K-Means objective function J = Σ_k Σ_{x∈Sk} ||x - μk||² minimizes what quantity?

Question 5: Why is unsupervised anomaly detection particularly valuable for detecting zero-day cyberattacks?

Question 6: A K-Means clustering run assigns a new customer with [16 visits, $320 spend] to cluster 1 (centroid: [15.67, $310]). Why is this assignment correct?

Previous: Supervised Learning: Classification & Regression Next: Introduction to Reinforcement Learning

1	# =========================================================
2	# K-Means Clustering with Scikit-Learn
3	# Prepared for Google Colab (Run with Shift + Enter)
4	# =========================================================
5
6	# Step 1: Install required packages
7	!pip install scikit-learn numpy
8
9	# Step 2: Import libraries
10	from sklearn.cluster import KMeans
11	import numpy as np
12
13	# Step 3: Run Demonstration
14	# Customer data: [Visits per month, Average Spend in Dollars]
15	customers = np.array([[1, 20], [2, 25], [1, 15], [15, 300], [18, 350], [14, 280]])
16	kmeans = KMeans(n_clusters=2, random_state=0).fit(customers)
17	print("Cluster Centers (Centroids):\n", kmeans.cluster_centers_)
18	new_shopper = [[16, 320]]
19	print("Assigned Cluster for [16 visits, $320]:", kmeans.predict(new_shopper)[0])