First time studying this topic?

Launch the interactive walkthrough to discover each component, including the 3D Q-Sphere Lab & Python in Colab.

Topic 2.2

Data and Features: Representation & Engineering

Understand feature engineering, numerical and categorical variables, scaling, normalization, and handling missing data.

Learning Objectives

✓Distinguish between numerical, categorical, and ordinal data types
✓Apply One-Hot Encoding and Label Encoding for categorical data
✓Understand the critical importance of feature scaling (Standardization vs MinMax)
✓Formulate strategies for handling missing values and outliers

📖Let's Begin with a Story

In the early days of automated real estate valuation, an algorithm was trained on house sales in California. The dataset had features like "Square Footage" and "Zip Code". The model treated Zip Code as a plain number: Beverly Hills (90210) was treated as "larger" than Miami (33101), leading the model to believe that multiplying zip codes by a weight had geometric meaning! The predictions were disastrous. Numbers in computer science aren't all the same: some are continuous quantities, some are categories, and some are arbitrary ID codes. The success of any machine learning project hinges first and foremost on proper feature representation.

Why Should You Learn Data and Features: Representation & Engineering?

🎯 Practical & Engineering Stakes

"Garbage In, Garbage Out"

The greatest algorithm on earth will fail if the features fed into it are improperly scaled, uncleaned, or incorrectly encoded. Industry data scientists spend up to 80% of their time on data preparation and feature engineering.

What Is It?

Simple Explanation

Data is the food that machine learning models eat. But computers only understand clean numbers. If you give a computer words like "Red", "Green", or "Blue", you must convert them into numbers using smart techniques (like One-Hot Encoding). If one feature is measured in thousands (like salary) and another in single digits (like age), you must scale them so the model doesn't ignore age.

Technical Explanation

Feature engineering transforms raw observations

\mathbf{x}_{raw} \in \mathcal{X}

into structured feature vectors

\mathbf{x} \in \mathbb{R}^d

. This entails categorical embedding/one-hot encoding

\phi_{one-hot}: \mathcal{C} \rightarrow \{0, 1\}^{|\mathcal{C}|}

, z-score standardization

z = \frac{x - \mu}{\sigma}

, and handling imputation via probabilistic or median strategies.

📚 Key Terms

One-Hot Encoding

Simple: Turning categories into separate columns of 1s and 0s.

Technical: Mapping a categorical variable with K states into a K-dimensional binary indicator vector.

Standardization (Z-score)

Simple: Centering numbers so their average is 0 and spread is 1.

Technical: Affine transformation

z = \frac{x - \mu}{\sigma}

ensuring zero mean and unit variance.

$Imputation$

Simple: Filling in missing holes in a dataset with smart estimates (like the average).

Technical: Replacing missing data with substituted values derived from mean, median, KNN, or MICE estimators.

📐 Mathematical Foundation

LaTeX KaTeX

z = \frac{x - \mu}{\sigma}, \quad x_{norm} = \frac{x - x_{min}}{x_{max} - x_{min}}

What does it mean?

Z-score Standardization (left) and Min-Max Normalization (right): two standard methods to bring features to comparable numerical ranges.

Understanding the Symbols

Symbol / Term	Meaning	Beginner Interpretation
$\mu$	$Mean$	Average value of feature
$\sigma$	Standard Deviation	Spread of values
$x_{min}, x_{max}$	Extreme bounds	Minimum and maximum values in dataset

📊 Numerical Example

Standardizing an income of $80,000 where mean is $50,000 and standard deviation is $15,000.

z = \frac{80000 - 50000}{15000} = \frac{30000}{15000} = 2.0

Result: Income is 2.0 standard deviations above the mean

⚙️ How Does It Work?

Data Profiling

Inspect distributions, missing value percentages, and data types.

Encoding

Convert categorical strings to numerical dummy vectors via one-hot encoding.

Scaling

Apply StandardScaler or MinMaxScaler to prevent high-magnitude features from dominating gradients.

🌍 Where Is It Used?

Credit Card Application Processing

Problem: Dataset contains mixed text (Employment: "Engineer", "Nurse") and numbers (Income: $85,000, Age: 34).

Solution: Pipeline one-hot encodes job titles and standardizes income/age for input into neural networks.

Customer Churn Analysis

Problem: Raw data contains missing customer tenure values.

Solution: Median imputation fills missing fields without distorting underlying distributions.

🎯 Try It Yourself

A column has values: ["Low", "Medium", "High"]. What is the most appropriate encoding?

Interactive activity component

🐍 Python Hands-On: Data Preprocessing: Handling Missing Values, One-Hot Encoding & Scaling

Preprocess raw tabular data with missing values, categorical features, and numerical scaling using Pandas and Scikit-Learn.

Open Colab

1Install Required Packages

For Google Colab or Jupyter notebooks, run this shell command to install the required dependencies:

Terminal / Notebook Install Commandbash

!pip install pandas scikit-learn numpy

Packages:

pandasscikit-learnnumpy

2Import Libraries

Import Statementspython

1 import pandas as pd
2 from sklearn.preprocessing import OneHotEncoder, StandardScaler

import pandas as pd → Pandas for tabular data processing

from sklearn.preprocessing import OneHotEncoder, StandardScaler → Scikit-Learn preprocessing transformers

3Demonstration Code & Logic

Python Execution Codepython

1 # Sample dataset with missing values and categorical data
2 data = {'Age': [25, 30, 35, 40, None], 'Salary': [50000, 60000, 70000, None, 90000], 'Gender': ['Male', 'Female', 'Male', 'Female', 'Male']}
3 df = pd.DataFrame(data)
4  
5 # Handling missing values (fill with column mean)
6 df["Age"] = df["Age"].fillna(df["Age"].mean())
7 df["Salary"] = df["Salary"].fillna(df["Salary"].mean())
8  
9 # Encoding categorical variable
10 encoder = OneHotEncoder(sparse_output=False)
11 gender_encoded = encoder.fit_transform(df[['Gender']])
12 df_gender = pd.DataFrame(gender_encoded, columns=['Male', 'Female'])
13  
14 # Scaling numerical features with StandardScaler
15 scaler = StandardScaler()
16 df[['Age', 'Salary']] = scaler.fit_transform(df[['Age', 'Salary']])
17  
18 # Concatenating final processed data
19 df_final = pd.concat([df[['Age', 'Salary']], df_gender], axis=1)
20 print("Preprocessed Data:\n", df_final)

Step-by-Step Code Breakdown:

# Sample dataset with missing values and categorical data

Sample data

data = {'Age': [25, 30, 35, 40, None], 'Salary': [50000, 60000, 70000, None, 90000], 'Gender': ['Male', 'Female', 'Male', 'Female', 'Male']}

Raw dictionary

df = pd.DataFrame(data)

Create DataFrame

# Handling missing values (fill with column mean)

Mean imputation

df["Age"] = df["Age"].fillna(df["Age"].mean())

Fill missing Age

df["Salary"] = df["Salary"].fillna(df["Salary"].mean())

Fill missing Salary

# Encoding categorical variable

One-hot encoding

encoder = OneHotEncoder(sparse_output=False)

Instantiate dense OneHotEncoder

gender_encoded = encoder.fit_transform(df[['Gender']])

Fit & transform gender

df_gender = pd.DataFrame(gender_encoded, columns=['Male', 'Female'])

Categorical DataFrame

# Scaling numerical features with StandardScaler

Standardization

scaler = StandardScaler()

Instantiate StandardScaler

df[['Age', 'Salary']] = scaler.fit_transform(df[['Age', 'Salary']])

Scale age and salary

# Concatenating final processed data

Merge features

df_final = pd.concat([df[['Age', 'Salary']], df_gender], axis=1)

Combine numerical + encoded features

print("Preprocessed Data:\n", df_final)

Display preprocessed DataFrame

📋 Complete Script (All-in-One)

Click "Copy for Colab", open Google Colab, paste into a single cell, and run.

Google Colab Notebook Scriptpython

1 # =========================================================
2 # Data Preprocessing: Handling Missing Values, One-Hot Encoding & Scaling
3 # Prepared for Google Colab (Run with Shift + Enter)
4 # =========================================================
5  
6 # Step 1: Install required packages
7 !pip install pandas scikit-learn numpy
8  
9 # Step 2: Import libraries
10 import pandas as pd
11 from sklearn.preprocessing import OneHotEncoder, StandardScaler
12  
13 # Step 3: Run Demonstration
14 # Sample dataset with missing values and categorical data
15 data = {'Age': [25, 30, 35, 40, None], 'Salary': [50000, 60000, 70000, None, 90000], 'Gender': ['Male', 'Female', 'Male', 'Female', 'Male']}
16 df = pd.DataFrame(data)
17  
18 # Handling missing values (fill with column mean)
19 df["Age"] = df["Age"].fillna(df["Age"].mean())
20 df["Salary"] = df["Salary"].fillna(df["Salary"].mean())
21  
22 # Encoding categorical variable
23 encoder = OneHotEncoder(sparse_output=False)
24 gender_encoded = encoder.fit_transform(df[['Gender']])
25 df_gender = pd.DataFrame(gender_encoded, columns=['Male', 'Female'])
26  
27 # Scaling numerical features with StandardScaler
28 scaler = StandardScaler()
29 df[['Age', 'Salary']] = scaler.fit_transform(df[['Age', 'Salary']])
30  
31 # Concatenating final processed data
32 df_final = pd.concat([df[['Age', 'Salary']], df_gender], axis=1)
33 print("Preprocessed Data:\n", df_final)

⚡ How to Run in Google Colab (3 Quick Steps)

Click "Copy for Google Colab" above.
Click Open New Google Colab Notebook (opens in a new tab).
Click on the first empty code cell, press Ctrl + V (or Cmd + V) to paste, then press Shift + Enter to execute!

Launch Blank Colab Notebook

🔄 Algorithmic Execution Flow

Imputation

Missing values in Age and Salary are filled with their respective column means.

→

Categorical Encoding

Gender is one-hot encoded into binary columns [Male, Female].

→

Feature Scaling

Age and Salary are standardized to zero mean and unit variance.

📥 Input / Initial State

Tabular dataset with missing numbers (None) and categorical Gender strings

📤 Expected Output

Preprocessed Data:
        Age    Salary  Male  Female
0 -1.632993 -1.341641   0.0     1.0
1 -0.544331 -0.447214   1.0     0.0
2  0.544331  0.447214   0.0     1.0
3  1.632993  0.000000   1.0     0.0
4  0.000000  1.341641   0.0     1.0

💡 What Does This Demonstration Show?

The data is fully cleaned, imputed, encoded, and scaled without information loss, ready for machine learning model training.

Course Learning Resource • Slide DeckLesson 417 Slides

Lesson 4: Python, Data Preparation and Classical ML Foundations

Data preprocessing, handling missing values, one-hot encoding, and standard scaling.

Author: Dr. Syed Muzamil Basha• FDP Approved Curriculum

Open in New Tab

📝 Learning Check

Test your understanding with these questions. Remember, this is for learning, not examination!

Question 1: Why is feature scaling essential for distance-based algorithms like KNN and gradient descent?

Question 2: What is feature engineering in machine learning?

Question 3: Why is feature scaling (normalization) important for many ML algorithms?

Question 4: What is the "curse of dimensionality"?

Question 5: What is the typical train-test split ratio in machine learning?

Previous: What is Machine Learning?Next: Training and Testing: The Golden Rule of ML

1	import pandas as pd
2	from sklearn.preprocessing import OneHotEncoder, StandardScaler

1	# =========================================================
2	# Data Preprocessing: Handling Missing Values, One-Hot Encoding & Scaling
3	# Prepared for Google Colab (Run with Shift + Enter)
4	# =========================================================
5
6	# Step 1: Install required packages
7	!pip install pandas scikit-learn numpy
8
9	# Step 2: Import libraries
10	import pandas as pd
11	from sklearn.preprocessing import OneHotEncoder, StandardScaler
12
13	# Step 3: Run Demonstration
14	# Sample dataset with missing values and categorical data
15	data = {'Age': [25, 30, 35, 40, None], 'Salary': [50000, 60000, 70000, None, 90000], 'Gender': ['Male', 'Female', 'Male', 'Female', 'Male']}
16	df = pd.DataFrame(data)
17
18	# Handling missing values (fill with column mean)
19	df["Age"] = df["Age"].fillna(df["Age"].mean())
20	df["Salary"] = df["Salary"].fillna(df["Salary"].mean())
21
22	# Encoding categorical variable
23	encoder = OneHotEncoder(sparse_output=False)
24	gender_encoded = encoder.fit_transform(df[['Gender']])
25	df_gender = pd.DataFrame(gender_encoded, columns=['Male', 'Female'])
26
27	# Scaling numerical features with StandardScaler
28	scaler = StandardScaler()
29	df[['Age', 'Salary']] = scaler.fit_transform(df[['Age', 'Salary']])
30
31	# Concatenating final processed data
32	df_final = pd.concat([df[['Age', 'Salary']], df_gender], axis=1)
33	print("Preprocessed Data:\n", df_final)