Why Handling Missing Values In Dataset Is Important 🎯. | by Muhammad Taha | Feb, 2025

1. Figuring out Lacking Values

Earlier than dealing with lacking values, we have to detect them.

import pandas as pd

# Pattern dataset with lacking values
knowledge = {'Identify': ['Alice', 'Bob', 'Carol', 'Dave'],
'Age': [25, 30, None, 40],
'Wage': [50000, 60000, None, 70000]}df = pd.DataFrame(knowledge)# Test for lacking values
print(df.isnull())  # True signifies a lacking worth
print(df.isnull().sum())  # Depend of lacking values in every column

2. Eradicating Lacking Values

a) Eradicating Rows with Lacking Values

df_cleaned = df.dropna()  # Removes any row with at the least one lacking worth
print(df_cleaned)

b) Eradicating Columns with Lacking Values

df_cleaned = df.dropna(axis=1)  # Removes columns with lacking values
print(df_cleaned)

⚠ Disadvantage: This could trigger knowledge loss if too many rows or columns are eliminated.

3. Filling Lacking Values (Imputation)

a) Filling with a Particular Worth

df_filled = df.fillna(0)  # Exchange lacking values with 0
print(df_filled)

b) Filling with Imply, Median, or Mode

df['Age'].fillna(df['Age'].imply(), inplace=True)  # Fill with imply
df['Salary'].fillna(df['Salary'].median(), inplace=True)  # Fill with median
print(df)

c) Filling with the Earlier or Subsequent Worth

df.fillna(methodology='ffill', inplace=True)  # Ahead fill (use earlier worth)
df.fillna(methodology='bfill', inplace=True)  # Backward fill (use subsequent worth)

4. Interpolating Lacking Values

Interpolation estimates lacking values based mostly on different values within the column.

df['Age'] = df['Age'].interpolate()
df['Salary'] = df['Salary'].interpolate()
print(df)

5. Dealing with Lacking Information in Machine Studying

Some ML fashions can’t deal with lacking values immediately. We are able to:

• Fill lacking values earlier than coaching.
• Use fashions like XGBoost that deal with lacking knowledge mechanically.

Instance: Filling Lacking Values Earlier than Coaching

from sklearn.impute import SimpleImputer
import numpy as np

imputer = SimpleImputer(technique='imply')  # Select 'imply', 'median', or 'most_frequent'
df[['Age', 'Salary']] = imputer.fit_transform(df[['Age', 'Salary']])
print(df)