Experiment: Classification and Anomaly Detection on IoT Sensor Data

Aim

To develop and implement machine learning models to classify IoT sensor data into different device or environmental states and detect anomalous sensor readings.

Objectives

To understand the application of machine learning in IoT systems.
To collect and prepare temperature and humidity sensor data for machine learning.
To classify IoT sensor readings into Normal, Warning and Critical states.
To implement a Decision Tree classification model.
To evaluate the performance of the classification model.
To understand the concept of anomaly detection in IoT sensor data.
To implement an Isolation Forest model for detecting anomalous sensor readings.
To visualize the classification results and detected anomalies.

Components Required

Sr. No.	Component / Software	Quantity
1	ESP32 development board	1
2	DHT11 temperature and humidity sensor	1
3	USB cable	1
4	Patch cords / jumper wires	As required
5	Wi-Fi network / Internet connection	1
6	Computer or laptop	1
7	Cognifront Cloud platform	1
8	Python	1
9	Jupyter Notebook / Google Colab / VS Code	1
10	Pandas library	1
11	NumPy library	1
12	Matplotlib library	1
13	Scikit-learn library	1

Theory

Internet of Things (IoT) is a technology in which physical devices equipped with sensors, processing units and communication capabilities collect and exchange data over a network. IoT systems continuously generate sensor data that can be transmitted to cloud platforms and used for monitoring, analysis and intelligent decision-making.

In this experiment, a DHT11 sensor is used to measure temperature and humidity. The sensor is connected to an ESP32 development board. The ESP32 reads the sensor values and can transmit the collected data to a cloud platform using Wi-Fi. The collected sensor data can be exported as a CSV file and used as input for machine learning.

Machine Learning (ML) is a branch of artificial intelligence in which computer systems learn patterns from data and use those patterns to make predictions or decisions.

Machine learning techniques can be used in IoT systems for applications such as device-state classification, activity recognition, fault detection and abnormal-condition detection.

This experiment demonstrates two machine learning tasks:

Classification using a Decision Tree.
Anomaly Detection using Isolation Forest.

Classification

Classification is a supervised machine learning task in which a model learns to assign input data to predefined classes.

In this experiment, temperature and humidity values are used as input features. The sensor readings are classified into three environmental states:

Class	Description
Normal	Temperature and humidity values are within the normal operating range.
Warning	Temperature or humidity values indicate a condition that requires attention.
Critical	Temperature or humidity values indicate a significantly abnormal condition.

A Decision Tree Classifier is used for classification. A decision tree makes decisions by applying a sequence of conditions to the input features.

For example, a simplified decision process may be represented as:

              Temperature
                  |
        ┌─────────┼─────────┐
        ↓         ↓         ↓
      Normal    Warning   Critical

The actual model uses temperature and humidity together to learn the classification boundaries from the training data.

Anomaly Detection

Anomaly detection is the process of identifying observations that differ significantly from the normal pattern of data.

An anomaly in an IoT sensor dataset may occur because of:

Sensor malfunction.
Communication errors.
Incorrect sensor readings.
Environmental disturbances.
Hardware faults.
Data transmission errors.

Unlike classification, anomaly detection does not require every abnormal observation to have a predefined class label.

In this experiment, Isolation Forest is used for anomaly detection.

Isolation Forest is an unsupervised machine learning algorithm that identifies unusual observations by isolating them from the rest of the dataset. Observations that are significantly different from normal observations are more easily isolated and are therefore considered potential anomalies.

For example, if normal temperature readings are approximately between 23 °C and 32 °C, values such as 80 °C or -10 °C may be identified as anomalous.

The classification and anomaly-detection processes are therefore different:

Classification:

Temperature + Humidity
          ↓
   Decision Tree
          ↓
Normal / Warning / Critical


Anomaly Detection:

Temperature + Humidity
          ↓
   Isolation Forest
          ↓
Normal / Anomaly

The overall workflow of the experiment is:

DHT11 Sensor
      ↓
    ESP32
      ↓
    Wi-Fi
      ↓
 IoT Dataset
      ↓
Data Preprocessing
      ↓
 ┌───────────────────────┐
 │                       │
 ↓                       ↓
Classification       Anomaly Detection
 │                       │
 ↓                       ↓
Decision Tree       Isolation Forest
 │                       │
 ↓                       ↓
Normal/Warning/      Normal/Anomaly
Critical

DHT11 Pin Connections

The following connections can be used when collecting the temperature and humidity data using an ESP32.

DHT11 Pin	ESP32 Pin	Description
VCC	3.3V	Power supply
DATA	GPIO 4	Digital temperature and humidity data
GND	GND	Ground connection

Note: The DATA pin can be connected to another suitable ESP32 GPIO if the program is modified accordingly.

Dataset

The dataset used in this experiment contains temperature and humidity readings obtained from a DHT11 sensor.

The dataset contains the following fields:

Column	Description
ID	Unique record identifier
Temperature	Temperature measured by the DHT11 sensor in °C
Humidity	Relative humidity measured by the DHT11 sensor in %
Date	Date and time of the sensor reading

The clean dataset is used as the original reference dataset.

A separate copy of the dataset can be created for anomaly detection. Controlled anomalous values can be introduced into this copy to demonstrate the ability of the Isolation Forest model to detect abnormal sensor readings.

Important: The original clean dataset should not be modified. The anomalous dataset should be created as a separate copy.

Preparing the Dataset

For the classification task, a class label named State is generated from the temperature and humidity values.

The following example rules are used to generate the labels:

Condition	State
Temperature < 30 °C and Humidity < 70 %	Normal
Temperature ≥ 30 °C or Humidity ≥ 70 %	Warning
Temperature ≥ 33 °C or Humidity ≥ 80 %	Critical

The Critical condition is checked before the Warning condition so that a critical reading is not incorrectly classified as Warning.

These labels are used only to create the labelled dataset required for demonstrating supervised classification.

Program

Install the required Python libraries before running the program.

pip install pandas numpy matplotlib scikit-learn

The following program can be executed using Jupyter Notebook, Google Colab or VS Code.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn.metrics import (
    accuracy_score,
    confusion_matrix,
    classification_report
)
from sklearn.ensemble import IsolationForest

# -------------------------------------------------
# 1. Load the dataset
# -------------------------------------------------

df = pd.read_csv("clean-dht11-temperature-humidity.csv")

print("Original Dataset")
print(df.head())

print("\nDataset Information")
print(df.info())

print("\nMissing Values")
print(df.isnull().sum())

print("\nDuplicate Records")
print(df.duplicated().sum())


# -------------------------------------------------
# 2. Data preprocessing
# -------------------------------------------------

df["Temperature"] = pd.to_numeric(
    df["Temperature"],
    errors="coerce"
)

df["Humidity"] = pd.to_numeric(
    df["Humidity"],
    errors="coerce"
)

df["Date"] = pd.to_datetime(
    df["Date"],
    errors="coerce"
)

# Remove invalid records
df = df.dropna(
    subset=["Temperature", "Humidity", "Date"]
)

# Remove duplicate records
df = df.drop_duplicates()


# -------------------------------------------------
# 3. Create classification labels
# -------------------------------------------------

def classify_state(row):

    temperature = row["Temperature"]
    humidity = row["Humidity"]

    if temperature >= 33 or humidity >= 80:
        return "Critical"

    elif temperature >= 30 or humidity >= 70:
        return "Warning"

    else:
        return "Normal"


df["State"] = df.apply(
    classify_state,
    axis=1
)

print("\nClass Distribution")
print(df["State"].value_counts())


# -------------------------------------------------
# 4. Classification
# -------------------------------------------------

# Input features
X = df[
    ["Temperature", "Humidity"]
]

# Target class
y = df["State"]

# Split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    random_state=42,
    stratify=y
)

# Create Decision Tree model
classifier = DecisionTreeClassifier(
    max_depth=4,
    random_state=42
)

# Train the model
classifier.fit(
    X_train,
    y_train
)

# Predict test data
y_pred = classifier.predict(X_test)


# -------------------------------------------------
# 5. Classification evaluation
# -------------------------------------------------

accuracy = accuracy_score(
    y_test,
    y_pred
)

print("\nClassification Accuracy:")
print(accuracy)

print("\nClassification Report:")
print(
    classification_report(
        y_test,
        y_pred
    )
)

print("\nConfusion Matrix:")
cm = confusion_matrix(
    y_test,
    y_pred
)

print(cm)


# -------------------------------------------------
# 6. Visualize the Decision Tree
# -------------------------------------------------

plt.figure(figsize=(14, 8))

plot_tree(
    classifier,
    feature_names=[
        "Temperature",
        "Humidity"
    ],
    class_names=classifier.classes_,
    filled=True
)

plt.title(
    "Decision Tree for IoT Sensor Classification"
)

plt.show()


# -------------------------------------------------
# 7. Load anomalous dataset
# -------------------------------------------------

anomaly_df = pd.read_csv(
    "anomalous-dht11-temperature-humidity.csv"
)

anomaly_df["Temperature"] = pd.to_numeric(
    anomaly_df["Temperature"],
    errors="coerce"
)

anomaly_df["Humidity"] = pd.to_numeric(
    anomaly_df["Humidity"],
    errors="coerce"
)

anomaly_df = anomaly_df.dropna(
    subset=[
        "Temperature",
        "Humidity"
    ]
)


# -------------------------------------------------
# 8. Isolation Forest anomaly detection
# -------------------------------------------------

features = anomaly_df[
    ["Temperature", "Humidity"]
]

isolation_model = IsolationForest(
    contamination=0.02,
    random_state=42
)

isolation_model.fit(
    features
)

# Predict anomalies
anomaly_prediction = isolation_model.predict(
    features
)

anomaly_df["Anomaly"] = anomaly_prediction

# Convert prediction to readable labels
anomaly_df["Status"] = anomaly_df[
    "Anomaly"
].map({
    1: "Normal",
    -1: "Anomaly"
})


# -------------------------------------------------
# 9. Display detected anomalies
# -------------------------------------------------

detected_anomalies = anomaly_df[
    anomaly_df["Status"] == "Anomaly"
]

print("\nDetected Anomalies:")
print(
    detected_anomalies[
        [
            "ID",
            "Temperature",
            "Humidity",
            "Date",
            "Status"
        ]
    ]
)

print(
    "\nNumber of detected anomalies:",
    len(detected_anomalies)
)


# -------------------------------------------------
# 10. Visualize anomaly detection
# -------------------------------------------------

plt.figure(figsize=(10, 6))

normal_data = anomaly_df[
    anomaly_df["Status"] == "Normal"
]

anomaly_data = anomaly_df[
    anomaly_df["Status"] == "Anomaly"
]

plt.scatter(
    normal_data["Temperature"],
    normal_data["Humidity"],
    label="Normal"
)

plt.scatter(
    anomaly_data["Temperature"],
    anomaly_data["Humidity"],
    label="Anomaly",
    marker="x",
    s=80
)

plt.xlabel("Temperature (°C)")
plt.ylabel("Humidity (%)")
plt.title(
    "IoT Sensor Anomaly Detection"
)

plt.legend()
plt.tight_layout()
plt.show()


# -------------------------------------------------
# 11. Save classification dataset
# -------------------------------------------------

df.to_csv(
    "classified-dht11-data.csv",
    index=False
)

# Save anomaly detection results
anomaly_df.to_csv(
    "dht11-anomaly-detection-results.csv",
    index=False
)

print(
    "\nProcessed datasets saved successfully."
)

Code Explanation

pd.read_csv(): Loads the IoT temperature and humidity dataset from a CSV file.

pd.to_numeric(): Converts the Temperature and Humidity columns into numerical values. Invalid values are converted to missing values.

pd.to_datetime(): Converts the Date column into a date-time format.

dropna(): Removes records containing invalid or missing temperature, humidity or date values.

drop_duplicates(): Removes duplicate records from the dataset.

classify_state(): Generates the target class label based on the temperature and humidity values.

State: Represents the class assigned to each sensor reading. The classes used are Normal, Warning and Critical.

train_test_split(): Divides the labelled dataset into training and testing datasets.

stratify: Maintains approximately the same proportion of each class in the training and testing datasets.

DecisionTreeClassifier(): Creates a Decision Tree machine learning classification model.

max_depth: Limits the maximum depth of the decision tree and helps keep the model simple and interpretable.

fit(): Trains the Decision Tree model using the training data.

predict(): Uses the trained model to predict the class of unseen sensor readings.

accuracy_score(): Calculates the percentage of correctly classified test samples.

classification_report(): Displays precision, recall and F1-score for each class.

confusion_matrix(): Shows the number of correctly and incorrectly classified observations for each class.

plot_tree(): Displays the structure of the trained Decision Tree.

IsolationForest(): Creates an unsupervised machine learning model for detecting anomalous sensor observations.

contamination: Specifies the approximate proportion of observations expected to be anomalous. In this experiment, 0.02 represents approximately 2% of the data.

fit(): Trains the Isolation Forest model using the temperature and humidity features.

Isolation Forest prediction: A value of 1 represents a normal observation, while -1 represents a potential anomaly.

Status: Converts the numerical Isolation Forest output into readable labels such as Normal and Anomaly.

detected_anomalies: Contains only the sensor observations identified as anomalous by the Isolation Forest model.

plt.scatter(): Creates a scatter plot showing the relationship between temperature and humidity and highlights detected anomalies.

to_csv(): Saves the classification results and anomaly-detection results as CSV files.

Introducing Manual Anomalies

The original clean dataset should be preserved.

Create a copy of the original dataset and rename it:

anomalous-dht11-temperature-humidity.csv

Artificial anomalies can then be introduced into this copy.

For example:

Anomaly Type	Example Temperature	Example Humidity	Purpose
Normal reading	27.5 °C	60 %	Represents normal sensor operation
High temperature	80 °C	60 %	Demonstrates abnormal temperature
Very low temperature	-10 °C	60 %	Demonstrates abnormal temperature
High humidity	27 °C	98 %	Demonstrates abnormal humidity
Unusual combination	80 °C	10 %	Demonstrates an unusual sensor condition

Only a small number of anomalous records should be introduced. For example, approximately 10–15 anomalous records can be added or modified in a dataset containing several hundred records.

The anomalies should be distributed at different positions in the dataset rather than placed together.

Important: These values are intentionally introduced for demonstrating anomaly detection. They should not be presented as actual DHT11 sensor faults.

The original dataset remains unchanged:

clean-dht11-temperature-humidity.csv

The modified dataset is used for anomaly detection:

anomalous-dht11-temperature-humidity.csv

The output of the anomaly-detection program is:

dht11-anomaly-detection-results.csv

Procedure

Connect the DHT11 sensor to the ESP32 according to the pin connection table.
Connect the ESP32 to the computer using a USB cable.
Configure the ESP32 to read temperature and humidity from the DHT11 sensor.
Connect the ESP32 to a Wi-Fi network.
Transmit the sensor readings to the cloud platform if real-time data collection is being performed.
Export the collected sensor data as a CSV file.
Alternatively, use the provided clean DHT11 temperature and humidity dataset.
Preserve the original clean dataset.
Create a copy of the dataset for anomaly-detection experiments.
Introduce a small number of controlled anomalous temperature and humidity values into the copied dataset.
Open Jupyter Notebook, Google Colab or VS Code.
Install Pandas, NumPy, Matplotlib and Scikit-learn.
Load the clean dataset using Pandas.
Inspect the dataset for missing values and duplicate records.
Convert the temperature, humidity and date columns into appropriate data types.
Generate Normal, Warning and Critical class labels.
Separate the temperature and humidity columns as input features.
Split the labelled data into training and testing datasets.
Create and train the Decision Tree classification model.
Predict the classes of the testing data.
Calculate the classification accuracy.
Generate the classification report and confusion matrix.
Visualize the trained Decision Tree.
Load the anomalous dataset.
Select temperature and humidity as the features for anomaly detection.
Create and train the Isolation Forest model.
Predict normal and anomalous observations.
Display the sensor readings identified as anomalies.
Visualize normal and anomalous observations using a scatter plot.
Save the classification and anomaly-detection results as CSV files.
Observe and compare the machine learning results.

Observations and Output

Classification

The following table can be used to record the classification results:

Sr. No.	Parameter	Observation
1	Input features	Temperature and Humidity
2	Classification model	Decision Tree
3	Number of classes	3
4	Classes	Normal, Warning, Critical
5	Training samples	Record the calculated value
6	Testing samples	Record the calculated value
7	Accuracy	Record the calculated value
8	Precision	Record the calculated value
9	Recall	Record the calculated value
10	F1-score	Record the calculated value

An illustrative classification output is:

Class Distribution

Normal       ...
Warning      ...
Critical     ...

Classification Accuracy:
...

Classification Report:

              precision    recall    f1-score

Normal           ...        ...        ...
Warning          ...        ...        ...
Critical         ...        ...        ...

Anomaly Detection

The following table can be used to record the anomaly-detection results:

Sr. No.	Parameter	Observation
1	Anomaly detection model	Isolation Forest
2	Input features	Temperature and Humidity
3	Contamination	2%
4	Total records	Record the calculated value
5	Detected anomalies	Record the calculated value
6	Normal observations	Record the calculated value

An illustrative output is:

Detected Anomalies:

ID     Temperature     Humidity       Status
...       80.0            60          Anomaly
...      -10.0            60          Anomaly
...       27.0            98          Anomaly

Number of detected anomalies: ...

The scatter plot represents temperature and humidity observations. Normal observations and observations identified as anomalies are displayed using different markers.

Result

The IoT sensor dataset was successfully prepared for machine learning. A Decision Tree classification model was implemented to classify sensor readings into Normal, Warning and Critical states. The classification model was evaluated using accuracy, precision, recall, F1-score and a confusion matrix.

An Isolation Forest model was also implemented to identify anomalous temperature and humidity readings. The detected anomalies were visualized and the processed classification and anomaly-detection results were exported as CSV files.

Thus, machine learning techniques were successfully applied to IoT sensor data for both classification and anomaly detection.

Viva Questions

What is machine learning?
What is the difference between supervised and unsupervised learning?
What is classification?
What is anomaly detection?
What is the difference between classification and anomaly detection?
What are the input features used in this experiment?
What is the target variable in the classification task?
What is a Decision Tree?
Why is a Decision Tree used for classification?
What is the purpose of training and testing datasets?
What is classification accuracy?
What is precision?
What is recall?
What is an F1-score?
What is a confusion matrix?
What is Isolation Forest?
Why is Isolation Forest suitable for anomaly detection?
What does an Isolation Forest prediction of -1 represent?
What does an Isolation Forest prediction of 1 represent?
What is the contamination parameter?
Why are artificial anomalies introduced into the dataset?
Give examples of anomalies in IoT sensor data.
How can anomaly detection be useful in industrial IoT?
What is the difference between an actual sensor fault and an artificially introduced anomaly?
How can the classification model be improved?
How can the anomaly-detection system be improved?