Case 072024IoT Engineer & ML Engineer

Hand Sign Recognition with IoT

AI-powered smart glove system

Hand Sign Recognition with IoT was built from a vision: communication should be accessible to everyone. Powered by embedded sensors and machine learning, I created an AI-powered IoT smart glove system that recognizes hand signs and sign language gestures in real-time - transforming gestures into understanding with 95.7% accuracy.

PythonC++Arduino NanoESP8266MPU-6500Flex Sensors+4

1 / 2

95.7% accuracy across variable hand positions

<150ms inference latency

20+ sign language gestures recognized

SnapshotAt a glance

IoT Engineer & ML Engineer

3 months

Outcomes

95.7% accuracy across variable hand positions

<150ms inference latency

20+ sign language gestures recognized

3,000+ custom training samples

Hybrid Edge-Cloud inference pipeline

Technologies Used

10 Technologies Integrated

Python

C++

Arduino Nano

ESP8266

MPU-6500

Flex Sensors

AWS EC2

Flask

scikit-learn

Sensor Fusion

95.7%

Impact

Key Features

Feature Implementation

10 Features

90%

Feature CoverageProject Scope

Project Vision

Transform hand signs and sign language gestures into accessible communication through AI-powered IoT systems.

Core Process

The process of Developing it.

Designed IoT hardware with sensors, collected 3,000+ gesture samples, trained orientation-invariant model, deployed hybrid Edge-Cloud inference pipeline.

Build notesWhat I built

IoT Gesture Recognition Hardware
Prototyped a low-latency (150ms) IoT Data Glove integrating 5-axis Flex sensors and MPU-6500 gyroscopes to digitize and classify 20+ sign language gestures via sensor fusion algorithms.
Machine Learning Classification Model
Trained an orientation-invariant classification model on 3,000+ custom samples, achieving 95.7% accuracy across variable hand positions and eliminating gravity-dependence drift.
Hybrid Edge-Cloud Inference Pipeline
Architected a hybrid Edge-Cloud inference pipeline, processing real-time signals on ESP8266 (Edge) while offloading complex analytics to AWS EC2 via Flask, ensuring seamless data transmission.

Inspiration

The need to enhance communication accessibility for hearing-impaired individuals through real-time gesture recognition.

Features

Real-time hand sign recognition
20+ recognized gestures
95.7% model accuracy
<150ms inference latency
IoT sensor integration (MPU-6500, Flex Sensors)
Cloud API deployment on AWS EC2
99.9% API uptime
Microgravity-compatible gesture recognition
Wi-Fi data transmission via NodeMCU
RESTful API with Flask

Challenges

Sensor calibration and data synchronization
Gravity-dependence drift in orientation data
Real-time Edge-Cloud data transmission

Solution

Implemented sensor fusion algorithms and orientation-invariant feature extraction to eliminate drift and achieve consistent accuracy.