Case 072024
Hand Sign Recognition with IoT
AI-powered smart glove system
A smart glove that reads hand signs. Flex and motion sensors on an ESP8266 classify gestures at the edge, with heavier analytics offloaded to a Flask service on EC2. Built to find out how much of a gesture pipeline can actually run on a microcontroller. 95.7% on the test set.
1 / 2Impact
- 0195.7% accuracy across variable hand positions
- 02<150ms inference latency
- 0320+ sign language gestures recognized
What I built
IoT Gesture Recognition Hardware
Prototyped a low-latency (150ms) IoT Data Glove integrating 5-axis Flex sensors and MPU-6500 gyroscopes to digitize and classify 20+ sign language gestures via sensor fusion algorithms.
Machine Learning Classification Model
Trained an orientation-invariant classification model on 3,000+ custom samples, achieving 95.7% accuracy across variable hand positions and eliminating gravity-dependence drift.
Hybrid Edge-Cloud Inference Pipeline
Architected a hybrid Edge-Cloud inference pipeline, processing real-time signals on ESP8266 (Edge) while offloading complex analytics to AWS EC2 via Flask, keeping the edge and cloud stages in sync.
Inspiration
The need to enhance communication accessibility for hearing-impaired individuals through real-time gesture recognition.
Features
- Real-time hand sign recognition
- 20+ recognized gestures
- 95.7% model accuracy
- <150ms inference latency
- IoT sensor integration (MPU-6500, Flex Sensors)
- Cloud API deployment on AWS EC2
- 99.9% API uptime
- Microgravity-compatible gesture recognition
- Wi-Fi data transmission via NodeMCU
- RESTful API with Flask
Challenges
- Sensor calibration and data synchronization
- Gravity-dependence drift in orientation data
- Real-time Edge-Cloud data transmission
SnapshotAt a glance
IoT Engineer & ML Engineer
3 months
95.7% accuracy across variable hand positions
<150ms inference latency
20+ sign language gestures recognized
3,000+ custom training samples
Hybrid Edge-Cloud inference pipeline
Technologies Used
10 Technologies Integrated
Impact
Key Features
Feature Implementation
Project Vision
Transform hand signs and sign language gestures into accessible communication through AI-powered IoT systems.
Core Process
The process of Developing it.
Designed IoT hardware with sensors, collected 3,000+ gesture samples, trained orientation-invariant model, deployed hybrid Edge-Cloud inference pipeline.
