Case 072024IoT Engineer & ML Engineer

Hand Sign Recognition with IoT

AI-powered smart glove system

A smart glove that reads hand signs. Flex and motion sensors on an ESP8266 classify gestures at the edge, with heavier analytics offloaded to a Flask service on EC2. Built to find out how much of a gesture pipeline can actually run on a microcontroller. 95.7% on the test set.

PythonC++Arduino NanoESP8266MPU-6500Flex Sensors+4

1 / 2

95.7% accuracy across variable hand positions

<150ms inference latency

20+ sign language gestures recognized

SnapshotAt a glance

IoT Engineer & ML Engineer

3 months

Outcomes

95.7% accuracy across variable hand positions

<150ms inference latency

20+ sign language gestures recognized

3,000+ custom training samples

Hybrid Edge-Cloud inference pipeline

Technologies Used

10 Technologies Integrated

Python

C++

Arduino Nano

ESP8266

MPU-6500

Flex Sensors

AWS EC2

Flask

scikit-learn

Sensor Fusion

95.7%

Impact

Key Features

Feature Implementation

10 Features

90%

Feature CoverageProject Scope

Project Vision

Transform hand signs and sign language gestures into accessible communication through AI-powered IoT systems.

Core Process

The process of Developing it.

Designed IoT hardware with sensors, collected 3,000+ gesture samples, trained orientation-invariant model, deployed hybrid Edge-Cloud inference pipeline.

Build notesWhat I built

IoT Gesture Recognition Hardware
Prototyped a low-latency (150ms) IoT Data Glove integrating 5-axis Flex sensors and MPU-6500 gyroscopes to digitize and classify 20+ sign language gestures via sensor fusion algorithms.
Machine Learning Classification Model
Trained an orientation-invariant classification model on 3,000+ custom samples, achieving 95.7% accuracy across variable hand positions and eliminating gravity-dependence drift.
Hybrid Edge-Cloud Inference Pipeline
Architected a hybrid Edge-Cloud inference pipeline, processing real-time signals on ESP8266 (Edge) while offloading complex analytics to AWS EC2 via Flask, keeping the edge and cloud stages in sync.

Inspiration

The need to enhance communication accessibility for hearing-impaired individuals through real-time gesture recognition.

Features

Real-time hand sign recognition
20+ recognized gestures
95.7% model accuracy
<150ms inference latency
IoT sensor integration (MPU-6500, Flex Sensors)
Cloud API deployment on AWS EC2
99.9% API uptime
Microgravity-compatible gesture recognition
Wi-Fi data transmission via NodeMCU
RESTful API with Flask

Challenges

Sensor calibration and data synchronization
Gravity-dependence drift in orientation data
Real-time Edge-Cloud data transmission

Solution

Implemented sensor fusion algorithms and orientation-invariant feature extraction to eliminate drift and achieve consistent accuracy.