Authors : Swetha Tarigoppula; Sri Nitya Sankranthi; Shivani Kumbham; Dikshita Vedire; Pandari Malakummari

Volume/Issue : Volume 11 - 2026, Issue 8 - August

Google Scholar : https://tinyurl.com/54s5fykr

DOI : https://doi.org/10.38124/ijisrt/26aug1144

Note : A published paper may take 4-5 working days from the publication date to appear in PlumX Metrics, Semantic Scholar, and ResearchGate.

Abstract : Over the past decade, touch-less and gesture-driven interfaces have drawn serious attention from researchers and developers alike. Standard music control tools — keyboards, touchscreens, physical knobs — do the job, but they put a hard boundary between the performer and the sound. Every action requires reaching for something, which breaks the naturalness of the interaction. Rhythm-flow is a browser-based platform built around that exact problem. Users manipulate music playback and trigger synthesized sounds through hand gestures read from an ordinary webcam — no special hardware, no calibration routine. Frames from the camera are processed on the fly using computer vision to extract hand shape and motion data. Media-Pipe Hands is the landmark detection backbone, identifying 21 key-points on the hand per frame. Tone.js handles audio generation while Three.js renders a live visual layer tied to hand movement. Across evaluation sessions, the system averaged roughly 90% gesture recognition accuracy using nothing but a standard laptop webcam.

Keywords : Hand Gesture Recognition, Computer Vision, Media-Pipe Hands, Touch-Less Interaction, Audio Synthesis, Human– Computer Interaction.

References :

1. Y. Sun et al., "Real-Time Gesture Recognition using Edge Computing," 2020. Available: https://arxiv.org/abs/2005.10145
2. A. Katiyar et al., "Hand Gesture Recognition using Mediapipe," IRE Journals, 2023. Available: https://www.irejournals.com/paper-details/1708836
3. K. Lupinetti et al., "3D Dynamic Hand Gesture Recognition using CNN," 2020. Available: https://arxiv.org/abs/2003.01450
4. G. Bradski, "The OpenCV Library," 2000. Available: https://opencv.org
5. F. Zhang et al., "MediaPipe Hands: On-device Real-time Hand Tracking," 2020. Available: https://arxiv.org/abs/2006.10214
6. P. Neto et al., "Real-Time Hand Gesture Recognition using Neural Networks," 2013. Available: https://arxiv.org/abs/1309.2084
7. Y. Meng et al., "Real-Time Hand Gesture Monitoring Model Based on MediaPipe," 2024. Available: https://pmc.ncbi.nlm.nih.gov/articles/PMC11478756/
8. Lavanya Vaishnavi et al., "MediaPipe to Recognise the Hand Gestures," 2022. Available: https://www.researchgate.net/publication/361711114_MediaPipe_to_Recognise_the_Hand_Gestures
9. A. Katiyar et al., "Hand Gesture Recognition using Mediapipe," IRE Journals, 2023. Available: https://www.irejournals.com/paper-details/1708836
10. Google, "MediaPipe Framework Documentation," 2023. Available: https://ai.google.dev/edge/mediapipe/solutions/guide
11. E. Uboweja et al., "On-device Real-time Custom Hand Gesture Recognition," 2023. Available: https://arxiv.org/abs/2309.10858.