Interaktionsmöglichkeiten mit Händen und Gesten in Table-Top-Projektionen

Interactive interfaces with Kinect depth cameras

Initial situation

The project is for Augmenticon, a company in the pharmaceutical sector. They require a solution for interactive user input in laboratory settings, which poses its own unique set of constraints. Workers should be able to document process steps with minimal distraction. Contact with any objects should be minimized, to reduce contamination risks. The current practice of using paper forms is not satisfactory in that regard.

Problem statement / Project goal

Develop a prototypical application that shows how gestures can be used for interaction with hands and gestures in table-top projections

Solution developed and its benefits

A combination of Python, OpenCV and MediaPipe was chosen as a basis for the software. The MediaPipe Gesture recognition model allowed rapid prototyping without time intensive model training. A test setup was created, and several algorithms were designed and evaluated. Gestures, both static and dynamic were evaluated for use in such an interactive system, and a collection of suitable gestures have been tested. Results show that robust interaction, under active illumination from a projector, even with gloved hands, is possible. A prototypical application for practical demonstrations is presented.

Key terms

• keyword 1Computer Vision
• keyword 2Interaction
• keyword 3Machine learning

Customer

Company Name AG
Augmenticon
Europa-Strasse 19a
CH-8152 Glattbrugg
Website

Team