Development and evaluation of an iOS Prototype to test the feasability of the action classifier for pose detection of excercises by elderly people, to test the iPhone13Pro TrueDepth camera for this usecase.
Xcode, CreateML, Swift, SwiftUI, Machine learning (ML), iOS, iPhone13Pro, TrueDepthCam
The aim of the project is to evaluate how well the TrueDepth technology of the iPhone 13 is suited for this use case and to implement it with iOS. Machine learning is used to classify the human posture on the smartphone. This is purely a real-real-time image processing based on machine learning. These exercises are then automatically analysed and optionally feedback is activated to instruct the user to correct and optimise their form and execution during the exercises.
Regardless of whether we are healthy and fit or suffer from certain health problems, we all need exercise. In this project, there is a special focus on frail older adults who have health problems related to malnutrition and lack of exercise. The Virtual Trainer is designed to help this target group to perform certain exercises correctly and correct them if necessary. As a first step an iOS prototype got created and then tested with different models.
As this is an iOS project, Apple's CreateML framework was used to create machine learning models. This system is effective and suitable, but is very time consuming for video processing because of the large number of data sets that have to be created and calculated when a new model is trained. Tests have shown that the TrueDepth camera has a small advantage over the the normal (wide angle cam) front camera of the iPhone13Pro, for this use case under certain conditions.
Raphael Banz, Omanda AG
https://omanda.ch/
Alessandro Calcagno
Prof. Dr. Arzu Çöltekin (arzu.coltekin@fhnw.ch), Cloe Hüsser (cloe.huesser@fhnw.ch)
