Motion-Driven Synthetic IMU Data Generation

We generate human motions from text prompts and synthesize matching IMU signals via an interactive web application.

Initial situation

Collecting large and diverse IMU datasets in real-world environments can be resource-intensive and time-consuming. Existing approaches often require time-consuming recording campaigns to cover a wide variety of activities and sensor placements.

Project goal

Our goal was to build a system that generates human motion from natural-language prompts describing the action, and then synthesizes temporally aligned IMU data for virtually placed sensors. To make this accessible, we built a web application that lets users place sensors and generate/export motions and IMU streams interactively.

Solution and benefits

We implemented a diffusion-based model to generate motion from textual descriptions and an IMU synthesis pipeline that attaches multiple virtual sensors to the avatar with realistic placement and orientation. Everything is wrapped in a web interface for fast experimentation.

• Text-to-motion generation
• Virtual IMUs with arbitrary placement & orientation on the avatar surface
• Export of synchronized accelerometer and gyroscope data
• Interactive web app

In a proof-of-concept, synthetic IMU traces qualitatively resembled real recordings. Remaining challenges include high-frequency jitter in both the generated motions and the synthesized IMU data, as well as limited overall motion realism.

Key terms

• motion generationcreating a motion sequence for a virtual avatar
• diffusion modela generative model used here to produce motions from text
• SMPLa parametric body model used to represent motion
• IMU synthesiscreating synthetic accelerometer and gyroscope data from motion

Team