This project makes it possible to stream audio recorded by IoT devices of the Mitwelten project and visualizing the occurence of birds cries within. It converts and processes audio data on the fly whenever requested, while keeping the delay for the user at a minimum.
Rust, Axum, React, Docker, REST, audio streaming, on the fly processing, audio visualization
The goal of the media player is to make the playback of recordings possible in real time and in a discoverable manner while also visualizing the data.
The FHNW research project Mitwelten project uses IoT devices to record audio of birds. This data is saved and analyzed by a machine learning model to detect bird cries. This data is vast and is not discoverable or able to be played by someone interested.
The result is a docker image containing a single static binary capable of serving its frontend and backend. It can be embedded into the larger Mitwelten applications by calling the host url of the project with url parameters. It converts audio data on the fly and generate spectrograms as well as waveform visualizations for the data.
Type: IP5
Duration: 19. September 2022 - 24. March 2023
Effort in hours: 180h
Team size: 1
Thomas Amberg, thomas.amberg@fhnw.ch
