The research collects the important information for preprocessing and delivers an insight of the techniques and challenges. Furthermore, by implementing a proof of concept for the annotation challenge, a part of the preprocessing pipeline, a viable way of handling datasets and its data is introduced.
Datasets, Multimodal, Preprocessing, Angular, Electron, Annotation Challange, Emotion recognition
The research study revolves around three core research questions, each addressing distinct aspects of data preprocessing and interface design within the context of multimodal machine learning.
Research Question A:
How can the existing solutions for preprocessing data streamlines be improved via optimizing the procedures and developing a software prototype on existing frameworks?
The primary goal of Research Question A is to explore avenues for enhancing existing data preprocessing solutions by optimizing their procedures and developing a software prototype within established frameworks. The objective here is to identify bottlenecks, inefficiencies, or gaps in current preprocessing methods and propose improvements that streamline the entire data preparation process.
Research Question B:
What is the state of the art of preparing data for multimodal machine learning?
The central objective of Research Question B is to comprehensively investigate the current landscape of solutions and approaches utilized in preparing data for multimodal machine learning. This involves an in-depth analysis of the existing techniques, methods, and strategies employed in handling multimodal data sources.
Research Question C:
How should a user-friendly interface be designed for the use of scientists and outsourcing labeling efforts for online data collection?
Research Question C focuses on the design aspect of creating a user-friendly interface tailored to the needs of scientists and outsourcing labeling efforts for online data collection. The objective is to devise an intuitive and accessible interface that enables scientists to interact with the preprocessing tool effectively.
This project is in collaboration with the startup Earkick, which provides a multimodal machine learning prototype for pre-built databases to recognize human emotions as well as a platform to gather data and integration of the tool. Multiple companies have existing technologies and ways for processing data to be used in machine learning. Those methods do not fit every type of data and are different in each their own ways with pros and cons that help identify problems in this area. In case of multi modality, this thesis focuses on emotion recognition data extracted from images and videos. The fascinating area of data combined with modern technologies create an interesting topic with an open limit for exploration.
The Exploration shows that regardless of the technique used and processes applied, crafting a useful dataset demands high costs and effort. However, investing such resources greatly benefits the system. Additionally, it supports the claim of needing a tool to aid managing such datasets pre integration in machine learning algorithms. To contribute to this problem, a concept of a tool for handling datasets was created with the ability to extend and adapt to different preprocessing techniques. The concept consists of architectural decisions, visual designs and functional ideas. The evaluation of the functionality resolved around emotion recognition and is therefore specialized in annotating feelings and sentiment.
| Projekttyp | Bachelorarbeit Informatik IP6 |
| Projektdauer | 20. Februar 2023 - 18. August 2023 |
| Teamgrösse | 1 Student |
| Aufwand in Personenstunden | ca. 360 Stunden |
| Client | Earkick |
| Name | Gagan Narula |
| Website | Earkick |
Cyril Lindenmann, BSc Informatik, cyril.lindenmann@students.fhnw.ch
Arzu Cöltekin arzu.coltekin@fhnw.ch
Leticia Fernández Moguel leticia.fernandezmoguel@fhnw.ch
