This project explores real-time object recognition within a simulated VR environment.
Unity, YOLO, R-CNN, EfficientDet, VR, Object Recognition, Depth Mapping, Flask, Python
The objective of the VR Everywhere II project is to develop a proof of concept for integrating real-time object recognition into a virtual reality environment, focusing on adaptive and immersive user experiences.
Current VR systems typically require a pre-mapped or fixed space to function effectively, which limits their flexibility and adaptability in dynamic environments. These systems often rely on predefined safe zones or detailed mapping of the physical environment to ensure accurate object recognition and user safety. However, this approach restricts the user experience, particularly in situations where the VR environment needs to adapt to new or changing surroundings. Additionally, privacy concerns arise when accessing live camera feeds, as most VR devices restrict real-time data access to protect user privacy. To address these challenges, this project utilizes a simulated environment combined with advanced object detection models like YOLO, Faster R-CNN, and EfficientDet. The goal is to explore methods for real-time object recognition that do not rely on pre-mapped spaces, enabling more flexible and responsive VR experiences.
The VR Everywhere II project successfully demonstrated the feasibility of integrating real-time object recognition within a simulated virtual reality (VR) environment. By employing advanced detection models such as Faster R-CNN the system was able to identify various objects in real-time, providing a proof of concept for more dynamic and interactive VR experiences. However, the project also revealed several challenges, particularly in the areas of precision and accuracy in depth mapping. The system struggled with correctly positioning objects within the 3D space, especially when dealing with occlusions or objects positioned at angles. Additionally, real-time performance was hindered by processing delays, which impacted the fluidity of user interaction and the overall immersion of the VR environment.
Despite these challenges, the project laid crucial groundwork for future research and development in adaptive VR systems. The findings highlighted key areas that require further optimization, such as improving the accuracy of depth data processing, reducing latency in object recognition, and enhancing the system's ability to adapt to changing environments without relying on pre-mapped spaces. The project also underscored the importance of balancing real-time processing capabilities with the need for high precision, particularly in applications where safety and user experience are paramount. The insights gained from this work pave the way for future advancements in VR technology, with the potential for real-world applications in areas such as healthcare, industrial training, and immersive entertainment.
Iro Armeni, Professor at CEE Dept., Stanford University
