This project addresses the necessity of safety features in virtual reality. These features are immersion breaking and the developed prototypes aim to remove the necessity of certain safety features in VR.
Programming Language: C#
Game Engine: Unity
Frameworks: AR Foundation, Vuforia
The goal of this project is to find and determine possible implementations to merge real objects from the surrounding reality into the virtual world, so that the experience is not compromised by the user’s environment in which they are currently in. The VR camera must detect real-world obstacles in the play area of the user. Detected obstacles can then be replaced by suitable virtual reality objects. The solution process should in theory be scalable to multiple use cases. Safety while using the application must be ensured without breaking immersion. Objects should become visible, partly visible, or invisible based on their orientation in respect of the users point-of-view and other objects. The user’s experience should not be compromised by the render-time of objects being added or reappearing in virtual space. The implementations must be evaluated based on their efficiency by numeric values such as detection range. Evaluations based on their quality of immersion will be conducted through tests. Multiple proof-of-concepts were designed and implemented based on existing software and hardware technologies. Following the evaluation of the proof-of-concepts, technical limitations were determined, and possible solutions are presented.
The project goals and problem statements were created based on the current state of safety and immersion in virtual reality. With current virtual reality technologies people can easily immerse themselves in new and exciting environments and escape reality. These virtual worlds can be interacted with and in some instances be physically traversed. The problem with the technology is keeping the user immersed throughout the experience without immense preparation or massive immersive facilities. Obstructions such as walls, furniture or even other people can break immersion due to the objects being invisible in the virtual world and the user colliding with them. To avoid this, people using virtual reality technology must define “play areas” in which movement is possible. This can be performed with spatial mapping or plane detection from the headset or by the user directly inputting the dimensions of the play area. Upon reaching the edges of these areas, immersion must be broken to avoid user injury using warnings such as sounds or opening a portal to show the real environment. Currently this is a problem that has not been solved, in which these boundaries are no longer defined by the user, but instead recognized by technology and be seamlessly integrated into the virtual space without notifying the user or breaking immersion. Boundaries and obstructions should keep their real physicality in virtual environments, e.g., while a table may be visually replaced with a boulder, the user should still be able to physically interact with it without breaking the immersion. These objects rather than becoming obstructions or obstacles would instead integrate with the virtual world and add to the scene itself.
The first prototype detects real-world objects using the ML (machine learning) model Vuforia and tracks them in real time. These objects’ appearance, orientation and position in the real world are displayed as a 3D model in VR (virtual reality) and allow the user to interact with them.
In the above image, the user is holding a real-world yoga brick and the detected object is being physically manipulated while being tracked in the virtual world.
The second prototype detects the surfaces of obstacles in the real world and overlays these obstacles with plane primitives. These primitives can be replaced with 3D meshes to fit the world’s story. The user has nearly half of second to avoid colliding with real world objects using this detection method, this is including the average human’s reaction time.
Using the above method, under normal conditions (the user is walking at a normal pace), the user will always have enough time to react to any new detected objects.
These objects warn the user of a collision without breaking immersion.
Project Duration: 18.09.2023 - 19.01.2024
Person hours: 360 hours
Team size: 2
Iro Armeni
iarmeni@stanford.edu
Jarno Degano, jarno.degano@students.fhnw.ch | BSc iCompetence
Kyle Egli, kyle.egli@students.fhnw.ch | BSc iCompetence
Arzu Cöltekin, arzu.coltekin@fhnw.ch
Martin Lacayo, martin.lacayo@fhnw.ch
Yves Simmen, yves.simmen@fhnw.ch
