Meta and BMW collaborate to explore AR/VR tech inside vehicles

Meta has announced its partnership with BMW Group to investigate the integration of AR and VR technology within fast-moving vehicles. This joint research project aims to enhance the passenger experience and potentially revolutionize travel in cars, trains, planes, and beyond.

The collaboration between Meta and BMW, established in 2021, envisions a future where AR and VR are seamlessly integrated into smart vehicles, providing passengers with more productive, social, and entertaining experiences on the road. By leveraging cutting-edge technology, this partnership seeks to unlock new forms of hands-free communication, entertainment, and utility, transcending the limitations of current screens and instruments found in vehicles today.


One of the challenges encountered in incorporating VR technology into moving vehicles is the conflict between inertial motion sensors (IMUs) and cameras. While VR headsets, such as Meta Quest 2, employ these sensors for precise motion tracking, they struggle to display stable virtual content when the vehicle is in motion. To address this issue, Meta has worked closely with BMW to integrate real-time IMU data from a BMW car's sensor array into the tracking system of the Project Aria research glasses.


This significant breakthrough enables the system to calculate the glasses' location relative to the vehicle, overcoming the conflict between camera-observed motion inside the car and IMU-measured acceleration and rotational velocity relative to the world. By transferring this tracking system to the Meta Quest Pro, virtual objects can now be accurately anchored to a moving car using a digital twin representation. This advancement has allowed the demonstration of compelling virtual and mixed reality passenger experiences within moving vehicles.
Moving forward, Meta and BMW are committed to leveraging the advanced machine perception capabilities of modern cars to explore future possibilities. Access to the car's precise six degrees of freedom (6DOF) positioning system could enable the rendering of world-locked virtual content outside of the vehicle, including the identification of landmarks and other points of interest. Such capabilities are expected to be invaluable for future AR glasses and personalized AI assistants.


The Meta-BMW partnership signifies a significant step towards revolutionizing the passenger experience and paving the way for new advancements in AR and VR technology. By merging innovation from both companies, the collaboration aims to unlock the full potential of immersive experiences within vehicles, setting the stage for the future of transportation

Media
@adgully

News in the domain of Advertising, Marketing, Media and Business of Entertainment

More in Media