When Volkswagen was testing augmented reality in a head-up display on a vehicle – those holographic directional arrows, driving speed and other information that appear on a windshield to guide drivers – it thought about using Microsoft’s HoloLens AR headset.

But the problem was that when the HoloLens was put in a moving vehicle, its sensors lost their tracking ability and its holograms disappeared. That’s because the headset’s two types of sensors − visible light cameras and an inertial measurement unit (IMU) – disagreed about whether there was motion. The IMU recognized motion and the cameras did not.

This is similar to why humans get motion sickness: when the eyes see one thing but other senses feel something else.

Volkswagen approached Microsoft to find a solution to this problem.

To get around this impasse, VW connected an in-car GPS to the headset to give the wearable a better understanding of its position and surroundings. The German automaker came upon the discovery as it was tinkering with HoloLens in a bid to create head-up displays for cars, providing drivers with information on things such as speed and direction.

It addresses a long-standing HoloLens issue of moving surroundings confusing the headset’s sensors. The enhancement was developed in collaboration with Volkswagen, which has been experimenting with using the headset as a head-up display in its vehicles.

“We think mixed reality information is the most intuitive information we could provide to enhance our customers’ user experience,” said Andro Kleen, head of the data science team at Volkswagen Group Innovation. “What you see there, and what you need to process, is very close to what humans normally see and process. It’s not so abstract.”

VW initially began using HoloLens as a means to teach inexperienced drivers to become more confident on the roads. However, the motion understanding issue caused this use case to flounder.

Since the fix a few months ago, Microsoft said it has begun rolling out the moving platform feature, which has already attracted interest from several firms in the maritime space as it is currently only available for use on large ships. However, the company said it plans to refine and expand the service for use in elevators, trains, cars and other moving environments.

“We think of this as moving toward a mobility system where different products and mobility solutions will be connected,” Kleen added. “The basic assumption is that this technology will become lighter and smaller … (and) as that happens, more people will get their hands on it and integrate it into their daily lives — and thus into their way of moving from A to B.”

One market that has expressed interest in Microsoft’s hologram headset is defense. Last April, the U.S. Army struck a $21.9 billion deal with the tech provider to purchase 120,000 HoloLens systems. The units will be custom-built and will be used to project digital images in the wearer’s field of view of battlefield objects and objectives.

This article first appeared in IoT World Today’s sister publication AI Business. 

Log in with your IoT World Today account

Alternatively, post a comment by completing the form below:

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

This white paper by @braincubeEn explores how the changes of 2020 and 2021 are shaping the future of #IIoT. Learn w… twitter.com/i/web/status/1…

Driving in the Future: Danger Alerts in Less Than 1 Second dlvr.it/SQJp0Q https://t.co/72SL5KwYDj

Oshkosh, @RoboticResearch team to bring automation to service vehicles #autonomousvehicles dlvr.it/SQJg59 https://t.co/ft2cVLDRda

Access the insights on IoT deployments, emerging tech and new applications now. Sign up to our dedicated… twitter.com/i/web/status/1…

Robots need more human skills dlvr.it/SQGtF8 https://t.co/ketVJNlVpR

@IoTWorldSeries and The #AISummit are joining forces again this year to take the shows to new heights. Now located… twitter.com/i/web/status/1…

RGo Robotics exits stealth mode after completing $20 million investment round, making its Perception Engine technol… twitter.com/i/web/status/1…

@Hyundai has plans to roll out its #selfdriving #robotaxi next year, first in Las Vegas before expanding to other m… twitter.com/i/web/status/1…

Sign up for IoT World Today newsletters: vertical industry coverage on Tuesdays and horizontal tech coverage on Thursdays.

Our Special Reports take an in-depth look at key topics within the IoT space. Download our latest reports.

Find the latest white papers and other resources from selected vendors.

Want to reach our audience? Access our media kit.