Science and Arts Intersection: Open the Magic AR Tour of HoloLens

The sharing session on May 9th was held as scheduled in B22. The sharing guest was Mr. Jin Yunshui. He shared his experience of developing the AR function for HoloLens during his stay in the UK. Participating students also took the opportunity to experience the amazing AR of HoloLens.

During the sharing session, Mr. Jin showed students some of his research achievements in the UK. He introduced a demo which is based on Microsoft's augmented reality devices HoloLens and Unity. The difference between augmented reality devices and virtual reality devices is that users can see the real world when using augmented reality devices.

Mr. Jin synced the active picture in HoloLens to a computer, so that all interested students can watch the effects of the Demo. The reason why HoloLens is expensive is that Microsoft's HoloLens has many unique functions, such as its excellent spatial scanning function. Mr. Jin explained that after the scan is successful, the user can see the spatial grid structure of the scanned object. At the same time, the user can control the scanning interval (usually per 3-8 seconds) and after-scanning operation.

After the spatial scanning, previous 3D-scanned model can be processed with retopology. The results after retopology will be more accurate, but can no longer be adjusted by the user. The default is 8×8cm grid. After the spatial scanning, HoloLens will create a closed space. On this basis, Microsoft can also do space understanding, that is, it can identify the ground, the wall, the roof, the area where you can sit, etc. With these functions, we can do a lot of interesting things with HoloLens.

Mr. Jin then showed the students the demos made with HoloLens. Because HoloLens is an augmented reality device that can watch the real world, the virtual scene and the real world may have overlapping and shading interactions. In HoloLens, these interactions are also realized with retopology. Mr. Jin also showed HoloLens' gesture recognition and voice control functions. There are mainly two gestures: click and throw. The click gesture means the click of a button that can realize the effect of mouse click, the gesture of throwing is to activate the main menu, and these functions are triggered in the form of events in HoloLens. Voice recognition is implemented using Microsoft library. In the Demo, Mr. Jin often uses voice commands such as "scan mode", "stop scan", "analysis mode", "analysis complete", "take a sit" and "can you push up" to interact with the characters in HoloLens and Demo.

Through gesture recognition and voice control, Mr. Jin made the characters in the Demo do what’s asked such as sitting down in the sitting area and doing push-ups.

At the end of the session, students got the opportunity to try in person the amazing AR functions in HoloLens.