Blending the real and virtual worlds

Industry is increasingly using VR/AR applications creatively

By Antoinette Price

Virtual reality (VR) and augmented reality (AR) technology is all around us. Whether playing a mind-blowing game, training for surgery, enhancing classroom learning, or stepping inside a building that hasn’t yet been constructed to solve problems before they happen, diverse industry sectors are using VR/AR applications in creative ways. According to a report by Digi-Capital, a company advising AR/VR, mobile and games leaders in Asia, Europe and the US, AR/VR could hit USD 150 billion revenue by 2020, with AR accounting for USD 120 billion and VR for the remaining USD 30 billion.

VR_for_engineering
Using 3D modelling tools and visualization techniques in the design process (Photo: www.vrs.org.uk)

So what's the difference?

Both technologies aim to immerse the user with headsets providing stereo 3D high definition video and audio, but they do it differently. VR users wear a closed headset that blocks out the external world and they are usually seated. Though popular for gaming, VR is also used in medical, military and educational situations.

AR by design remains connected to the real world, which the user can always see. Smart glasses can be worn anywhere, while doing most things. Additionally, many mobile phone apps use AR, making its potential impact immense thanks to hundreds of millions of mobile phone users and major investors such as Apple, Google, Microsoft, Facebook and others.

How does the technology work?

AR and VR apps are evolving at a great rate. Complex objects can be tracked and rendered in real-time, while new sensing technologies (depth cameras and miniature 3D scanners) enable mobile devices to “see” the world and digitally capture a surrounding environment to be analysed and augmented with real-time feedback.

The hardware required consists of processors, sensors and input devices (speech and gesture recognition systems), displays (monitors and handheld devices such as smartphones and tablets which contain microelectromechanical systems – MEMS – sensors including optical, accelerometers, gyroscopes, GPS and cameras for tracking), eyeglasses, head-up display (HUD), computers, software and algorithms which help AR systems realistically integrate augmentations with the real world. Special 3D AR software programmes allow the developer to link animation or contextual digital information in the computer programme with an AR "marker" in the real world.

Even the virtual world needs standardization

Standardization ensures that manufacturers, companies and users can produce, sell and use reliable, safe products and services. Several IEC technical committees (TCs) and subcommittees (SCs) prepare International Standards for the technology behind AR applications.

ISO/IEC JTC 1/SC 29: Coding of audio, picture, multi-media and hypermedia information, has published ISO/IEC 23000-13, Information technology - Multimedia application format (MPEG-A) – Part 13: Augmented reality application format. ARAF reference software and conformance. This Standard focuses on the data formats used to provide an AR presentation and is designed to enable the use of 2D/3D multimedia content.

The work of IEC TC 47: Semiconductor devices, and IEC SC 47F: Microelectromechanical systems, ensures that sensors and microelectromechanical systems (MEMS) work reliably and efficiently.

IEC TC 100: Audio, video and multimedia systems and equipment, produces Standards which contribute to the quality and performance of these systems and to their interoperability with other systems and equipment.

From the imagined to the real

Over the last five years AR applications have matured in the architecture, engineering and construction (AEC) industries. For example, the SmartReality mobile AR app puts a 3D model in context by placing smart devices before a 2D set of plans or an actual site. It then recognizes the design and the screen overlays a virtual model of what the project will look like once completed. By virtually walking through this model, architects and contractors can review and avoid constructability issues at the planning stage. It also helps with the prefabrication of building components, and clients can see how their future building will look in the existing conditions and better visualize the space and layout.

Manufacturing goes between the cyber and the physical

More businesses in aerospace, automotive, energy, defence and medical are using AR and VR manufacturing and production apps to plan production and assembly processes fully and virtually, down to the finest details of production line location, flow, cleaning and maintenance.

Engineers can “walk” inside a virtual power plant, manipulate a turbine model in real time, view products at life size, judge required components more accurately and measure walkways to improve safety. These apps also facilitate product reviews, simulation analyses, field service training, sales and marketing and customer and supplier interaction.

Faster, safer and more efficient car manufacturing

This technology integrates intelligent systems in factories to improve productivity, efficiency and accuracy of manufacturing sites.

A leading German car manufacturer uses AR wearable technology to discover flaws before embarking on full production.

During production line quality control, assessors take photos and record videos of the components with their smart glasses, replacing less accurate handwritten notes. They finalize reports by voice at the vehicle, instead of walking between cars to input data at computer terminals. This avoids workflow disruption and saves time, given that many cars require between 10 and 25 tests.

Improving the entire train travel experience

A French multinational global leader in rail transport markets uses a VR room to improve its design, industrialization and maintenance procedures. Here it evaluates technical issues (collisions, malfunctions), shows clients its future trains and does research.

The room seats 23, has a large image wall and stereoscopy capabilities. Ten personal computers PCs and four video projectors display 3D images, while a tracking system follows the user’s perspective in real time. The user is connected to the company’s data management system and can display and interact with any 3D model.

Streaming surgical training around the world

In April this year, the Medical Realities app enabled the first VR live stream operation to be watched worldwide, from a UK Hospital. A 360° camera above the operating table allowed viewers to zoom into any part of the theatre, while the surgeon explained procedures. Students from as far away as Bangladesh used smartphones, headsets and tablets to get an unobstructed view. This experience could help train doctors in developing countries which do not have the same access to operating theatres and the latest technology.

Surgeons located remotely can help colleagues who are operating via the AR/VR app called VIPAAR. The operating surgeon’s Google Glass sends actual surgery images to the screen of a colleague located elsewhere. The remote surgeon then reaches into the virtual field and points out deeper structures or advises on the placement of surgical instruments. These new images are in turn sent back to the operating surgeon’s Google Glass. This assistance offers an extra layer of safety from an expert who is located remotely. 

As different industries increasingly incorporate AR and VR applications in their work processes, IEC will follow developments and continue its important work in standardization and quality assurance, so contributing to the performance, reliability, safety and interoperability of this technology.

Find out how the leisure industry is using AR/VR in our other article, Experiencing life like never before.

Gallery
BMW_VR Using virtual reality to develop new vehicle models (Photo: BMW)
VR_for_engineering Using 3D modelling tools and visualization techniques in the design process (Photo: www.vrs.org.uk)
VR_medical_training Medical training application in the CAVE2™ Hybrid Reality System by the Electronic Visualization Laboratory (EVL), University of Illinois, Chicago. CAVE2 is a trademark of the University of Illinois Board of Trustees (Photo: EVL)