New ways of seeing, working and doing business

Virtual and augmented reality applications can improve surgery techniques, increase manufacturing efficiency and take the way we view sports to a new level

By Antoinette Price

Initially developed for military and subsequently gaming scenarios, virtual reality (VR) and augmented reality (AR) applications have found their way into many industries, which are enhancing their products and services through innovative technology.

VR medical
Medical training application in the CAVE2™ Hybrid Reality System by the Electronic Visualization Laboratory (EVL), University of Illinois, Chicago. CAVE2 is a trademark of the University of Illinois Board of Trustees (Photo: EVL)

The booming augmented and virtual worlds

According to a report by Digi-Capital, a company advising AR/VR, mobile and games leaders in Asia, Europe and the US, AR/VR could hit USD 150 billion revenue by 2020, with AR accounting for USD 120 billion and VR for the remaining USD 30 billion.

The hardware that changes perspectives

Virtual reality replicates an environment and a perception of three-dimensional depth of space. It does this through computer simulation conveying audio and visual information via a range of multimedia systems. Users wear head-up displays (HUDs), which completely block the real world. Augmented reality, on the other hand, inserts live elements into a real-life environment, overlaying additional information. It is mostly done on a variety of displays, for activities such as driving or flying. Users wear smart glasses; however, in the future AR could also appear on windshields of cars, providing drivers with extra information.

How does the technology work?

AR and VR apps are evolving at a great rate. Complex objects can be tracked and rendered in real-time, while new sensing technologies (depth cameras and miniature 3D scanners) enable mobile devices to “see” the world and digitally capture a surrounding environment to be analyzed and augmented with real-time feedback.

Microelectromechanical systems (MEMS) sensors can be found in everything from automobiles, toys, PCs, medical devices and industrial applications to – more recently – portable consumer electronics. This is due to a reduction in their cost, size and power consumption. MEMS, which detect the orientation of a device, the direction in which it is moving and its absolute location in three dimensional space, are a key part of the technology required for location-based services using AR and VR applications.

The hardware required consists of:

  • Processors
  • Sensors and input devices (speech and gesture recognition systems)
  • Displays including monitors and handheld devices such as smartphones and tablets which contain microelectromechanical systems – MEMS – sensors including optical, accelerometers, gyroscopes, GPS and cameras for tracking)
  • Eyeglasses, HUDs
  • Computers, software and algorithms which help AR systems realistically integrate augmentations with the real world
  • Special 3D AR software programmes allow the developer to link animation or contextual digital information in the computer programme with an AR "marker" in the real world

Standardization behind the technology scenes

Standardization ensures that manufacturers and companies can produce and sell reliable, safe products and services. Several IEC Joint Technical Committees (JTCs), Technical Committees (TCs) and Subcommittees (SCs) prepare International Standards for AR and VR applications and the technology that enables them.

IEC SC 47E: Discrete semiconductor devices, and IEC SC 47F: Microelectromechanical systems, prepare a number of International Standards that enable manufacturers to build better, more efficient, more reliable sensors and MEMS. They facilitate the design, manufacture, use and reuse of MEMS.

ISO/IEC JTC1/SC 29: Coding of audio, picture, multi-media and hypermedia information, has published ISO/IEC 23000-13, Information technology - Multimedia application format (MPEG-A) – Part 13: Augmented reality application format. This International Standard focuses on the data formats used to provide an AR presentation and is designed to enable the use of 2D/3D multimedia content.

Electronic displays are everywhere, in the HUDs used in VR hardware, smartphones, tablets, laptops, TV screens and other portable devices on which AR and VR applications are used. IEC TC 110: Electronic display devices, prepares International Standards for all of them. Because of the increased use of different eyewear display technologies, TC 110 decided to create an ad hoc group, ahG 12, in Oct 2015. It is tasked with developing an outline of the technical requirements for eyewear displays.  

Many of these products are powered by batteries, for which IEC TC 21: Secondary cells and batteries, develops product Standards.

The growing list of industries using AR and VR

Surrounding us today, VR and AR applications are already being used in many fields. As the technology develops, more industries will find new ways to tailor AR and VR to their area of expertise. Below are some examples:

Addressing health issues, surgery and training

AR is transforming the medical and healthcare sectors significantly for patients, doctors and pharmacy management alike.

In April this year, the Medical Realities app enabled the first VR live stream operation to be watched worldwide, from a UK Hospital. A 360° camera above the operating table allowed viewers to zoom into any part of the theatre, while the surgeon explained procedures. Students from as far away as Bangladesh used smartphones, headsets and tablets to get an unobstructed view. This experience could help train doctors in developing countries which do not have the same access to operating theatres and the latest technology.

PTSD Coach, an app for veterans and military service members with post-traumatic stress disorder, provides information about professional care, self-assessment, support and tools for managing daily stresses.

Sports broadcasting

From the Superbowl to the World Cup and Olympics, the way spectators watch sports has changed forever.

A new sports broadcasting technology is making fans feel as if they are actually at the game. 360-degree VR is an audio-visual simulation of an altered environment around users wearing headsets. It can include live, real-time or pre-recorded footage, which enables users to look around the stadium in all directions, as they would in real life. Viewers of American football, athletics, basketball, boxing, golf, gymnastics, motorcycle racing, soccer, tennis, volleyball and more can stream live in VR.

American football and some European soccer players also use this technology very successfully for training.

Manufacturing goes between the cyber and the physical

More businesses use AR and VR manufacturing and production apps to plan full production and assembly processes virtually, down to the finest details of production line location, flow, cleaning and maintenance.

Engineers can “walk” inside a virtual power plant, manipulate a turbine model in real time, view products at life size, judge required components more accurately and measure walkways to improve safety. These apps also facilitate product reviews, simulation analyses, field service training, sales and marketing as well as customer and supplier interaction.

A leading German car manufacturer uses AR wearable technology to discover flaws before going to full production. Quality control, assessors take photos and video components with their smart glasses, replacing less accurate handwritten notes. They finalize reports by voice at the vehicle, instead of walking over to input data at computer terminals. This saves time, given that many cars require between 10 and 25 tests.

Discovering the past as it was through your phone

Moving inside, digital creativity is revitalizing museums and making exhibitions more engaging. For example, 3D scanning transforms artefacts back to their original state, by mending them virtually, adding colour and depth, and overlaying useful facts and information. Museums use the Sketchfab platform to publish 3D and VR content anywhere online. Once the partial artefact has been scanned and converted into a 3D model, it is then virtually reconstructed for viewing as the whole, original piece.

Obstacles to mainstream use

While many industries have embraced AR and seen great improvements in their products, services and operational processes, there are still some important hurdles in the way of mainstream use.

Technical glitches: for applications which need to track location, GPS is only accurate to within about eight metres and doesn’t work well indoors. There are also the usual issues for the devices running the apps, such as Wi-Fi connectivity and battery life.

Users of wearables such as smart glasses may experience information overload when they switch between the data on a car windscreen and the actual road ahead. Before this kind or technology becomes mainstream, much more testing will be required.

Privacy: having quick access to all this information while on the go is one thing, but imagine being able to see another user’s Facebook, Twitter, Amazon or LinkedIn profiles by pointing the phone at that person. This will soon be possible using image recognition software and AR applications. Online data security and protection are not new topics, but this kind of instant accessibility needs to be considered carefully from legal and ethical viewpoints before building it into devices. ISO/IEC Joint Technical Committee (JTC) 1 produces International Standards for the security of information technology. Additionally, the IEC Advisory Committee on information security and data privacy (ACSEC) deals with information security and data privacy matters, coordinating activities related to these topics and providing any TC with guidance for the implementation of information security and data privacy.

Following the tech trends

As different industries increasingly incorporate AR and VR applications in their work processes, IEC will follow these developments and continue its standardization activities and quality assurance, which contribute to the performance, reliability, safety and interoperability of this technology.

Gallery
VR stadium at home VR live streaming brings the excitement of the game into your living room (Photo: Livelike VR)
VR medical Medical training application in the CAVE2™ Hybrid Reality System by the Electronic Visualization Laboratory (EVL), University of Illinois, Chicago. CAVE2 is a trademark of the University of Illinois Board of Trustees (Photo: EVL)
VR for engineering Using virtual reality to develop new vehicle models (Photo: BMW)