The complexities of communicating with machines

An update on the work of ISO/IEC JTC 1/SC 35 on user interfaces

By Natalie Mouyal

Our lives are surrounded by machines for tasks ranging from the mundane to the complex. We rely on them to help us with our daily chores and to make better choices. They allow us to communicate with friends and colleagues, source new information and entertain ourselves.

Group photo from the most recent ISO/IEC JTC 1/SC 35 meeting Participants at the most recent ISO/IEC JTC 1/SC 35 meeting

But how do we communicate with machines? In the field of computer technology, human to machine interfaces have progressed from the use of punch cards to onscreen textual commands and graphical user interfaces. Recent developments have incorporated touch screens and voice commands commonly used with digital assistants, tablets and smart phones. Human auditory, visual, and tactile senses are being exploited to interact with our digital environment.

As these technologies become more widespread, however, all members of society should be able to participate. Consideration must be given to ensure that access to IT devices is equitable and inclusive, including for those with special needs. This not only helps to ensure wider product adoption but also facilitates access to societal and business opportunities for everyone.

More than 20 years ago, the joint IEC and ISO technical committee for ICT (ISO/IEC JTC 1) established a subcommittee that provides standardization in the field of user system interfaces (ISO/IEC JTC 1/SC 35). It seeks to provide standardized and user-friendly interfaces for all users, including those with accessibility issues or with special needs, and ensure that the interfaces can be adaptable across cultural and linguistic barriers.

e-tech recently spoke with the Chair of ISO/IEC JTC 1/SC 35, Khalid Choukri, to learn more about the subcommittee’s standardization work.

Tell us about the work of ISO/IEC JTC 1/SC 35

The SC 35 committee focuses on standardization of user system interfaces in the ICT environment.

We would like to make sure that people can control or navigate their devices using a visual, auditory or tactile modality. For example, we can use our voices, body movements or gestures similar to those we use with our smartphones as interfaces in a standardized way. What we would like to do is ensure that such interfaces serve all users, including the ones with specific needs.

In addition, our work specifically addresses culture adjustability and linguistic compliance. This is one of our main objectives. We try to go beyond the basic legal obligations with our recommendations on developing user interfaces.

Keyboards are an example of an ICT interface that is commonly used. What has the subcommittee standardized in this area?

The initial standards that we published addressed keyboards using different types of script. One of the tasks was to design the layout so that people could use different scripts in a multilingual environment.

However, the problem remains on how you switch between languages. For example, if you are working on your keyboard in English, how can you easily switch to a French language keyboard? This is something that we are working on now. We are currently in the process of updating the ISO/IEC 9995 series to address new national keyboard requirements as well as develop new sections to complement and ease multi-language keyboard use worldwide.

Haptics uses technology to stimulate the senses of touch and motion. Are these areas of interest for your group?

We are working on standardizing tactile keyboards and I think that we will definitely be moving to other haptic modalities with the expected growth in virtual and augmented realities. We need to ask how interfaces can be seen or felt with VR and AR using gloves or other wearable devices. For VR keyboards, we could imagine swiping or actual touch. A new project has been set up that will look into haptic interfaces for wearable devices.

We are all familiar with tactile or touch screen through our use of smartphones and tablets. But we could think about extending this to other devices such as cameras that could, for example, identify gestures or a keyboard that is a type of hologram. In 2015, we published ISO/IEC 30113-1, the first in a series of standards for gesture-based interfaces.

We are also considering keyboards with characters that could be used as an interface for sign language. For example, we could consider having a machine that generates avatars for sign language that is then converted into speech or translated into another language.

Increasingly, users interact with their computers through voice commands. What standardization work is underway to convert text into audio and vice-versa?

We have worked since 2015 on the possibility to convert text into audio information but also audio into text like we see with subtitles and captioning. We would like to have the audio descriptions standardized with some explicit requirements. Our corresponding guidelines, ISO/IEC TS 20071-21, are under review to account for recent developments.

In terms of text to speech, we need to address speech detection. We have an initial standard for voice command which we developed several years ago but now we would like to work on a more sophisticated voice recognition project that can consider human and machine interaction using continuous speech.

One of our working groups is looking at speech interaction using full duplex which means that the speaker can speak at the same time as the machine. There is no interruption needed to, for example, push a button after asking the machine a question, which can be unnatural.

Accessibility is an important topic. Since 2014, SC 35 has represented JTC 1 accessibility in a number of standardization organizations. What has been accomplished and are there new projects planned?

SC 35 has published a number of standards related to accessibility. We have recently published ISO/IEC 29138-1 on user accessibility needs and ISO/IEC 30071-1 which provides guidelines for developing accessible ICT products and services. We have adopted a holistic approach and would like to make sure that organizations developing these products and services are aware of the requirements surrounding accessibility.

Moving forward, we need to make content easy to read and easy to understand. Today, if you read text, your understanding will depend on factors such as your background, level of education, your language skill, but also your visual capacities. Content may be complicated because it uses specific jargon or difficult to read because the characters are small, or the colour contrasts selected.

We now need to consider how we can help with these issues. For example, if I am reading a text, characters should be at a suitable size, a dictionary should be easy to access so that words can be defined or translated when I scroll over them. At a next stage, such possibilities should also be available when we convert text to audio.

Much attention has been given to the UN Sustainable Development Goals (SDG) which seek to improve the world by 2030. How does your work fit into these goals?

SC 35 contributes with several standards to the UN SDGs, in particular for issues related to the following: Quality education, Decent work and economic growth, Industry, innovation and infrastructure and Reduced inequalities.

Our work is important for access to education. We are also focusing on users with special needs such as the elderly or those who work in multilingual contexts, while carefully monitoring the needs of industry for standards.

Gallery
Group photo from the most recent ISO/IEC JTC 1/SC 35 meeting Participants at the most recent ISO/IEC JTC 1/SC 35 meeting