Technology is evolving, but human decision-making remains important

For factories and social infrastructure such as energy and transportation where the stoppage of operation is never allowed, the quality of O&M is absolutely vital in maintaining stable performance of device and equipment. The quality of O&M is highly dependent on the capabilities of individuals, and all companies share common challenge in passing the advanced techniques and know-how of veteran operators to next generation workers and in training them. However, the reality is that this passing on of techniques seldom goes as smoothly as planned.

In order to pass on O&M operation capabilities, many companies go forward with standardization of operation processes and decision-making criteria, and create manuals. This aims at leaving enormous and complicated work procedures and the know-how of experienced workers in a form that can be conveyed, and maintaining environments where an equally high level operation quality can be performed by each operator. However, except for simple work, there are limits to create and maintain manuals by hand from scratch. Work contents and work frequency varies depending on the equipment and operations involved, and work environments keep changing due to installation of new equipment or improvement of operation processes, etc. It is quite significant workload to maintain these manuals updated under such circumstance.

In case work procedures and contents are shared with other operators by word of mouth, it is extremely difficult to pass on finer points such as gut instincts and knacks, and extremely difficult to understand and put them into practice. The insights, understanding and decisions by experienced operators are of tremendous use under the abnormal conditions where the causes of malfunctions and problems are not simple and, therefore, difficult to specify. In many of these cases, decisions are made subconsciously based on implicit knowledge rooted in the experts' extensive experience, and expressing this with words or numbers is no easy task.

Advances in digital technologies such as the Internet of Things (IoT) and artificial intelligence (AI) are expected to make it possible to visualize the status of devices and equipment in greater detail. There are hopes that high precision status analysis will make O&M easier. Of course, there are situations in which digital technologies are highly effective, but when the causes of abnormalities cannot be determined even after analyzing data from worksites, there are times when companies must rely on the experience and intuitions of experts, developed through the course of their work in the field. No matter how much technology advances, human decision-making remains essential. To achieve stable operation, in which devices and equipment never go down, an important task that must be tackled is using digital technology to codify the implicit knowledge of experts in text and numbers, in a format that is as easy to understand as possible, so that they can be surely conveyed and shared.

Supporting site operators by using AR-based "digitization of craftsman"

To take on these O&M challenges, we have begun providing a worksite operation digitization solution, "Meister AR Suite," as a new variation to the manufacturing IoT solution, Meister Series. Through AR technology, this innovative solution digitizes the work procedures and decision-making criteria of experts and veterans, and offers easy-to-understand support to site operators – "craftsman digitization," so to speak.

AR is a hot technology which virtually augments the real world in front of us by overlaying virtual information on top of physical environments such as the objects and spaces people see. Toshiba has actively employed AR in various industrial fields, and has carried out research and development, and verification within Toshiba Group. "Meister AR Suite" is the culmination of the extensive expertise and technologies we have developed through these efforts. It is a package that includes "AR Content Generator," which enables anyone to create and edit AR content without requiring specialized knowledge, and the AR applications "AR Manual" and "AR Navigation" which make it possible to handle created AR content easily on tablet devices. It provides support for digitization of craftsman and the resulting realization of efficient site operations (Fig. 1).

Easy to use content generator and applications that can be used anywhere

One of the advantage of AR Content Generator is intuitive operability. It enables anyone to create AR content using text, diagrams, sounds, images, video, and more, without specialized programming knowledge and skills. Creating AR content is as simple as making a slide documents. Another notable feature of the AR Content Generator is that it does not require special markers to display AR content. Once a certain part of equipment or facilities in work locations are registered as an image marker preliminarily, AR information of work contents and procedures will be displayed on the operator's device in which AR application is installed, as the device is held over the registered part as a reference point. This high level of operability makes it easy to create manuals which minutely incorporate the flow of actions and judgment criteria, such as what experts are looking at in worksites and which actions they take in which situations. As AR contents can be modified as desired at the user end, it is simple to add new know-how and knowledge, and the user can promptly reflect the changes in equipment and situations.

The AR Navigation and AR Manual applications utilize the AR content that has been created and edited, to provide supports for actual work performed at sites.

AR Navigation displays operators' destinations and movement directions on a map shown on a tablet device. At worksites where various equipment and facilities are dotted, it enables operators who are still unfamiliar with the site to go straight to the equipment and facilities they need to get to, via safe and optimized routes.

Once they arrive at the destination, it is AR Manual's turn. Operators simply hold their tablet devices over the target equipment they need to work on. Work points and procedures are displayed as AR images, overlaid on the real equipment or facilities, accompanied by easy to understand guidance with voice and video as well. The AR instructions follow the targets shown on screen when the tablet is moved, ensuring the work to be carried out quickly and accurately. Even operators who are unfamiliar with operations can smoothly carry out work appropriately, from a vantage similar to that of an expert. Furthermore, it assists in creating operation reports and accumulating knowledge by recording the operations they carried out with image and text. This information can then be systematically organized and converted into AR content to further improve operation quality.

Video: How AR Manual works on tablet device at worksite (1 min 14 sec)

With AR Navigation, it is also possible to assign multiple checkpoints along the way to a destination. The AR Manual can be configured for each checkpoint so that multiple operations are carried out in a single scenario, helping improve O&M efficiency.

These AR applications can also be used offline. They were designed also to accommodate to work sites where network connectivity is not available, due to security considerations, poor reception or other factors.

In the future, we plan to further optimize the applications for customer usage scenarios, such as supporting wearable devices that enable users to operate with their both hands free. We are also planning to connect them with operation systems, IoT systems, AI-based analysis systems, and the like. Furthermore, we are exploring the potential for applying them in scenes other than worksites, such as in employee training, simulations performed before going onsite, and so on. We will spread the use of AR, which is highly effective in various industrial fields, through co-creation with our customers and partners.

The "Meister AR Suite" resolves the social problems of the shortage and aging of working population through digitization of craftsman. It expands the capabilities of people, which become more and more important as technology advances, through digitizing diversified scenes in factories, social infrastructures and any other various industrial sectors, as well as supporting the safety and security of them.