5 ways to use AR for building a digital shipyard

Augmented reality (AR), the industrial internet of things (IIoT) and other digital technologies can significantly improve the efficiency and effectiveness of a shipyard.

A vast number of devices can leverage the power of AR. Headsets provide for a richer immersion experience and come with the benefit of being relatively hands-free. They are controlled with gestures, where you move and wave your hands to mimic physical actions such as clicking and swiping. Many headsets also can be controlled via eyes and voice. But organizations can also use tablets or other mobile devices to leverage AR. More often than not, deciding which to use comes down to nonfunctional requirements such as cost, durability, reliability, maintainability, security, performance and interoperability. Regardless, any type of device can yield immediate benefits.

Safety first

While it makes sense to focus on using digital technologies to sharpen your competitive edge, improve efficiencies and increase profits, I am often disappointed that organizations overlook the ways these technologies can be used to prevent employee injuries or, heaven forbid, fatalities.

There are many reasons for workplace injuries, but one that is quite prevalent is worker fatigue. Too few breaks, too many hours on the job in a 24-hour period, irregular or overnight shifts, even working in hot or confined spaces — all can contribute to worker fatigue. Although there are a number of tools that can help improve safety, AR and IIoT are paving the way for more effective prevention.

Picture an AR headset that can assess key information about a worker, such as the time he or she last took a break, the amount of time worked during a current shift and the number of hours worked in the last seven days. The headset could also assess when the employee last worked an overnight shift or in a hazardous or difficult environment. When a safety officer puts on the AR-enabled headset and simply looks at a worker, the headset’s screen will display a fatigue index that indicates the worker’s general alertness. If the worker is below a predetermined threshold, a warning icon could flash red. This might result in the worker being sent home or directed to take a break.

There are several ways to make this capability a reality. One is to use a back-end system linked to an AR-enabled headset that captures workers’ information such as time on shifts, breaks, roles, work environments and other data. Much of the information might come from a human resources management system (HRMS).

There also are different ways to relay that information to a safety officer’s headset so a worker’s alertness can be conveyed. One way would be to affix a 2D bar code onto a worker’s clothing or safety gear; the AR headset could scan that bar code, retrieve the information associated with that bar code from the database, and then display the value on the headset’s screen. An alternative method would be to use an IIoT device incorporated into the worker’s clothing or gear and let it inform the AR headset of the details.

Quality check

Just before a ship is delivered to a customer, it has to go through a rigorous and manually intensive quality inspection. Typically, engineers inspect every compartment of the ship using pens, notepads and cameras to capture the information. They then download all that information into the product life-cycle management (PLM) system when they get back to their desks. That double handling is hugely inefficient. AR can remove that requirement by doing it in situ.

Mobile devices such as phones, tablets and AR headsets have enough onboard processing power to collect, store and manipulate data, so there’s no need to send the data to a back-end server for processing. When the engineer has collected, reviewed and acted on all the required information, the processed data can be uploaded to a server as soon as a network connection is established.

Figure 1. AR view of a ship compartment quality checklist through the eyes (and headset or tablet) of a quality officer

For example, rather than manually work through an inspection checklist in each compartment of a ship to capture relevant evidence with a handheld camera, a quality officer could wear an AR headset that provides a relatively lightweight, nearly hands-free approach to capturing that information. The headset could scan a bar code or IIoT tag and retrieve any associated media from a server or application such as PLM, or even from resources preloaded onto the headset. The headset also can display a list of items that must be checked, and the officer can gesture to mark off items once they pass inspection. AR headsets and AR-enabled tablets can be equipped with cameras so the officer can easily add video or photos to the inspection record.

Proactive maintenance and support

One of the most compelling uses for AR, especially when connected to an IIoT mesh network, is for in-service support and maintenance. Consider operational maintenance, something that a ship requires constantly.

Traditionally, a maintenance technician either waits for something to break or relies on lead indicator gauges to send alerts. Unfortunately, both of those situations require a lot of manual interaction.

Ideally, equipment is proactively maintained, and that is where the AR/IIoT duo can provide potent capabilities. IIoT devices in a mesh network can proactively communicate with one another and with master nodes incorporated into systems, such as a pump. The master node constantly receives information from sensors also embedded in the pump, measuring temperature, pressure, vibration, tilt, revolutions per minute (RPM) and other conditions. If the master node detects that any of those measures, in isolation or in combination, might cause a failure, it can present that information to the wearer of an AR device as he or she walks through a compartment. Not only might an alert be given, but the AR headset could also display instructions to mitigate the problem.

AR also can be used to augment standard maintenance activities.

As an example, a technician could receive prerecorded videos or even live video feeds of proactive maintenance tasks rather than carrying around a heap of manuals or relying solely on his or her own experience.

If an issue with a part is identified during a maintenance check, the technician can simply order a replacement part for it by using gestures to select an order part icon on the headset’s screen. Once a network connection is established, that order can instantly be sent from the headset to the organization’s enterprise resource planning/manufacturing resource planning (ERP/MRP) system.

Let's add one more feature to make this even more impressive. Suppose the technician has hit a snag with a maintenance activity because a pump has failed due to a cracked sleeve. Rather than ordering a new part, which might affect the ship’s availability, the technician could simply pull up a computer-aided design (CAD) drawing and print the required part to a 3D printer, all orchestrated by the AR device.

Figure 2. AR view of part with exploded diagram and 3D print request being initiated

Digital design

Digital twins (digital reproductions of actual engines or other components) and 3D modeling have given organizations the opportunity to completely transform the way they design a ship. Once a digital twin is created, 3D modeling can be used to simulate day-to-day tasks and provide critical information regarding what will and won’t work with a certain design.

AR can take 3D modeling to the next level. Instead of doing engineering change design and modeling on a PC, it could be done in situ with either an AR-enabled headset or mobile device. Digital objects could be virtually inserted or removed, and represented on the screen of either the headset or the tablet. As the engineer works, precise guidance could also be displayed on the screen about whether the object will fit into that physical landscape, obstruct the line of sight for other activities, interfere with the safe evacuation of crew members in the event of an emergency, or collide with or restrict the movement or use of nearby equipment and machinery.

The digital objects can either be loaded onto the AR device or requested from an object database via a network connection. In some cases, the engineer can pin the digital object in place so it stays there, allowing the engineer to walk around the virtual object to view it from different angles. And if the AR headset has powerful enough onboard capabilities, it can constantly scan the environment to gather spatial information and create a 3D rendering.

Employee training

When it comes to shipyards, new employee training is enormously important. Shipyards are filled with high-voltage cables, heavy moving equipment, flammable liquids, plasma cutters, sharp objects, hot surfaces and even explosive substances. Evacuation routes, muster points, firefighting equipment, emergency phones, first aid kits, and general knowledge of how a site works are all really, really, important.

AR could be used to facilitate a self-paced tour of a shipyard and its associated facilities. The tour could walk a new employee through various scenarios, such as:

• Evacuation routes
• Areas that require personal protective equipment (PPE) and paths for individuals without PPE
• Use of critical equipment for firefighting, first aid and eye rinsing
• Use of special equipment, such as plasma cutters and similar machines, cranes, pipe bending equipment, etc.
• The location of key individuals and functions, such as emergency wardens, security, Human Resources and others
• The location of amenities, such as toilets, cafeteria, gyms, etc.

The advantage comes from AR’s simulation capabilities. Research shows that when people are engaged while learning — and actually enjoying themselves — their attention and therefore the retention of that information is far greater than if they are not. The beauty of an AR training solution is that once you have defined the information, it is reusable. Another advantage is that the solution doesn't have to rely heavily on a network connection, since all the information could be loaded onto the device itself.

Figure 3. AR view of an induction being provided to a new employee, specifying site location and an explanation on the plasma cutting machine in view. IIoT location devices are also specifying that the area requires PPE.

So, there we have it: five ways that significant value can be added by using AR. While AR is a highly complex capability still in its infancy, if you apply it well and keep it simple, it can be immensely powerful.