Augmented Reality for Sailors

One morning this past November, as I motored out of a marina in Pompano Beach, Florida, with Jim McGowan, Raymarine’s Americas marketing manager, something radically different jumped out at me from one of the boat’s three Raymarine Axiom XL 16 chart plotters. We were headed up the Intracoastal Waterway towards the open waters off of Hillsboro Inlet, when one of the plotters displayed a real-time video feed that was overlaid with augmented-reality tags hovering above aids to navigation and AIS-equipped vessels. The tags came compliments of Raymarine’s ClearCruise Augmented Reality and were extremely handy given the rising sun’s near-blinding angle. But rather than squinting at the horizon for buoys, channel markers or other boats, McGowan and I used the ClearCruise video feed to ­recognize aids to navigation and other marine traffic, which we easily identified by looking at—and tapping—the objects’ color-coded AR tags.

Sailors these days regularly encounter far more traffic than the Ancient Mariner could have possibly imagined. Fortunately, though, unlike ­seamen of old we have an assortment of tools—literally at our fingertips—to help us ­navigate perils at sea. ClearCruise collects data generated by other onboard electronic devices in a way that lets sailors quickly access critical information in real time.

Let’s start with the Global Navigation Satellite Systems. GNSS, as it’s called, includes the United States’ GPS satellites; Russia’s Globalnaya Navigazionnaya Sputnikovaya Sistema, or GLONASS; and will soon be joined by China’s BeiDou and the European Union’s Galileo satellite constellations. Combined, these networks help us pinpoint our location, altitude, speed and heading anywhere on the planet. Add to that the Automatic Identification System (AIS), which autonomously broadcasts vessel identity, status, position and other information to other similarly equipped craft (or to AIS repeaters and relay stations), and the helm stations of most modern boats are endowed with a staggering amount of situational awareness.

Raymarine, meanwhile, has taken things a step further by introducing a new and visually rich way to display all that information that is collected in order to better engage with our nautical surroundings. Here, then, is a look at the latest developments within AIS and marine-focused augmented reality, and how they can make you a safer cruiser.

System Components

Class A AIS became ­mandatory aboard seagoing ships in 2002 to help avert ship-to-ship collisions. In 2006, a lower-cost version of this collision-avoidance standard was published and referred to as Class B AIS. Today there are two Class B transponder variants: Class B/SO, which is almost identical to Class A but broadcasts at lower power and a reduced transmission rate, and Class B/CS, which broadcasts data once every 30 seconds and takes a backseat to other types of AIS transmissions. This means that Class A AIS traffic can usurp B/CS transmissions; say, if a tanker and a C&C 37 are both competing for the same AIS transmission slot, the Class A user wins. This is why B/CS can sometimes go entire minutes without reporting, especially in crowded waters. Not so with B/SO. Also Class A traffic can sometimes filter out B/CS traffic.

Recreational sailors can also purchase “listen-only” AIS receivers that will recognize all three classes of transmissions, but does not broadcast data about the vessel on which it’s mounted.

Irrespective of class, the basic technology is the same. An onboard transponder uses GPS data to determine and broadcast a vessel’s course, speed, and position information. AIS data also includes a vessel’s unique, nine-digit Maritime Mobile Service Identity (MMSI) number, which enables vessel-to-vessel communications via VHF radio channels. All nearby AIS-equipped vessels, aircraft and base stations can receive these signals. AIS signals are displayed in a number of ways, depending on the electronics aboard, but commonly, AIS icons appear as an overlay on a vessel’s chart plotter. Additionally, in U.S. waters, the United States Coast Guard receives and records AIS communications from over 130 nationwide AIS base stations (NAIS).

While AIS offers powerful communication and collision-avoidance capabilities, the system can deliver more. For example, the Coast Guard can use its NAIS stations to mark dangerous waters using electronic aids to navigation (eAtoN) broadcasts, which augment existing charted buoys or can be used to establish electronic markers where a physical aid to navigation doesn’t currently exist. The Coast Guard now commonly uses eAtoNs to highlight areas that are difficult to mark using a physical buoy or marker. Or they might predeploy eAtoNs to facilitate reopening ports after hurricanes.

“This has been very ­successful the last three hurricane seasons,” says Jorge Arroyo, a U.S. Coast Guard program analyst who has long been instrumental in developing AIS regulations and standards. “We can program up to 45 eAtoNs per NAIS station and currently broadcast some 430 eAtoNs every three minutes or less from them. Prior to a forecasted hurricane, we use this capacity to augment ­existing physical aids to navigation, and to mitigate their possible loss or movement during the hurricane.”

For example, during the 2017 hurricane season, the Coast Guard established 13 eAtoNs around Port Aransas, Texas, following Hurricane Harvey. This allowed the port to restore limited operations six days faster than if the Coast Guard had to replace the impacted physical navigation aids. Then, weeks later and leveraging lessons learned, the Coast Guard preemptively created 287 eAtoNs for waterways that were forecasted to be affected by Hurricane Irma’s predicted track. These were created at a Coast Guard office, then transferred to the affected NAIS and activated as needed. Critically, preloading these eAtoNs at the NAIS level allowed these potentially property-saving broadcasts to continue after storm-damaged radio towers lost network connectivity.

AIS can also be used to broadcast application-specific messages, or ASMs, which are read and displayed by software. “It allows for AIS to expand in the future,” Arroyo says. “It will be our way of providing notice-to-mariner information digitally to the ­vessel. We’re currently testing this capacity by broadcasting whale exclusion zones and ice edges,” he says, adding that AIS also started broadcasting National Oceanic and Atmospheric Administration (NOAA) weather information at the Army Corps of Engineers locks and by the Marine Exchange of Alaska.

Combining the Data

AIS and GPS are both critical elements of Raymarine’s new ClearCruise AR system, which I test-drove off of Pompano Beach. This video-based system uses real-time video shot by an onboard Raymarine camera, and then applies augmented-­reality tags to help crews with their watchkeeping duties. The tags deliver critical AIS information and data pulled from vector cartography.

The system requires at least one Raymarine Axiom, Axiom Pro or Axiom XL chart plotter that’s running Raymarine’s Lighthouse 3.7 or higher operating system. “ClearCruise AR is best suited to big-screen displays,” says Raymarine’s McGowan, suggesting that cruisers run a single large Axiom plotter in split-screen mode with the ClearCruise AR video feed displayed on one half of the display and a chart with AIS overlay on the other.

ClearCruise AR also requires a Raymarine AR200 IP Camera Stabilization Module that’s networked to up to 10 of the company’s CAM210IP or CAM220IP HD marine cameras, plus the boat’s networked AIS and GPS feeds. A basic system with the necessary components starts at just under $1,900.

Either camera delivers high-definition imagery and features infrared LED lights for night operations. The CAM210IP delivers a horizontal field of view of 33 degrees, while CAM220IP delivers a 93-degree field of view. The AR200 stabilizer features an embedded GNSS receiver and a nine-axis, solid-state attitude heading reference system. The system corrects for pitch, roll and yaw so that the networked chart plotter always displays a stable video feed, level horizon and properly aligned AR tags, irrespective of the sea state or the boat’s heel angle.

The Axiom chart plotter is compatible with charts from C-Map, Navionics and Raymarine’s own Lighthouse cartography. It also will display a real-time video feed, a graphical azimuth banner ring and overlaid AIS data, as well as color-coded AR tags, which the system places atop identifiable objects.

Because the system knows which direction each camera is pointing and the width of its field of view, it can access a chart’s supporting database and the boat’s networked AIS and GPS data to identify navigation aids, user-created waypoints and data from AIS-equipped vessels.

Vessels or objects that the system identifies as hazards receive red AR tags; ­nonthreatening targets get green tags. The system ranks targets based on their physical proximity, placing the tags representing nearby objects higher on the video feed’s horizon than distant concerns. Vessel tags contain information such as name, distance and direction of travel. Users tap these tags to access higher-level information such as course and speed over ground, closest point of approach, and the boat’s MMSI number.

“The AR tags fall right over the buoys,” McGowan says. “It’s the same with AIS targets. It’s easy to lose track of who is coming and going in crowded waters, but with ClearCruise AR, it’s instant recognition.”

While ClearCruise AR works aboard all boats, McGowan, a longtime sailor himself, explains the technology works especially well aboard sailboats.

“Flying sails create big blind spots,” he says. And if the ­camera is mounted aft and its view is blocked by a large headsail, a skipper will still see AR tags in the video feed. The visual imagery of the sail is there as a backdrop, but the AR200 is still doing its ­calculations based on the chart and GNSS and AIS data. Dense fog also may register as a wall of gray on the video display, but it will be broken by the system’s color-coded AR tags.

One obvious drawback is that the system can’t tag non-AIS transponder-equipped boats, including ones with a receive-only AIS device. Also, while ClearCruise AR can display eAtoN broadcasts, the system’s cameras could struggle to properly identify and tag a buoy or other marker that has dragged significantly from its charted location.

Finally, a cautionary note: While ClearCruise AR is impressive technology, it doesn’t absolve crews of their watchkeeping responsibilities, nor can it control a vessel’s steerage or throttle to avoid catastrophe. But, unlike radars, ClearCruise AR uses video imagery, which could be easier to read and comprehend at 0230 hours.

“I know there’s a tendency to go heads-down with navigation but the base view is the same as looking out the window,” McGowan says. “You always need to keep your head on a swivel, but I think AR will help.”

David Schmidt is _CW‘s electronics editor_.

AIS Concerns

Alongside the ­advancement in AIS capabilities come a couple areas of concern, says U.S. Coast Guard analyst Jorge Arroyo. The first involves users failing to update their navigation status or turn off their AIS when moored, which can cause network congestion.

The second is the purchase and use of devices that are not AIS-type certified and are thus illegal to operate on AIS frequencies. Some of these devices are causing unwarranted confusion, such as fishing-net markers purchased on the internet. These devices broadcast indiscriminately and appear as vessels to other AIS-equipped boats. Anyone buying electronic broadcast equipment such as fishing-gear markers should check to see they are in compliance with regulations.