The blustery waters off Massachusetts are prime real estate for wind farms key to meeting US climate targets. That area of the ocean is also a climate refuge for an iconic marine mammal teetering on the brink of extinction, the critically endangered North Atlantic right whale. As an offshore wind farm boom gets underway along the East Coast, new artificial intelligence and computer vision technology could help protect the whales from a jump in ship traffic.

Fatal collisions have decimated the right whale population in recent years as climate change alters their migration patterns. Scientists estimate that only about 340 individuals survive, with fewer than 100 breeding females left. That has led the federal government to impose extensive requirements on developers to ensure wind projects don’t contribute to the whales’ demise.

To that end, Vineyard Wind, which is building the nation’s first large-scale commercial offshore wind farm, is testing an AI computer vision system developed by Charles River Analytics on vessels ferrying materials to a construction site 15 miles south of Martha’s Vineyard. The 62 General Electric wind turbines set to begin rising from the ocean this summer will generate enough electricity to power more than 400,000 homes when the project is completed in 2024.  

The whale-detection technology, called Awarion, is one of several AI-enhanced systems that have been developed in recent years to identify whales and alert ship captains to slow down to avoid striking the animals, which scientists have discovered play a role in combatting climate change. (Conservative commentators and others have tried to link the recent death of humpback whales in New Jersey and New York to survey work for offshore wind farms, though federal officials say there is no evidence of a connection.)

“The whole goal is to integrate these technologies for future wind projects,” says Elizabeth Marsjanik, manager of environmental affairs at Vineyard Wind, a joint venture between Copenhagen Infrastructure Partners and Iberdrola, S.A. subsidiary Avangrid.

She notes that Vineyard Wind already uses human observers, aerial surveys and acoustical monitoring to detect marine mammals, but Awarion will help fill in gaps in that surveillance. “This AI software works with infrared video cameras and is able to detect small changes in temperature and that make it possible to detect whales during low-visibility conditions,” she says.

The federal government has determined that planned wind farms off Massachusetts would have a “moderate” impact on marine mammals such as the right whale. At the peak of construction, those projects would result in as many as 230 additional vessels plying already crowded waters.

Cambridge, Massachusetts-based Charles River Analytics develops technology for US government agencies and Awarion sprang from its work detecting ships and other potential obstacles for autonomous Navy vessels. That included marine mammals.

Soon the company began to receive inquiries from wind farm developers. “We realized that we've got a potentially valuable solution; if we can detect and classify ships and figure out where they are, we can do a lot of the same stuff with whales,” says Ross Eaton, the company’s principal scientist and director of marine systems.

Awarion cameras that can sense visible and infrared light are mounted on a ship and perform 360-degree sweeps of the surrounding ocean. When a camera spots an object on the surface, an algorithm determines whether it’s a vessel, fishing gear, birds or a marine mammal. Whales are particularly difficult to detect from a distance, says Eaton. An adult right whale can be as long as 52 feet and weigh 70 tons but Awarion must be trained to recognize the sliver of a whale’s body that’s visible when it surfaces.

“The challenge is in finding representative data that shows the whales in extremely unexciting circumstances,” he says.

There’s no shortage of dramatic shots of whales leaping out of the water taken by tourists, but few people post photos of a whale’s back. So Eaton’s team went out on whale-watching tours to shoot video of whales surfacing to build a visual library to train Awarion’s algorithms. The system can pinpoint a whale or spray from its blowhole from more than a mile away. When it does, it maps the animal’s location and sends out an alert.

Awarion is still being tested and refined but Eaton says the system, which had a whale detection rate of 80% during trials, should be ready for commercial launch this year or next.

While the technology cannot identify individual whale species, it may be well suited for right whales, according to Matus Hodul, a PhD student at the University of Ottawa who helped lead a recent study that was able to identify individual right whales from space. He’s now working on an AI-powered program trained to recognize the whales from high-resolution satellite images to determine their location. Satellite detection of right whales allows researchers to assess a far larger area of the ocean than can be covered by expensive aircraft surveys.

“Right whales skim along the ocean to catch their food and that creates a good opportunity for us to image them because they're pretty reliably spending a lot of time on the surface,” says Hodul.

Locating right whales has become increasingly crucial for the animal’s survival, according to scientists. The federal government’s leasing of hundreds of thousands of acres of the ocean for wind farms south of Martha’s Vineyard coincided with right whales’ appearance in the area as a rapidly warming ocean has forced the marine mammals to pursue migrating prey.

“We need these new detection technologies to understand what parts of that ocean are most important to the whales,” says Douglas McCauley, director of the Benioff Ocean Science Laboratory at the University of California at Santa Barbara. “That's not a static answer, because for the poor whales every year climate change is throwing curveballs at them so they're adapting their historic migration paths.”

McCauley’s lab collaborated with the Woods Hole Oceanographic Institution and other organizations to develop Whale Safe, an initiative that deploys underwater microphones called hydrophones off the California coast to listen for blue whale vocalizations near busy shipping lanes. Once whale sounds are detected, an AI program compares it to a catalog of known whale songs and identifies the species so container ship captains can be notified to slow down.

On the East Coast, Woods Hole senior scientist Mark Baumgartner has dispatched underwater robots called gliders to listen for right whales as they traverse the Gulf of Maine.

Each whale detection technology has its blind spots. Awarion’s cameras can’t see below the surface and bad weather can interfere with their ability to scan the ocean. The same is true for satellite detection of right whales. Whale Safe, on the other hand, can’t find whales underwater when they’re silent.

“An integrated system that's drawing on the strengths of each individual method for detecting a right whale is where we really need to be headed because we don't want to miss even one whale,” says Jessica Redfern, associate vice president of ocean conservation science at the New England Aquarium.