Imagine there are ten different types of shells, each the size of a grain of sand, mixed in the bowl of mud in front of you. You want to find each type of individual shell —what do you do?

For paleoceanographers studying the history of the oceans, this ridiculous question is a big chunk of their research. And for 60 years, they have looked at the mixture under a microscope and used a paintbrush to sort shells out one by one. But a group of scientists is looking to automate the process and free up the paleoceanographers to spend more of their time analyzing the microscopic shell fossils called foraminifera that can clarify the role of the ocean in climate change.

While visiting his wife at a paleoceanography lab, Ritayan Mitra, a geoscientist at University of Colorado-Boulder, was appalled at how strenuous sorting foraminifera by hand. Because the species interacts with light in different ways, not only was she dusting them up by hand, but also moving the light source on her microscope constantly to pick out the shells. Mitra made a prototype, placing a ring of LEDs around the end of the scope that could be automatically adjusted to provide different angles of light. Eventually he ran into the end of his robotics ability.

“I’m not the robotics guy, neither am I the oceanography guy, I just saw a problem and wanted to find a solution,” he says. To solve the problem he needed both.

An outline of how the identification system could work.
An outline of how the identification system could work.

So he turned to Edgar Lobaton, an electrical engineer at North Carolina State University, where Mitra was at the time. Researchers have tried and failed to automate the process of sorting the tiny shells before. Lobaton rose to the challenge, writing the grants that would enable this multidisciplinary collaboration. Instead of trying to get a computer to identify all the possible images of all the species of foraminifera (which is what people tried with neural nets in the past) his team is going to train the computer to only recognize the half dozen species of foraminifera regularly used in research. (This will reduce the computing load.)

Lobaton also has a secret weapon thanks to Mitra – Tom Marchitto, a paleoceanographer at University of Colorado-Boulder, on the project. When Lobaton and his lab go to visit Marchitto for the first time in early August, they will be given a crash course in paleoceanography. “We’ll look at foraminifera together,” says Marchitto. In the process, he hopes to transfer some of the knowledge of how people make decisions about which shells are what species. From this, Marchitto says: “Hopefully we can transfer those decisions to an artificial intelligence type of network that can do this automatically.”

Currently, paleoceanographers spend a large amount of their time sorting samples. Since the species prefer different nutrients and temperatures, some of their data about the conditions of ancient oceans comes from the proportions of different species found at different times. If the sorting is automated, this data would be collected exponentially faster than it is now. It would also free the researchers to analyze the chemical composition of the fossilized shells. Foraminifera have been found throughout the oceans for millions of years, so their changes in chemistry over time is a window into the past ocean climate.

The first step for the researchers over the next two years is designing the identification software. If that goes well, Lobaton has plans for robotic arms that can actually sort out the species. Teaching a computer to find the microscopic fossil you want, in a pile of microscopic fossils, is an impossible-sounding task. But if they can succeed, we could be looking at a surge in information about the ancient oceans like we’ve never been seen before. And for microscopic fossil shells, that’s pretty cool.

Photos via Edgar Lobaton, Giphy, Tom Marchitto

Dyani Sabin is a science writer from small-town Ohio transplanted to New York City. Former biology researcher and library supervisor, you can also find her writing at Scienceline.