Blog Main Image - Scientific Collagen Fibers Skin
December 14, 2022

Using AI and VR to Advance Research

Caroline Stefani, PhD, became fascinated with the immune system while pursuing her doctorate in microbiology. She loved using imaging tools to examine the worlds of cells and bacteria. But one thing frustrated her. 

“We generated so much data and had to process it all manually — I’d go home at the end of the day and spend hours entering data with movies on in the background. I knew there had to be a better way,” Dr. Stefani says.

Caroline  Stefani

In 2013, she joined BRI’s Lacy-Hulbert Lab as a postdoc eager to learn about germs and the immune system. She was excited to explore tools that could pull even more information from images. Her desire to automate some of research’s busy work never left her mind.

“While I was on maternity leave, I began training myself on coding software, exploring tools like artificial intelligence (AI) and virtual reality (VR) to pull more data from images,” Dr. Stefani says. “Now, we’re using those tools to gather data from images. It cuts down on the manual work and helps speed up research.” 

In addition to conducting her own research, Dr. Stefani now leads BRI’s Imaging Core, applying the latest imaging tools to studies across BRI. 

Painting cells, zooming in, and exploring in 3D

Dr. Stefani studies cell membranes, which separate the insides of cells from the outside world. She aims to learn how germs infect cells and how cells repair themselves. 

“Cells can’t see, they can’t smell. Yet they respond differently to different invaders,” she says. “Sometimes, when cells are infected with bacteria, they yell and scream and call in more immune cells to fight it off. Other times, they see cancer cells and don’t say a word, allowing tumors to grow. I want to know why.” 

Dr. Stefani uses a technique called cell painting to apply different stains to cells. This enables her to find similarities, like cells that are the same shape or size. It also illuminates patterns, like cells that express the same genes or that are affected by a certain drug. 

“It’s like looking at a big crowd and sorting out who is wearing glasses or who has blond hair,” she says. 

Another tool called expansion microscopy lets her zoom way in on cells. This can make them up to 40 times their original size. 

Finally, she uses a tool called Confocal VR, which she developed with Tom Skillman (BRI’s former head of research technology). This tool creates 3D images of those zoomed-in cells and allows you to explore them in VR. 

“A flat image can tell us that a toxin entered a cell. The tool that Tom and I developed allows us to follow that toxin inside the cell and see what it’s doing,” she says. 

Her team has already discovered a new gene that helps cells repair themselves. She hopes to study this gene in cancer, to see if it could explain why cancer sometimes comes back after treatment. 

“The gene we found tells the cell ‘repair, repair, repair’ when it’s infected with a toxin,” Dr. Stefani says. “But if it says ‘repair, repair, repair’ in a cancer cell, does that enable the cancer to keep coming back? We’re putting together a grant application to find out.”

Using AI for faster, more accurate results

Antinuclear antibody (ANA) tests show whether a person has immune cells that attack healthy tissue. These tests can help diagnose diseases like lupus and multiple sclerosis. They are often used in research to find which immune cells are attacking healthy tissue. But each test needs to be read individually. And if a researcher collects hundreds of tests, it can take days to interpret every single one.

Graphic 2D Data ANA Staining

Dr. Stefani’s team is building AI software that could save scientists tons of time. 

“Our software works like Google Photos, which can recognize people’s faces, know a cat is a cat and a dog is a dog,” says Dr. Stefani. “We teach the computer to recognize patterns across lots of images, then sort by shape, size and other features.” 

Current ANA tests only allow for one color stain, which shows just one type of immune cell attack. The platform will allow researchers to use different colors to identify different proteins, enabling scientists to extract more data from one sample. Dr. Stefani also plans to build this technology into an app that allows you to look for patterns across different samples, examining whether things like age, gender or race have an impact on disease.

Featured 2D Data Spatial Transcriptomics Dual-Pane

Using spatial transcriptomics to get more data

Spatial transcriptomics helps researchers collect vast amounts of information, like where cells are located in tissue, which genes are expressed inside of cells, and how different cells interact. Dr. Stefani and Oliver Harrison, DPhil, recently implemented this leading-edge tool at BRI. As a test case, they examined a structure in the gut called a Peyer’s patch. 

“Spatial transcriptomics was really useful because we could gather information broadly and test a specific hypothesis,” Dr. Harrison says. “We were able to pull a huge amount of information out of the tissue and look for patterns. We were also able to learn more about an interesting cell receptor that we had identified in other studies.”

This technology holds great potential for new insights across BRI. 

“Rather than being alone at the microscope, looking at a flat image, we’re working with people across BRI to create a large data set,” Dr. Stefani says. “Together, we’re creating millions of data points and finding patterns among them — like which cells might lead to cancer and which ones can protect us from disease. This was just the first test case, but we’re really excited to continue to use this tool and see what we find.”

Immuno-what? Hear the latest from BRI

Keep up to date on our latest research, new clinical trials and exciting publications.