A new AI tool can guess what you’re looking at just by analyzing your brain scans—and recreate that image with remarkable precision. It can go the other way too, and predict a person’s brain activity based on what they’re looking at.
In the image above, for example, the left-hand image of each pair is what the user actually saw—and its right-hand counterpart is what the model recreated based on the brain scan.
Michal Irani, who developed the tool with her colleagues at the Weizmann Institute of Science in Rehovot, Israel, hopes her “mindreading” tool will ultimately reveal more about how the brain works, and could perhaps be used to help locked-in people communicate, or allow scientists to recreate the content of dreams.
Judy Illes, a neuroethicist and professor of neurology at the University of British Columbia in Canada, who was not involved in the research, describes the work as “magnificent.” “The idea [of using this approach] to help people with neurologic conditions … therapeutically is tremendously exciting,” she says.
But other scientists warn that a similar approach could be used to reveal the inner thoughts and mental imagery of people, potentially without their consent. “The results seem very impressive,” says Tommy Sprague, a neuroscientist at the University of California Santa Barbara.
“But if there's a way to surreptitiously extract information about what you're thinking about, then…150 years of sci-fi can come true anytime, and that’s worrisome in a lot of ways.”
Peeking into the brain
Neuroscientists have been working on ways to reconstruct what people see—and what’s going on in their minds—for years. The first attempts produced images that were blurry and hard to make sense of. Advances in technology—both in the fMRI scans themselves and the tools used to make sense of the results—have led to improvements over the years.
Irani and her colleagues started by analyzing publicly available brain scan data. Other researchers had already collected scans from volunteers who were shown hundreds of images while they lay in fMRI scanners.
fMRI uses a giant magnet to track the flow of oxygenated blood through the brain. Brain areas that “light up” on fMRI scans are thought to be those that are particularly active at any given moment. They’re not especially specific—in typical fMRI scanners, each highlighted “voxel” of activity covers around three cubic millimiters, containing around 16,000 neurons.
But Irani and her colleagues used newer datasets collected using scanners with a higher resolution—with each voxel covering around one cubic millimeter of neurons, she says. Those datasets showed what the brain activity of volunteers looked like when they viewed various images.
Other teams have done this, too, and several other tools have been used to recreate images based on brain scan data. But they’re not good enough, says Irani. Say a person saw a banana. These models can generate an image of a banana, but it would look different, she says. “It wouldn’t have the same structure, the same position.”
A better decoder
The team wanted to more closely recreate the images that had been seen. The first step was to train an AI model on already available data from eight people who each had been shown around 9,000 images while in a high-resolution fMRI scanner.
Crucially, their “brain decoder” has two branches—one to predict the structure of an image (where the colors are, for instance) and a second to predict its content (for example, a bunch of bananas on a plate). The predictions allow a diffusion model, a type of AI best known for creating video and images by gradually cleaning up a noisy mess of pixels, to produce a much more accurate representation of what the person saw.
But to improve the models they needed more data—far more than was actually available.
To get around this problem, she and her colleagues trained another model in the other direction—an encoder that can predict brain activity from an image. The team then used the encoder and decoder together to improve both tools.
It works like this: start with a new image, say, of a leopard. Then use the encoder to predict what the fMRI brain scan of a person would look like when they saw that picture. The decoder is then used to reconstruct the image again. At first, that image probably won’t look much like a leopard, says Irani. But repeatedly training the models this way eventually leads to dramatic improvements.
This approach also allows the team to train their models on as many images as they want, even though they might never have been shown to a person in an fMRI scanner. Irani says that around 70% of the training data is from images that were not originally paired with fMRI scans.
By combining data from multiple studies, they were also able to identify brain regions that seem to share functions across all individuals. One region seemed to respond to images of food, for example, while another responded to images of sports. Irani, a computer scientist, says she is now working with neuroscientists “to see if we can actually use these tools that we've developed to really find out new things about the brain.”
The resulting “universal brain encoder” can work on a scan from a new person with minimal calibration. In other attempts, a tool typically requires about 40 hours of fMRI data on a new person before it can be used to predict what they’re seeing. Irani’s decoder only needs one hour of data, she says. The finding was presented at the Cognitive Computational Neuroscience conference in New York last month.
That could make it valuable for neuroscientists studying the brain, says Sprague. “None of us can afford 40 hours of imaging for a new subject,” he says. “It's something like $600 to $1000 an hour.” Tools like this one could speed up research, he says.
State of the art
The encoder and decoder aren’t perfect. “Of course we have failures,” says Irani. Over a Zoom call, she pointed out an image of a cake that her tool reconstructed as a pile of three sandwiches, and another of a dog in a bathtub that was reconstructed as a similarly-colored goat in a bathtub.
But they represent the state of the art. In a comparison test, the tool was found to be much better than previously described ones. “All in all, really we outperformed the others by a significant margin,” Irani says. “Mindreading” is a “cute, jazzy name” for what they’re doing, she adds.
Irani is now planning to move beyond images, and onto video and audio. She wants to be able to reconstruct what people are thinking about or imagining, and the contents of their dreams. “That’s something we don’t have yet,” she says. “But we’re striving to achieve it.”
Such a tool might also enable people who are “locked-in” and completely paralyzed to communicate using their brain activity alone, she says. It could also help scientists unpick some enduring mysteries surrounding the inner workings of our minds, such as what PTSD flashbacks look like.
Advances like this inevitably raise questions about mental privacy. What if some bad actor could recreate a person’s mental image, replaying their thoughts or something they’ve seen?
“If you’d asked me that 10 years ago, I’d have laughed a lot,” says Sprague. Getting a person to lie still in a scanner and actively engage with a research question is hard enough, let alone doing so against their will. But Irani and other scientists are working on similar approaches to decode brain activity from EEG—electrical brain activity measures collected via a cap of electrodes or even through headphones.
And as models improve, it will become even easier to analyze the brain activity collected this way. “We have to be a little more serious about the ethical considerations,” says Sprague. He thinks Irani’s approach would probably “work quite well” in predicting images that a person is thinking about but not looking at.
The move to EEG would be a “gamechanger,” says Marcello Ienca, a neuroscientist and philosopher at the Technical University of Munich, Germany. Once an EEG device has been calibrated to a user’s own brain, it could be relatively easy for companies to extract additional information from that person’s brain—potentially without their consent. Ienca can also imagine some courts allowing mental image reconstructions as legal evidence.
“I have no doubt that this is, you know, well-intentioned research, but I think it's also pretty obvious that it could be co-opted for … ethically and societally problematic commercial uses,” he says.
Irani acknowledges the potential for misuse with the use of EEG. But she’s not concerned for now. “I’m trying to think only of good things,” she says.
Deep Dive
Biotechnology and health
A startup claims it’s found a drug to make your blood young
Generation Lab claims its drug combo can “stop the spread of aging” around the body. And it’s looking for influencers to give it a try.
ByThis geneticist’s age-reversal tech could help restore sight
Yuancheng (Ryan) Lu is behind one of the buzziest results in rejuvenation science.
ByMeet a mouse whose brain cortex is made up of human cells
“Xenocortical mice” are a dramatic demonstration of organoid technologies
ByScientists just created female clones of male mice
The sex reversal technique could be used to help rescue endangered species, say the researchers behind the work.
ByStay connected
Illustration by Rose WongGet the latest updates from
MIT Technology Review
Discover special offers, top stories, upcoming events, and more.
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接