快猫短视频

Can you guess what I鈥檓 looking at? Artificial intelligence can. A new system developed in Japan can describe a picture someone is viewing, using brain scans alone.

Algorithms have recently become pretty good at generating image captions, however they normally get to see the images they are captioning. Now it seems the same techniques can be used to generate captions via scans of a person鈥檚 brain.

鈥淚 consider it a form of mind reading, or perhaps at this point just mind skimming,鈥� says at Radboud University in the Netherlands, who was not involved in the research.

Good and bad

To generate a caption, the AI is given an image of a person鈥檚 brain, taken with an fMRI scanner while the person was looking at an image. The fMRI scanner shows the surges in blood flow that correspond with activity in the brain, so the different parts of the brain that process the image light up on the scan. From this, the AI then produces a caption based on what it thinks the person was viewing. For example, one caption it generated was 鈥淎 dog is sitting on the floor in front of an open door鈥�, which correctly described the scene.

Working out the accuracy of the system is tricky because there is no definitive best caption. However, of the six captions included in the paper, most of them fairly accurately described the original image and made grammatical sense. 鈥淪ome results were quite good, others were bad,鈥� says at Ochanomizu University in Japan.

To train the AI, Kobayashi and his colleagues split the process into two parts. The first identifies features in an image like 鈥渕an鈥�, 鈥渟urfboard鈥�, or 鈥渙cean鈥�, and the second forms captions by putting these together into a basic sentence such as 鈥渁 man is surfing in the ocean on his surfboard鈥�.

Both components of the AI contain a type of algorithm called a neural network, which consists of thousands of different connections, inspired by the way neurons connect in the brain. Neural networks typically require tens or hundreds of thousands of examples to become good at a task. But fMRI imaging is expensive, so a dataset of 100,000 brain scans of people looking at images just doesn鈥檛 exist.

To get around this, the team trained the caption-forming part of the AI on regular images with captions. This is the most complex task the AI performs, so requires a lot more training data to become accurate. They then trained the feature-extracting part on brain scans of a person viewing images. It learned to associate certain patterns of brain activity with certain features within the images the person was viewing.. It鈥檚 easier to extract features than generate captions, so this required much less data. Putting the two components together gave the final AI.

There are still questions around how good an AI system like this can be, says at University College London. 鈥淭here is obviously enough information in a brain to generate a description of a scene from scratch, as that鈥檚 what humans can do when looking at a scene.鈥� But fMRIs do not record everything the brain is doing, just a snapshot. This means there may be a limit to the amount of detail that can be extracted using this method.

In the past, teams have shown that it is possible to make very rough video clips estimating what a person is looking at using brain scans, as well detecting .

Applications are still a long way off, but companies like Facebook and Elon Musk鈥檚 are exploring technology to control computers directly with the brain. Rather than using fMRIs, which are big clunky machines, attempts along these lines normally focus on electroencephalography (EEG), which measures electrical activity in the brain via small electrodes placed on the scalp.

鈥淥nce such methods start to reliably decode what we imagine or think rather than what we see, I expect them to play an important role in the development of new neuroprosthetic devices,鈥� says G眉莽l眉.

Reference: arXiv,听听