Stitching video together from 54 cameras using AI produces incredibly detailed three-dimensional videos. The technique has been used to film many tiny creatures at once.
To understand why large groups of small animals, like ants or flies, behave the way they do, researchers want to see what every individual in the group is doing in each moment. But recording devices can normally capture either a single animal in great detail or many animals without those details, says at the University of California, Berkeley. He and his colleagues have now combined 54 cameras into a device that captures detail in both cases and at a very fast rate.
They placed individual harvester ants (Pogonomyrmex barbatus), zebrafish larvae (Danio rerio) and fruit flies (Drosophila hydei) in a shallow box 25 centimetres below a 6-by-9 grid of small cameras, each less than a millimetre tall and wide, and illuminated them with LED lights. All cameras simultaneously recorded video and the researchers trained a neural network to stitch the recordings together.
Advertisement
Zhou says the AI learned to do something similar to how smartphones create panoramas from series of images, but in a more complicated setting and in 3D. 鈥淥ur brain does something similar too, but here the challenge was having 54 eyes,鈥� he says.
The finished video displayed about 5 gigapixels per second or a hundred times more than a high-definition television. It captured movements that are so fast they are imperceptible to the human eye.
The researchers saw how every zebrafish larva in a group of 40 tilted up or down while approaching food, and the bending of joints on any leg of any of a dozen ants, all at once. Other teams have previously studied single animals in such detail, but the new device can help determine whether that is what many animals in a group do.
at Stanford University in California says that a limitation of the device may be that it can鈥檛 capture creatures smaller than ants or flies. Microscopic organisms, for instance, are often semi-transparent, which could throw of the AI鈥檚 stitching method, he says.
Zhou says that the team is now working on adapting the device for objects as small as single cells and making it capable of imaging some processes inside of small animals that have been injected with special glowing compounds.
Nature Photonics
