Artificial intelligence can now rapidly search through billions of aerial and satellite images to find buildings or land features that are alike, such as football fields and Arctic ponds. This capability could help researchers classify the amount of land taken up by forests or farms, or could be used by the military to identify bases or specific weapons in other countries.
Xander Rudelis and his colleagues at Descartes Labs, a geospatial data firm in聽New Mexico, have developed an AI that takes one-tenth of a second to search more than 2 billion images. Given a certain feature 鈥� for example, a power plant, forest or car park 鈥� the tool can identify similar places around the world.
The AI search engine, which has an interactive , may be used to train other algorithms that identify more specific features. 鈥淐an you find every anti-aircraft gun in North Korea 鈥� questions like that,鈥� says Rudelis. The firm has previously collaborated with the US Defense Advanced Research Projects Agency on other work.
Advertisement
The tool functions a bit like Google鈥檚 image search facility. To create it, the team customised an AI that was already trained to classify what appeared in photographs, such as plants, animals and vehicles.
They trained it using the US National Agriculture Imagery Program database 鈥� which contains 2 billion aerial images from 48 US states 鈥撀燼s well as images captured around the globe by Landsat 8, a NASA Earth-observation satellite. It uses聽a combination of 512 visual cues, including shapes and colours, to search for similar scenes, such as rows of boats in water that indicate a marina.
To calculate its accuracy, the team used a measure called top-30 precision 鈥� the number of correct images that appeared in the top 30 images the AI suggested for a given query. Among 10 features, the average top-30 precision was 86 per cent, but this varied from 36 per cent for aeroplanes up to 100 per cent for storage tanks and rail yards.
The resolution of the satellite imagery was relatively low 鈥� with one pixel corresponding to about 15 metres 鈥� which explains why the AI performed better for larger features, says Rudelis.
鈥淭he results you get back are almost certainly what you鈥檙e looking for but they鈥檙e not going to be everything,鈥� he says. That is because the AI uses several features to confirm it has found a match 鈥� say, a white oblong shape surrounded by blue water to identify a boat. If it doesn鈥檛 catch all of these details, it might miss a match.
Rudelis says the team may use it to train other AIs to recognise only one kind of feature, which would likely increase how comprehensive its search function is, but it would take far more data.聽鈥淚f I want to map every solar power plant on Earth, then I鈥檓 going to need to start out with a lot of examples,鈥� he says.
Sergey Mushinskiy, a data scientist in Minsk, Belarus, says that because the AI only looks for visual similarities, it will only find objects where the surrounding environment is the same. If the search query is an image of a ship in the water, 鈥渋t will find all ships in the water, but it won鈥檛 find a ship in the dock because the ground is different鈥�, he says.
Last month, the US Bureau of Industry and Security on AIs that analyse satellite images, ruling that such software 鈥渕ay provide a significant military or intelligence advantage鈥�.
This technology may also come in handy for understanding the effects of climate change or for environmental mapping.聽It could be used by researchers to find certain natural structures worldwide, such as specified forest or rock types, says Mushinskiy. 鈥淢ilitary-type applications will require more specific detectors,鈥� he says.
Reference: arXiv, arxiv.org/abs/2002.02624
