An AI linked to a camera can tell the physical properties of surfaces without touching them.
Matthew Purri and Kristin Dana at Rutgers University in New Jersey have trained an algorithm that can determine the tactile characteristics of an object when presented solely with a photograph or series of images of it.
They took photographs of more than 400 materials, including cloth, plastic, leather and surfaces. Then they took 100 images of each surface using a device with a mechanical arm that can be moved precisely to take images at exact camera angles.
Advertisement
These images were linked to an existing data set about the materials. For each material, 15 physical properties were logged in categories including friction, adhesion and texture.
Using all this, they trained a deep-learning algorithm and tested it on surfaces it hadn鈥檛 seen before. Given a single image taken from directly above an object, the algorithm could reliably estimate 14 out of 15 of its surface properties. It struggled with , being unable to tell how sticky an object was based on vision alone.
When provided with more images at different camera angles, the algorithm鈥檚 accuracy improved, particularly for textures with larger features.聽From a straight-down view, the algorithm might be unable to determine that a roughly textured surface was bumpy. 鈥淏ut from very extreme angles, it can see the geometric properties changing,鈥� says Purri.
The researchers believe the algorithm could be used in robots and in 聽to help estimate the surface properties of roads.
鈥淏lack ice is very hard to see and very dangerous,鈥� says Purri. 鈥淵ou may be able to estimate [its properties] under certain conditions based on visual information alone.鈥� Being able to proactively estimate how slippery a road is may make braking systems safer, he says.
Reference:
