Artificial intelligence can be used to generate a 3D model of the human tongue from a single photograph of a subject鈥檚 face. The technique could lead to more realistic 3D avatars in virtual reality and computer games.
Techniques to create a 3D model of the human head from a single image have been demonstrated before but the research has historically ignored the tongue despite it having a large impact on appearance, says at Imperial College London.
鈥淭he avatar is not just the face, as everybody else has focused on, it鈥檚 other parts like the eyes, the eyelids, the tongue, the ears,鈥� he says.
Advertisement
鈥淲e have teeth inside our mouth and we also have the tongue, so when we speak you鈥檙e looking at my eyes but you鈥檙e also looking at my mouth in order to understand what I鈥檓 saying. And having those small components adds to the realism of the character.鈥�
Ploumpis argues that avatars with accurate oral cavities and tongues will look more realistic, and help to avoid the 鈥渦ncanny valley鈥� 鈥� the term given to the unsettling appearance of human models that look too realistic to be immediately dismissed as fake but not realistic enough to be truly compelling. This will be particularly beneficial when avatars speak, says Ploumpis.
However, the tongue is a difficult organ to model because it can form many shapes and its uniform surface appearance lacks convenient reference points or landmarks that algorithms can use to create models.
To look for solutions, Ploumpis and his colleagues collected 1800 3D scans from 700 people visiting the Science Museum in London. The individuals were instructed to put their tongues in various poses such as stuck out and to the left, to the right or straight out, to get a range of data.
The team used these 3D scans, as well as 2D photographs of the faces, to train an adversarial neural network to create new 3D models from unseen 2D images that included the face and the tongue. The adversarial approach involves two neural networks: one trained to create 3D images from 2D images, and another designed to spot how authentic the results are. By working against each other in this way and sharing results they are able to improve each other鈥檚 algorithm and create higher levels of realism.
Ploumpis says that the resulting model is now able to create compelling 3D models based on 2D images taken in variable conditions, including standard images taken from the internet.
Reference:
