An artificial intelligence that watches you while you eat aims to automatically gauge the calories and nutrients in each spoonful.
Using AI to measure these facets of a meal isn鈥檛 a new idea, with previous models able to take an image of food on a plate and provide an estimate.
But in a single image of a meal, some items 鈥� such as ingredients submerged in a bowl of stew 鈥� are easy to miss, says at the University of Waterloo in Canada.
Advertisement
To address this, Chen and his colleagues have developed a new model that analyses a spoonful at a time. This approach is more accurate and could be a useful way of monitoring calories and nutrition, particularly for the ill or elderly, he says.
The model analyses video of a person eating and detects each spoonful of food they consume. It then estimates the volume of food on the spoon, and the discrepancy between this and actual measurements is as little as 4.4 per cent.
The system isn鈥檛 yet able to identify foods and estimate their nutritional content, but Chen says the team is working on this. Ultimately, the hope is the tool will be able to recognise a wide range of different foods, even those it hasn鈥檛 seen before, and to analyse food held with a fork, a hand or chopsticks.
An AI model would typically be trained on images of food with accurate labels, but Chen hopes to integrate large language models into such a system to help it identify ingredients in unseen recipes, or even totally novel recipes.
鈥淲e鈥檙e shifting towards using those large language models like ChatGPT to leverage the common knowledge to understand what is in the food or maybe ask a basic question [like] 鈥榠s this chicken?鈥�,鈥� says Chen. 鈥淎 lot of time, especially for people eating at home, the dish may not be a named dish. It may be just whatever is available in the fridge that they鈥檝e mixed together.鈥�
at the University of Glasgow, UK, says remote measuring of calories can鈥檛 provide 100 per cent accuracy, so it may not be useful for strict scientific studies.
鈥淭here鈥檚 a lot of variability in vitamins in carrots,鈥� says Combet Aspray. 鈥淭here鈥檚 a lot of variability for everything in everything, and your specific carrot eaten today may be anywhere on that scale. A real challenge is the processing methods that we use to put food on our plates or, for example, estimating the amount of oil that has been used to fry a piece of meat. Those kind of camera-based tools aren鈥檛 necessarily able to evaluate that.鈥�
But the AI tool could be valuable for nutritionists and dieticians, or any application where an approximation is good enough, such as individuals tracking their calorie and nutrient intake, says Combet Aspray.
arXiv
