Computer Vision – From Image to Nutrition
It's easy to take a photo of a meal, but extracting reliable data about what was eaten and in what quantities requires true computer vision work.
Take photos of snacks and meals and calculate their nutritional values.
A food tracking system that works in everyday life would be so helpful. Instead of laboriously weighing and typing in every ingredient, from pasta to onions and tomato sauce to oil and salt, it would use photos.
We train AI models to recognize dishes and ingredients in a food photo and estimate portion sizes as accurately as possible using recipe databases and the corresponding food photos. Our multi-stage pipeline includes robust meal recognition, clean segmentation into individual foods, and weight estimation. Finally, we link the ingredients to the Bundeslebensmittelschlüssel, the official German nutritional database.
For research purposes, this means that the standardized collection of data on daily food intake is possible for participants with minimal effort, across large cohorts, and with consistent quality. For nutritional counseling, prevention programs, and medical care, this improves data quality, patient participation, and adherence.