Menu-Match: Restaurant-Specific Food Logging from Images

Menu-Match: Restaurant-Specific Food Logging from Images
Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV 2015), January 2015
Oscar Beijbom Neel Joshi Dan Morris Scott Saponas Siddharth Khullar
Microsoft Research

A sample of the food images we can recognize, from our three restaurant dataset.	ROC curves for our method applied to our dataset.

Abstract

Logging food and calorie intake has been shown to facilitate weight management. Unfortunately, current food logging methods are time-consuming and cumbersome, which limits their effectiveness. To address this limitation, we present an automated system for logging food and calorie intake using images. We focus on the "restaurant" scenario, which is often a challenging aspect of diet management. We introduce a key insight that addresses this problem specifically: restaurant plates are often both nutritionally and visually consistent across many servings, suggesting an opportunity for training restaurant-specific classifiers. We demonstrate a proofof- concept system on a challenging three-restaurant dataset and show that both food items and calorie information can be inferred accurately by matching a plate of food items to an existing database of known food items. We additionally present quantitative results on a publicly available food dataset.


	Paper
	Data

Microsoft Research