Menu-Match: Restaurant-Specific
Food Logging from Images

Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV 2015), January 2015

Oscar Beijbom       Neel Joshi       Dan Morris       Scott Saponas       Siddharth Khullar

Microsoft Research

A sample of the food images we can recognize, from our three restaurant dataset.           ROC curves for our method applied to our dataset.

Logging food and calorie intake has been shown to facilitate weight management. Unfortunately, current food logging methods are time-consuming and cumbersome, which limits their effectiveness. To address this limitation, we present an automated system for logging food and calorie intake using images. We focus on the "restaurant" scenario, which is often a challenging aspect of diet management. We introduce a key insight that addresses this problem specifically: restaurant plates are often both nutritionally and visually consistent across many servings, suggesting an opportunity for training restaurant-specific classifiers. We demonstrate a proofof- concept system on a challenging three-restaurant dataset and show that both food items and calorie information can be inferred accurately by matching a plate of food items to an existing database of known food items. We additionally present quantitative results on a publicly available food dataset.


Contact Terms Trademarks Privacy and Cookies Code of Conduct © Microsoft Corporation. All rights reserved.Microsoft