Abstract / Introduction

Food recognition is becoming very important in areas such as digital health, dietary monitoring, and food delivery apps. Existing food classification systems can identify food items from images, but they usually stop there — they don't tell you about portion sizes or calorie content.

This project aims to build a deep learning–based system that not only detects and classifies multiple food items in a single image but also estimates portion sizes and calculates approximate calorie content.

The system will use object detection models (such as YOLOv8) to recognize food items like roti, dal, rice, salad, eggs, fruits, and drinks. A post-processing step will then map each detected food item to a calorie database. Portion size will be estimated using bounding box area (or segmentation masks), and calories will be calculated accordingly.

By combining food detection, portion size estimation, and calorie calculation, this project creates a complete end-to-end application for diet tracking and nutrition awareness.

Objectives

The main objectives of the project are:

• Collect and preprocess a multi-food image dataset (minimum 400 images) with bounding box annotations.
• Train and fine-tune an object detection model (YOLOv8n or YOLOv8s) for detecting multiple food items.
• Design a calorie-mapping module that links detected food items to standard nutrition values.
• Estimate portion sizes from bounding box area (or segmentation masks) and calculate calories accordingly.
• Develop a web-based application where users can upload an image and view: Detected food items with bounding boxes and labels, A side panel showing calorie and nutrient breakdown (per item + total).
• Document the complete process and evaluate system performance with metrics such as mAP, precision, recall, and calorie estimation accuracy.

Note: Dear Students, kindly read the Prerequisites and Methodology carefully and evaluate your skills before selecting this project. If you do not have the required knowledge, or if you are not willing to learn it wholeheartedly, you will not be able to complete this project successfully. Therefore, be very careful and thoughtful while choosing this project.

Prerequisites (what you must know / install)

Students should have a basic understanding of:

Python Programming:

Variables, loops, functions, file handling, and working with libraries like os, numpy, pandas.

Image Processing Basics:

Reading, resizing, and displaying images (OpenCV, PIL), pixels, and image shapes.

Machine Learning & Deep Learning:

• Basics of supervised learning (train/test split, accuracy, evaluation).
• CNN (Convolutional Neural Networks) – how images are classified.
• Object detection concepts – bounding boxes, IoU, precision/recall.
• Awareness of YOLO, Faster R-CNN, Mask R-CNN (at least conceptually).

Dataset Preparation & Annotation:

• How to annotate bounding boxes using tools (LabelImg, Roboflow).
• How datasets are structured for object detection (images + labels in .txt or .xml).

Nutrition & Calorie Mapping (Basic):

• General idea of how food calorie values are measured.
• Awareness that portion size affects calories.
• Example: 1 roti ≈ 70 kcal, 1 cup rice ≈ 200 kcal, 1 boiled egg ≈ 75 kcal.

Methodology

Learn the Basics of Python and Machine Learning

Before starting the project, you must learn Python programming, deep learning basics, and object detection concepts.

You should cover the following topics:

• Python basics: variables, loops, functions, file handling.
• Python libraries: NumPy, Pandas, Matplotlib.
• Deep Learning basics: CNN (Convolutional Neural Networks).
• Object Detection concepts: bounding boxes, mAP, IoU.

If you are unable to understand these concepts do not choose this project.

Setup your development environment:

To implement the project, students must set up a proper development environment:

1. Download and install the python
2. Install a development environment
3. Install the necessary libraries that helps you with image processing, data handling, and model training

Data Collection:

1. Collect a diverse collection of multi-food images featuring different meal composition. You can collect manually using mobile cameras or use the free dataset (Food-101 Dataset, UECFOOD100 Dataset, Vireo Food-172 Dataset) and modify it.

Following are the requirements of Dataset:

• 10 classes (Roti/Chapati, Egg, Drink, Fruits, Vegetables, Salad, Pizza Slice, Pulao, Bread Slice).
• 40 images per class → 400 total images.
• Each image must contain 2 different food items.

2. Label images with bounding boxes or segmentation masks for each food item using tools like LabelImg or Roboflow.
3. Ensure that the dataset is structured properly with annotations and labels for object detection.

Load and visualize the data:

Before working on the model, students need to understand the dataset by loading the images and plotting samples from different categories. Visualization tools like Matplotlib and OpenCV can be used for this purpose.

Data Pre-processing:

In this step, you have to clean and prepare the data for analysis.

• Resize all images to a standard size (e.g., 416x416 pixels for YOLO).
• Normalize pixel values to a range between 0 and 1 for efficient processing.
• Apply data augmentation (rotation, contrast adjustments, brightness variations) to improve model robustness.

Split the Data into Training and Testing Sets:

Students need to divide the dataset into training and testing sets:

• Training Set: 80% of the data, used for model training.
• Testing Set: 20% of the data, used to evaluate the model's performance on unseen images.

Build a Model:

Students need to select an appropriate object detection model YOLOv8n (Nano) or YOLOv8s (Small) for food classification.

Model Training:

Train the object detection model on the prepared dataset. Fine-tune with annotated dataset to improve accuracy and robustness.

Note: You have the flexibility to use any algorithm for your implementation.

Evaluation:

Once the model is trained, you need to evaluate its performance on the testing data.

• Evaluate model performance using precision, recall, and mAP (mean Average Precision).
• Plot a confusion matrix to see where the model is performing well and where it's making mistakes.

Calorie Mapping:

• Create a lookup table with average calories per class (from USDA database).
• Estimate portion size by bounding box area → scale calories.

Example (per serving):

• Roti = 120 kcal
• Egg = 70 kcal
• Rice = 130 kcal per 100g
• Salad = 20 kcal per 100g

User Interface (UI) Design:

Develop a web or mobile application for real-time food detection: User uploads an image → model detects → bounding boxes + calorie breakdown displayed.

Note: Kindly read the following guidelines before choosing the project.

• Kindly read the proposal carefully and decide if you have completely understood the project requirements before selecting the project.
• You have to implement the requirements mentioned in project proposal completely. You are not allowed to add irrelevant and un-necessary requirements.
• You have to implement the project in mentioned tools and technology.
• Kindly do not request to use php or html for image processing project.
• Do not ask to share dataset because it is your task to find the appropriate dataset.
• Student must have knowledge of image processing techniques.