Loading Now

Summary of Deep Image-to-recipe Translation, by Jiangqin Ma et al.


Deep Image-to-Recipe Translation

by Jiangqin Ma, Bilal Mawji, Franz Williams

First submitted to arxiv on: 1 Jul 2024

Categories

  • Main: Computer Vision and Pattern Recognition (cs.CV)
  • Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)

     Abstract of paper      PDF of paper


GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty Written by Summary
High Paper authors High Difficulty Summary
Read the original abstract here
Medium GrooveSquid.com (original content) Medium Difficulty Summary
This project combines computer vision and natural language generation to connect food memories to culinary creation. The primary goal is to predict ingredients from a given food image using a custom convolutional network or one that leverages transfer learning. To generate recipe steps from ingredient lists, the researchers framed this as a sequence-to-sequence task and developed a recurrent neural network utilizing pre-trained word embeddings. They addressed challenges like imbalanced datasets, data cleaning, overfitting, and hyperparameter selection, emphasizing metrics like Intersection over Union (IoU) and F1 score. The recipe prediction model used perplexity as an important metric for language models. Transfer learning via pre-trained ResNet-50 weights and GloVe embeddings significantly boosted model performance.
Low GrooveSquid.com (original content) Low Difficulty Summary
Imagine a world where you can turn a picture of your favorite food into a recipe! That’s what this project is all about. They’re trying to figure out how to take a picture of, say, a pizza, and then give you the ingredients and steps to make it yourself. To do that, they developed special kinds of computer programs that can look at pictures and understand what’s in them. They also made other programs that can take lists of ingredients and turn them into recipe steps. The researchers had to overcome some big challenges along the way, but their work has already shown a lot of promise.

Keywords

» Artificial intelligence  » Convolutional network  » F1 score  » Glove  » Hyperparameter  » Neural network  » Overfitting  » Perplexity  » Resnet  » Transfer learning