Summary of Deep Image-to-recipe Translation, by Jiangqin Ma et al.
Deep Image-to-Recipe Translation
by Jiangqin Ma, Bilal Mawji, Franz Williams
First submitted to arxiv on: 1 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This project combines computer vision and natural language generation to connect food memories to culinary creation. The primary goal is to predict ingredients from a given food image using a custom convolutional network or one that leverages transfer learning. To generate recipe steps from ingredient lists, the researchers framed this as a sequence-to-sequence task and developed a recurrent neural network utilizing pre-trained word embeddings. They addressed challenges like imbalanced datasets, data cleaning, overfitting, and hyperparameter selection, emphasizing metrics like Intersection over Union (IoU) and F1 score. The recipe prediction model used perplexity as an important metric for language models. Transfer learning via pre-trained ResNet-50 weights and GloVe embeddings significantly boosted model performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine a world where you can turn a picture of your favorite food into a recipe! That’s what this project is all about. They’re trying to figure out how to take a picture of, say, a pizza, and then give you the ingredients and steps to make it yourself. To do that, they developed special kinds of computer programs that can look at pictures and understand what’s in them. They also made other programs that can take lists of ingredients and turn them into recipe steps. The researchers had to overcome some big challenges along the way, but their work has already shown a lot of promise. |
Keywords
» Artificial intelligence » Convolutional network » F1 score » Glove » Hyperparameter » Neural network » Overfitting » Perplexity » Resnet » Transfer learning
