Summary of Brotherhood at Wmt 2024: Leveraging Llm-generated Contextual Conversations For Cross-lingual Image Captioning, by Siddharth Betala and Ishan Chokshi
Brotherhood at WMT 2024: Leveraging LLM-Generated Contextual Conversations for Cross-Lingual Image Captioning
by Siddharth Betala, Ishan Chokshi
First submitted to arxiv on: 23 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents Brotherhood’s approach to multi-modal translation tasks for four language pairs: English-Hindi, English-Hausa, English-Bengali, and English-Malayalam. The team leverages Large Language Models (LLMs) like GPT-4o and Claude 3.5 Sonnet to enhance cross-lingual image captioning without traditional training or fine-tuning. They use instruction-tuned prompting to generate rich conversations about cropped images, translating these synthetic conversations into target languages. The method achieves competitive results, with a BLEU score of 37.90 on the English-Hindi Challenge Set and ranking first and second for English-Hausa on the Challenge and Evaluation Leaderboards. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about creating a way to translate pictures from one language to another without needing special training or fine-tuning. The team, called Brotherhood, uses big computers that can understand language (LLMs) like GPT-4o and Claude 3.5 Sonnet to help with this task. They use these computers to create fake conversations about pictures, using the original picture captions as context. Then, they translate these conversations into other languages. This method works pretty well, scoring high on a test and beating some other teams. |
Keywords
» Artificial intelligence » Bleu » Claude » Fine tuning » Gpt » Image captioning » Multi modal » Prompting » Translation
