Summary of Obi-bench: Can Lmms Aid in Study Of Ancient Script on Oracle Bones?, by Zijian Chen et al.
OBI-Bench: Can LMMs Aid in Study of Ancient Script on Oracle Bones?
by Zijian Chen, Tingzhu Chen, Wenjun Zhang, Guangtao Zhai
First submitted to arxiv on: 2 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces OBI-Bench, a benchmark designed to evaluate large multi-modal models (LMMs) on whole-process oracle bone inscriptions (OBI) processing tasks. The benchmark includes 5,523 images covering five domain problems: recognition, rejoining, classification, retrieval, and deciphering. These images span centuries of archaeological findings and years of research by front-line scholars. OBI-Bench focuses on advanced visual perception and reasoning with OBI-specific knowledge, challenging LMMs to perform tasks akin to those faced by experts. The evaluation of 23 proprietary and open-source LMMs highlights the substantial challenges posed by OBI-Bench. While some models are far from public-level humans in fine-grained perception tasks, they can offer new interpretative perspectives and generate creative guesses, comparable to untrained humans in deciphering tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper creates a special test for computer models that try to understand ancient writing. They made a big collection of images showing different parts of old oracle bones. These images are tricky because they need the model to recognize and understand things like letters, words, and even whole sentences. The test is hard because it’s not just about recognizing things – it’s also about understanding what those things mean. Some computer models did pretty well, but most didn’t do as well as humans who aren’t experts in ancient writing. This means that these models are good at making guesses or suggesting new ideas, even if they’re not perfect. |
Keywords
» Artificial intelligence » Classification » Multi modal
