Loading Now

Summary of See Then Tell: Enhancing Key Information Extraction with Vision Grounding, by Shuhang Liu et al.

See then Tell: Enhancing Key Information Extraction with Vision Grounding

by Shuhang Liu, Zhenrong Zhang, Pengfei Hu, Jiefeng Ma, Jun Du, Qing Wang, Jianshu Zhang, Chenyu Liu

First submitted to arxiv on: 29 Sep 2024

Categories

  • Main: Computer Vision and Pattern Recognition (cs.CV)
  • Secondary: Artificial Intelligence (cs.AI)

     Abstract of paper      PDF of paper

GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty Written by Summary
High Paper authors High Difficulty Summary
Read the original abstract here
Medium GrooveSquid.com (original content) Medium Difficulty Summary
A novel end-to-end model called STNet is introduced to deliver precise answers with relevant vision grounding for visually rich documents integrating text, complex layouts, and imagery. This approach bypasses traditional Optical Character Recognition (OCR) methods, which can introduce latency, computational overhead, and errors. STNet uses a unique token to observe pertinent image areas, aided by a decoder that interprets physical coordinates linked to this token. The model is trained on the TVG dataset, which combines text-based Question Answering (QA) pairs with precise vision grounding. This approach demonstrates substantial advancements in Key Information Extraction (KIE) performance, achieving state-of-the-art results on publicly available datasets such as CORD, SROIE, and DocVQA.
Low GrooveSquid.com (original content) Low Difficulty Summary
In this paper, a new way to understand documents is introduced. It’s called STNet, and it helps computers see important parts of images when answering questions about the document. This approach is better than old methods that use Optical Character Recognition (OCR) because it doesn’t have as many mistakes or take as long. The model looks at specific parts of the image using a special token called . It then uses this information to provide answers in text form. The authors created a new dataset called TVG that helps train the model. This approach works well and is better than other methods.

Keywords

» Artificial intelligence  » Decoder  » Grounding  » Question answering  » Token  

Related Posts