Summary of Representing Online Handwriting For Recognition in Large Vision-language Models, by Anastasiia Fadeeva et al.
Representing Online Handwriting for Recognition in Large Vision-Language Models
by Anastasiia Fadeeva, Philippe Schlattner, Andrii Maksai, Mark Collier, Efi Kokiopoulou, Jesse Berent, Claudiu Musat
First submitted to arxiv on: 23 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach to online handwriting recognition using vision-language models (VLMs). It begins by noting that VLMs are state-of-the-art for image understanding, but struggle with handwritten text. The authors create a tokenized representation of digital ink that combines time-ordered stroke sequences and images. This representation achieves results comparable or better than existing online handwriting recognizers, using two different VLM families on multiple public datasets. The approach can be applied to off-the-shelf VLMs without modifying their architecture, and is suitable for both fine-tuning and parameter-efficient tuning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper uses special computers that are really good at understanding pictures to help recognize handwritten text. These computers are great at recognizing things like cats or cars, but struggle with words written in pen. The authors came up with a new way of showing the computer what the handwritten text looks like, combining the order of how the person wrote each letter and what it actually looks like. This helps the computer understand the handwriting better and gets even better results than other methods that are already used. This is important because it could help people search for things on their tablets and use special tools to make writing easier. |
Keywords
* Artificial intelligence * Fine tuning * Parameter efficient
