Summary of Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms, by Miaosen Zhang et al.
Aligning Vision Models with Human Aesthetics in Retrieval: Benchmarks and Algorithms
by Miaosen Zhang, Yixuan Wei, Zhen Xing, Yifei Ma, Zuxuan Wu, Ji Li, Zheng Zhang, Qi Dai, Chong Luo, Xin Geng, Baining Guo
First submitted to arxiv on: 13 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a preference-based reinforcement learning method to fine-tune vision models to align with human aesthetic standards in retrieval systems. The approach utilizes large language models (LLMs) to rephrase search queries and extend aesthetic expectations, addressing limitations of current aesthetic models. The method is evaluated using rare benchmarks designed for evaluating retrieval systems and a novel dataset named HPIR, which validates the robustness of large multi-modality model (LMM) in assessing aesthetic performance. Experiments demonstrate significant enhancement of aesthetic behaviors of vision models under various metrics. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper aims to help computers understand human preferences when it comes to visual aesthetics. Right now, computer vision models can’t always follow what humans want them to do, especially when it comes to things like style or responsibility. To fix this, the researchers propose a new way to fine-tune these models using large language models (LLMs) that can reason about words and ideas. This approach shows promise in making computers more aesthetically pleasing. |
Keywords
» Artificial intelligence » Reinforcement learning
