Summary of Sketchref: a Benchmark Dataset and Evaluation Metrics For Automated Sketch Synthesis, by Xingyue Lin et al.
SketchRef: A Benchmark Dataset and Evaluation Metrics for Automated Sketch Synthesis
by Xingyue Lin, Xingjian Hu, Shuai Peng, Jianhua Zhu, Liangcai Gao
First submitted to arxiv on: 16 Aug 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A machine learning-based artistic technique called Sketch is gaining attention in the image synthesis field. Evaluating the quality of synthesized sketches poses unique challenges due to the lack of a unified benchmark dataset and reliance on classification accuracy for recognizability. To address these issues, the authors introduce SketchRef, a benchmark dataset comprising 4 categories of reference photos along with novel evaluation metrics. The proposed mean Object Keypoint Similarity (mOKS) metric assesses structural consistency between a sketch and its reference photo using pose estimation. Additionally, a recognizability calculation method constrained by simplicity ensures fair evaluation of sketches with different simplification levels. The authors collect 8K responses from art enthusiasts validating the effectiveness of their proposed evaluation methods. This work aims to provide a comprehensive evaluation of sketch synthesis algorithms aligning their performance more closely with human understanding. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Sketch is an artistic technique used in image synthesis that’s getting popular. It’s hard to tell if synthesized sketches are good or not because there’s no standard way to compare them, and we only look at how well they can be recognized. To solve this problem, researchers created a new dataset called SketchRef with pictures of different things (animals, faces, bodies, objects) and came up with new ways to measure sketch quality. They also used something called pose estimation to see if the sketch looks like the real picture. This helps make sure we’re comparing apples to apples when it comes to how simple or complicated the sketches are. Art enthusiasts agree that this way of evaluating sketches is a good one. |
Keywords
» Artificial intelligence » Attention » Classification » Image synthesis » Machine learning » Pose estimation
