Summary of Unsupervised Parameter-free Outlier Detection Using Hdbscan* Outlier Profiles, by Kushankur Ghosh et al.
Unsupervised Parameter-free Outlier Detection using HDBSCAN* Outlier Profiles
by Kushankur Ghosh, Murilo Coelho Naldi, Jörg Sander, Euijin Choo
First submitted to arxiv on: 13 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes an unsupervised strategy for outlier detection in machine learning, focusing on hierarchical density-based clustering methods like HDBSCAN*. The approach, called Global-Local Outlier Scores based on Hierarchies (GLOSH), estimates outlier scores by comparing data point densities to the highest density of their region. GLOSH may be sensitive to the minpts parameter that influences density estimation. To overcome this limitation, the authors propose an unsupervised strategy to find the “best” minpts value by leveraging GLOSH score ranges across different minpts values. Additionally, they suggest a threshold-based approach for classifying points as inliers or outliers without pre-defining any value. Experimental results demonstrate that these strategies can automatically identify optimal minpts and threshold values for effective outlier detection using GLOSH. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Outliers are unusual data points that can affect statistics and models. To find them, we need to detect anomalies in large datasets. The paper proposes a new method called GLOSH (Global-Local Outlier Scores based on Hierarchies) that uses clustering techniques like HDBSCAN*. GLOSH estimates how likely each point is an outlier by comparing its density to the highest density of its region. However, this approach depends on a parameter called minpts, which affects how clusters are formed. To solve this problem, the authors suggest two new methods: one finds the best minpts value and another defines when a point is considered an outlier. These strategies can help us identify outliers without knowing how many there are or what they look like. |
Keywords
» Artificial intelligence » Clustering » Density estimation » Machine learning » Outlier detection » Unsupervised
