Summary of Curating Stopwords in Marathi: a Tf-idf Approach For Improved Text Analysis and Information Retrieval, by Rohan Chavan et al.
Curating Stopwords in Marathi: A TF-IDF Approach for Improved Text Analysis and Information Retrieval
by Rohan Chavan, Gaurav Patil, Vishal Madle, Raviraj Joshi
First submitted to arxiv on: 16 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper focuses on developing a strong stopword list for the Marathi language, which is a low-resource Indian language. The authors use the MahaCorpus, containing 24.8 million sentences, and employ the TF-IDF approach to curate a list of 400 stopwords. They evaluate the effectiveness of this approach through human evaluation. The curated stopwords are then applied to the text classification task, demonstrating their efficacy. This work presents a simple recipe for stopword curation in low-resource languages. Additionally, the authors integrate the stopwords into the mahaNLP library and make them publicly available on GitHub. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about creating a list of important words called “stopwords” that are not useful for understanding texts in the Marathi language. Marathi is a language spoken in India, but it’s harder to work with because there aren’t as many resources available compared to languages like English. The authors use a big database of sentences and a special technique to create their list of stopwords. They test how well this approach works by asking people to review the results. This helps improve text classification, which is important for tasks like analyzing people’s opinions or classifying news articles. |
Keywords
» Artificial intelligence » Stopword » Text classification » Tf idf
