Summary of Large Language Models Memorize Sensor Datasets! Implications on Human Activity Recognition Research, by Harish Haresamudram et al.
Large Language Models Memorize Sensor Datasets! Implications on Human Activity Recognition Research
by Harish Haresamudram, Hrudhai Rajasekhar, Nikhil Murlidhar Shanbhogue, Thomas Ploetz
First submitted to arxiv on: 9 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Signal Processing (eess.SP)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the surprising success of Large Language Models (LLMs) in Human Activity Recognition (HAR) tasks, where wearable sensor data are fed into LLMs along with text instructions for activity classification. Despite remarkable results on standard benchmarks, the authors argue that care must be taken when evaluating LLM-based HAR systems due to potential contamination of training data. They apply memorization tests to LLMs, finding a non-negligible amount of matches suggesting that LLMs may have accessed standard HAR datasets during training. For example, GPT-4 is able to reproduce blocks of sensor readings for the Daphnet dataset. The authors discuss potential implications on HAR research, especially regarding reporting results on experimental evaluations. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how well Large Language Models (LLMs) can recognize human activities using data from sensors like smartwatches or fitness trackers. Normally, LLMs are great at recognizing text, but this paper wonders if they’re just too good because they’ve seen the test data before. They tested some LLMs to see if they could remember specific parts of sensor data, and found that some of them could! This means that when we test these models on real-life data, their results might not be as accurate as we think. The paper talks about what this means for scientists who study how well machines can recognize human activities. |
Keywords
* Artificial intelligence * Activity recognition * Classification * Gpt
