Summary of Can Language Beat Numerical Regression? Language-based Multimodal Trajectory Prediction, by Inhwan Bae and Junoh Lee and Hae-gon Jeon
Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction
by Inhwan Bae, Junoh Lee, Hae-Gon Jeon
First submitted to arxiv on: 27 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Robotics (cs.RO)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes LMTraj, a language-based multimodal trajectory predictor that recasts the traditional numerical regression model for pedestrian trajectory prediction. Inspired by language foundation models, it transforms input spaces into natural language prompts and uses image captioning to describe scene images as text information. The transformed data is then wrapped into a question-answering template for use in a language model. To guide the model’s understanding of high-level knowledge, an auxiliary multi-task question-and-answering approach is introduced. A numerical tokenizer is trained to separate integer and decimal parts well, capturing correlations between consecutive numbers in the language model. The paper shows that LMTraj outperforms existing numerical-based predictor methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper uses a new way to predict where people will walk. Instead of using just numbers, it turns the data into text prompts and asks a computer to answer questions based on those prompts. This helps the computer understand more about what’s happening in the scene, like who is near each other. The computer then makes predictions about where people will go next. The results show that this new approach works better than old methods. |
Keywords
* Artificial intelligence * Image captioning * Language model * Multi task * Question answering * Regression * Tokenizer
