Summary of Doce: Finding the Sweet Spot For Execution-based Code Generation, by Haau-sing Li et al.
DOCE: Finding the Sweet Spot for Execution-Based Code Generation
by Haau-Sing Li, Patrick Fernandes, Iryna Gurevych, André F.T. Martins
First submitted to arxiv on: 25 Aug 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Programming Languages (cs.PL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A comprehensive framework for LLM-based code generation is proposed, comprising candidate generation, n-best reranking, minimum Bayes risk (MBR) decoding, and self-debugging as core components. The framework is evaluated using execution-based metrics, highlighting the importance of execution-based methods and their differences from execution-free approaches. Additionally, filtering based on trial unit tests is found to be an effective yet often overlooked strategy, leading to state-of-the-art performance in reranking for code generation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way to make computers write code more accurately is being developed. This method uses different techniques to find the best code and then tests it to see if it’s correct. The researchers found that using this testing step makes a big difference in how well the code works. They also discovered that filtering out bad code by testing small parts of it first can greatly improve the results. This new approach could be very useful for people who need help writing code. |
