Summary of Speeding Up and Reducing Memory Usage For Scientific Machine Learning Via Mixed Precision, by Joel Hayford et al.
Speeding up and reducing memory usage for scientific machine learning via mixed precision
by Joel Hayford, Jacob Goldman-Wetzler, Eric Wang, Lu Lu
First submitted to arxiv on: 30 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this paper, the authors investigate the use of scientific machine learning (SciML) to solve complex problems in physics and engineering. Specifically, they focus on physics-informed neural networks (PINNs) and deep operator networks (DeepONets), which are designed to solve partial differential equations by incorporating physical laws and experimental data. However, these methods require significant computational resources, including long training times and large memory usage. To address this issue, the authors explore the use of mixed precision training, which combines float16 and float32 numerical formats to reduce memory usage and increase computational speed. Their experiments show that mixed precision training can significantly decrease training times and memory demands while maintaining model accuracy. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about using computers to solve complex problems in science and engineering. It talks about special kinds of computer programs called physics-informed neural networks (PINNs) and deep operator networks (DeepONets). These programs are good at solving puzzles that involve physical laws, like how things move or change. But they use a lot of computer power and memory, which can be a problem. To make them work better, the authors tried using something called mixed precision training. This is like using two different kinds of batteries in your calculator to make it run faster and use less space. The experiments show that this works really well and makes the programs run faster and use less computer power. |
Keywords
* Artificial intelligence * Machine learning * Precision
