Summary of Dof: Accelerating High-order Differential Operators with Forward Propagation, by Ruichen Li et al.
DOF: Accelerating High-order Differential Operators with Forward Propagation
by Ruichen Li, Chuwei Wang, Haotian Ye, Di He, Liwei Wang
First submitted to arxiv on: 15 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Solving partial differential equations efficiently is crucial for analyzing complex physical systems, and recent advances in deep learning have shown promise. The Physics-Informed Neural Networks (PINN) approach faces challenges handling high-order derivatives of neural network-parameterized functions. To address this, we propose Differential Operator with Forward-propagation (DOF), a framework calculating general second-order differential operators without sacrificing precision. Our method outperforms existing methods, demonstrating two times improvement in efficiency and reduced memory consumption on any architectures. Empirical results show DOF surpassing traditional automatic differentiation techniques, achieving 2x improvement for MLP structures and nearly 20x for MLP with Jacobian sparsity. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Solving partial differential equations helps us understand complex physical systems. Recently, deep learning has shown promise in solving these equations. However, some methods struggle to handle high-order derivatives. To fix this, we created a new way to calculate general second-order differential operators called DOF (Differential Operator with Forward-propagation). Our method is more efficient and uses less memory than other methods. We tested it on different architectures and found that it works better than traditional automatic differentiation techniques. |
Keywords
* Artificial intelligence * Deep learning * Neural network * Precision
