Summary of Adaptive Batch Size Schedules For Distributed Training Of Language Models with Data and Model Parallelism, by Tim Tsz-kit Lau et al.
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelismby…