Summary of Loretta: Low-rank Economic Tensor-train Adaptation For Ultra-low-parameter Fine-tuning Of Large Language Models, by Yifan Yang et al.
LoRETTA: Low-Rank Economic Tensor-Train Adaptation for Ultra-Low-Parameter Fine-Tuning of Large Language Modelsby Yifan Yang, Jiajun…