Summary of Hope: a Novel Positional Encoding Without Long-term Decay For Enhanced Context Awareness and Extrapolation, by Yuhan Chen et al.
HoPE: A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolationby Yuhan…