Position Encoding Attention Is All You Need 中提出了 Sinusoidal Position Encoding PE(pos,2i)=sin(posf2i/dmod)PE(pos,2i+1)=cos(posf2i/dmod)\begin{aligned} \mathrm{PE}(pos,2i) &= \sin \left(\frac{pos}{f^{2i/d_{mod}}}\right)\\ \mathrm{PE}(pos,2i+1)& = \cos \left(\frac{pos}{f^{2i/d_{mod}}}\right) \end{aligned} PE(pos,2i)PE(pos,2i+1)=sin(f2i/dmodpos)=cos(f2i/dmodpos) 对于token vector pos=pos = pos= LLMencoding
