Keys, queries, and values are all vectors inside the LLMs. RoPE [66] includes the rotation of the question and crucial representations at an a