Wait is attn rotate already enabled by default since this release tell it support SWA attention?

Wait is attn rotate already enabled by default since this release tell it support SWA attention?

For the past 2 weeks, my daily routine has included checking the main llama.cpp releases to see if attn rotate has been merged. Am I missing something? I mean, it should be there already since the core rotation PR has been merged. Is it enabled by default?

submitted by /u/Altruistic_Heat_9531
[link] [comments]

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top