Quants in vision (mmproj Q8 vs FP16)
Disclaimer: This is totally just my personal testing/messing around. Nothing scientific. TL;DR: I find FP16 mmproj pointless, and may even harm quality rather than help. I decided to check vision of the recent small models on llama.cpp. I didn't kn…