I want to build my own AI server, I already have multiple servers at home but none have GPUs neither are powerful enough to host +4B models.
I'd like to be able to host dense 27-30b parameters models, or some MoE with 3b activated parameters.
Let's say I could spend about 2k, what would be the best route? And what tokens speeds should I expect?
[link] [comments]