Strix Halo or GPUs?

I want to build my own AI server, I already have multiple servers at home but none have GPUs neither are powerful enough to host +4B models.

I'd like to be able to host dense 27-30b parameters models, or some MoE with 3b activated parameters.

Let's say I could spend about 2k, what would be the best route? And what tokens speeds should I expect?

submitted by /u/undernightcore
[link] [comments]

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top