For me I do, my graphics card is “old” GTX 1080, it is from 2016 or 2017, forgot when already, when they released it, the Nvidia Guy went on stage talking about the Pascal Architecture like they invented teleportation or something, and we all ran to give him our thousand dollars :)
So, I am still waiting for the "teleportation" feature to be enabled in the next driver :)
Today the error messages are sorry, blah blah pascal, balh blah unsupported, legacy, blah blah
Looks like 30b to 50b AI Models are evolving to become the sweet spot, the one “able to do work” models, and I will get a card that runs one the moment it is 1000$ ~ 2000$ and can do a few hundred tokens per second, which is maybe far away, or just a normal mobile phone in 2030 or 2035
So, meanwhile, I use subscriptions.
I am wondering if other Local LLM users are doing the same?
[link] [comments]