Karpathy’s MicroGPT running at 50,000 tps on an FPGA
Sure, it's only 4,192 parameters, but it's a start. Project write-up here: https://v2.talos.wtf/ and github repository here: https://github.com/Luthiraa/TALOS-V2 Some of the speed comes from having the weights onboard, rather than in external …