"What do you guys even use local LLMs for?" Me: A lot

"What do you guys even use local LLMs for?" Me: A lot

Created separate private API keys for each service within LiteLLM and started logging the usage via Prometheus to view in Grafana. Surprised the Frigate GenAI summaries tokens quickly add up!

This view is only the past 6 hours.

submitted by /u/andy2na
[link] [comments]

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top