Clarifai 12.3 introduces KV Cache-Aware Routing. Routes requests to replicas with relevant cache state for faster inference. Zero configuration required.
Clarifai 12.3: Introducing KV Cache-Aware Routing
Clarifai 12.3 introduces KV Cache-Aware Routing. Routes requests to replicas with relevant cache state for faster inference. Zero configuration required.