We can use continuous batching for agent swarm to drastically reduce the time for research or coding.
we can use continuous batching for an agent swarm to actually kill research time. found performance for qwen 27b on that intel b70 32gb card. if you just chat one on one, you get: avg prompt throughput: 85.4 tokens/s avg generation throughput: 13…