DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
arXiv:2604.17709v1 Announce Type: new
Abstract: Existing works on large language model (LLM) decomposition mainly focus on improving performance on downstream tasks, but they ignore the poor parallel inference performance when trying to scale up the m…