Abstract: Recently, AI inference services have adopted ensemble architectures, which are widely recognized and used for their advanced performance. However, the existing ensemble inference services ...