Abstract: Machine Learning (ML) is becoming pervasive and integrated into different kinds of businesses. Hosting multi-tenant ML services on a unified platform requires efficient orchestration.