This repo contains the pre-release implementation of Low-rank Riemannian Optimizater (LORO), proposed in LORO: Parameter and Memory Efficient Pretraining via Low-rank Riemannian Optimization, ICLR ...