On the Benefits of Multiple Gossip Steps in Communication Constrained Federated Learning

A Hashemi; Anish Acharya; R Das; H Vikalo; S Sanghavi; I Dhillon

On the Benefits of Multiple Gossip Steps in Communication Constrained Federated Learning

A Hashemi, Anish Acharya, R Das, H Vikalo, S Sanghavi, I Dhillon

IEEE Transactions on Parallel and Distributed Systems (IEEE TPDS), 2021

Shows that several gossip steps between gradient updates improve decentralized learning under lossy compressed communication, even after paying for the extra communication.

IEEE Xplore Code arXiv

All publications · Research