Distributed & Collaborative Learning

Collective optimization and coordination

Many learning systems consist of participants that hold different data, possess different capabilities, and communicate through a limited or changing topology. My long-term goal is to develop principles for collaborative learning across federated clients, decentralized workers, distributed encoders, autonomous agents, and physical robots. I study how local updates should be aggregated, what information must be communicated, how coordination topology affects convergence, and how collective learning can remain stable under heterogeneity, failures, and adversarial participants.

Six learners, each holding a different local data distribution, send compressed updates to a robust aggregator and gossip with their neighbours. One learner sends a corrupted update, which the robust median down-weights. Each learner has local data, local compute, a communication budget, and a sparse set of neighbours; client drift, failures, adversarial participants, and stale information are what can go wrong. — Scroll sideways to see the full diagram.

Key Questions

When can decentralized learners or robot fleets match centralized training under limited communication?
How should aggregation, communication topology, and local computation adapt to heterogeneous or unreliable participants?
Which principles from gossip and federated optimization extend to multi-agent coordination and collaborative policy learning?

Selected Publications

OrchOpt: Validation-Gated Architecture Search for Self-Organizing Multi-Agent Orchestration

COLM 2026 · Lifelong Agents Workshop

Searches over multi-agent orchestration architectures and keeps a change only when it passes validation.
Neural Distributed Source Coding

IEEE JSAIT 2024

Learns lossy distributed source coding with a conditional VQ-VAE, handling arbitrary correlation structure in high dimensions with state-of-the-art PSNR.
Faster Non-Convex Federated Learning via Global and Local Momentum

UAI 2022 · Spotlight

Combines variance-reduced momentum at the server and at the clients to speed up non-convex federated learning under client heterogeneity and compressed communication.
BGMD: Robust Training in High Dimensions via Block Coordinate Geometric Median Descent

AISTATS 2022 · Spotlight

Makes geometric-median SGD practical in high dimensions by applying it to one chosen block of coordinates at a time, keeping the optimal 1/2 breakdown point for smooth non-convex problems.
On the Benefits of Multiple Gossip Steps in Communication Constrained Federated Learning

IEEE TPDS 2021

Shows that several gossip steps between gradient updates improve decentralized learning under lossy compressed communication, even after paying for the extra communication.