Robust & Weakly Supervised Learning

Inference from imperfect supervision

This thread studies what can be learned when supervision is corrupted, incomplete, indirect, synthetic, or generated by another model. Robust learning and weak supervision represent different failure modes: in one the signal exists but may be corrupted; in the other the desired signal is only partially observed. My long-term goal is to develop statistically principled and computationally practical methods that distinguish ordinary noise, structured failure, missing information, and valid disagreement across data selection, training, reward modeling, and evaluation.

A cluster of inliers, only some of them labeled positive, with an adversarial tail of corrupted points. As the corruption arrives, the mean is dragged toward the tail while the geometric median stays with the inliers. Supervision may be corrupted, incomplete, indirect, or model-generated; robustness here means a high breakdown point, estimating only what is identifiable, never treating unlabeled data as negative, and keeping valid minority evidence. — Scroll sideways to see the full diagram.

Key Questions

What remains identifiable under corrupted, incomplete, or model-generated supervision?
Can practical high-dimensional methods attain optimal robustness without manufacturing missing labels?
How should a learner combine supervision sources with different reliability without suppressing valid minority evidence?

Selected Publications

RoPoLL: Robust Panel of LLM Judges

ICML 2026 · AgenticUQ Workshop

Shows that averaging an LLM jury is unboundedly biased once any judge fails systematically, and replaces the average with a geometric-median aggregator that has the optimal 1/2 breakdown point. A 3-judge 38B panel beats a 675B judge by 1.31x under 30% corruption.
Geometric Median Matching for Robust k-Subset Selection from Noisy Data

ICML 2025

Selects a k-subset whose mean tracks the geometric median of noisy data, giving an O(1/k) rate under arbitrary corruption, a quadratic improvement over random sampling.
BGMD: Robust Training in High Dimensions via Block Coordinate Geometric Median Descent

AISTATS 2022 · Spotlight

Makes geometric-median SGD practical in high dimensions by applying it to one chosen block of coordinates at a time, keeping the optimal 1/2 breakdown point for smooth non-convex problems.
Positive Unlabeled Contrastive Learning

ICML 2022 · PODS Workshop

Extends contrastive pretraining to positive-unlabeled data with an unbiased, lower-variance loss that uses the few labeled positives.