Anish Acharya, PhD

Research · 01

Recursive Self Improvement

Complete adaptation loop

My long-term goal is to develop learning systems that can adjust their policy (model weights, prompts, memory, tools, and coordination structures) while remaining faithful to their intended objectives. Such systems optimize reward models and evaluators that are necessarily incomplete; as the learner adapts, it can discover and exploit their errors. I study how reward modeling, post-training, constrained search, and validation-based acceptance can be designed together so that repeated adaptation improves true behavior rather than only increasing proxy reward.

A learning system that improves itself: evaluate, diagnose, decide, update, validate, and accept, around a memory kept across rounds. Anything the loop cannot fix goes to human review.
Scroll sideways to see the full diagram.

Key Questions

  1. How can weights, policies, prompts, tools, and coordination structures be optimized within a common adaptation loop?
  2. How should reward models be learned and updated when true utility is sparse, delayed, or unavailable, and the learner can exploit their errors?
  3. What conditions make repeated updates improve true utility without forgetting, capability collapse, or drift?

Current directions. Reward modeling, post-training and policy optimization, weight and prompt adaptation, orchestration search, and continual self improvement.