From Preferences to Principles: Rubric-Based Alignment for Grounded Knowledge Answers

Apple Machine Learning Research · 1 month ago · 46

The authors propose a rubric-based reward framework that generates query-specific, evidence-grounded rubrics and uses them for post-training to supervise open-domain question answering. Averaged across three evaluation axes (composition, grounding, and instruction-following), it improves 6.5% over an instruction-tuned baseline. The approach changes training by replacing a single scalar reward with multi-dimensional rubrics conditioned on retrieved evidence, yielding more factual support and better coherence and instruction adherence.