Auditing Preference Biases and Fine-Tuning Language Models with Direct Preference Optimization on Anthropic HH-RLHF Using TRL and LoRA
MarkTechPost 1 month ago 11
The tutorial builds an end-to-end preference-learning workflow that fine-tunes a language model with Direct Preference Optimization (DPO) on the Anthropic HH-RLHF dataset while auditing for structural and length-based preference biases and testing for lexical shortcut signals.