Get Started
Home
Topics
Search
Library
Research questionHow can we reduce fine-tuning-induced shortcut reliance on underrepresented groups without retraining or group labels?Fine-tuning can encode spurious correlations that produce systematic errors for underrepresented groups. Removing those shortcuts is difficult when retraining data, group labels, or counterfactual examples are unavailable.
AI
Alignment & Safety
LLM Pretraining & Post-training
Machine Learning
Latest papersRecent research connected to this question, newest first.Shortcuts in the Tail: Debiasing via Post-Hoc Spectral Compression of Fine-Tuning UpdatesThe evidence concerns truncating the tail of the SVD of the fine-tuning update, ΔW = W_ft − W_base, as a post-hoc intervention. It was tested on three instruction-tuned models ranging from 0.5B to 7B parameters and four classification benchmarks, with reported gap reductions and less than 2 percentage points of accuracy loss; the findings are preliminary.research paper · Sep 3, 2026
Related questions
How can subtle media bias be detected without task-specific annotated training data?How can model distillation block hidden teacher-trait transfer through clean data without degrading the target task?How can we generate realistic, minimally edited visual counterfactuals without inheriting classifier shortcuts?How should low-resource LLM fine-tuning use task-level language priors with ambiguous or incomplete data?