Dynamically Scaled Activation Steering
Apple Machine Learning Research
Dynamically Scaled Activation Steering (DSAS) introduced a method-agnostic activation steering approach that scales interventions instead of applying them uniformly. DSAS adaptively modulates steering strength across layers and inputs based on detected undesired behavior. As a result, steering is applied strongly only when needed, aiming to avoid performance degradation when steering is unnecessary.
Why it matters
Activation steering has emerged as a powerful method for guiding the behavior of generative models towards desired outcomes such as toxicity mitigation. However, most existing methods apply interventions uniformly across all inputs, degrading model performance when steering is unnecessary. We introduce Dynamically Scaled Activation Steering (DSAS), a method-agnostic steering framework that decouples when to steer from how to steer. DSAS adaptively modulates the strength of existing steering transformations across layers and inputs, intervening strongly only when undesired behavior is detected…
