Refusal in Language Models Is Mediated by a Single Direction

Researchers have discovered that language models' refusal to engage in conversation is often triggered by a single directional prompt, specifically one that begins with a negation, such as "don't" or "not," which can short-circuit the model's ability to generate coherent responses. This finding has significant implications for the development of more sophisticated conversational AI systems. The study's results challenge prevailing assumptions about the nature of language model refusal. AI-assisted, human-reviewed.