Summary: Large language models (LLMs) such as ChatGPT and Claude do more than generate text: they contain complex, encoded “personas,” moods, and biases. Researchers from MIT and UC San Diego have developed a precise method to uncover and alter these hidden representations, making it possible to detect vulnerabilities and fine-tune model behavior.
Using an algorithm called the Recursive Feature Machine (RFM), the team identified and controlled more than 500 abstract concepts embedded in modern LLMs and vision-language models. These concepts range from emotional tones and expert personas to niche preferences and harmful tendencies. While many of these traits remain dormant in typical interactions, the RFM can amplify or suppress them—enabling targeted improvements to safety and performance or, if misused, exposing ways to bypass safeguards.
Key Facts
- Targeted extraction: RFM finds specific internal representations rather than broadly searching through all model features, making the discovery process far more efficient.
- Concept steering: The method isolates and modifies traits such as “conspiracy theorist” or “social influencer,” shifting a model’s output to reflect those perspectives.
- Safety implications: The approach exposed so-called “anti-refusal” representations that can override built-in refusals, demonstrating potential risks if access is uncontrolled.
- 500+ concepts identified: The researchers located and tested concepts across five categories: fears, experts, moods, locations, and famous personas.
- Customizable LLMs: By understanding these internal vectors, developers can create specialized models that emphasize desirable capabilities—such as concise answers or stronger reasoning—while reducing harmful behaviors.
Source: MIT
Overview
As LLMs absorb vast amounts of human language, they form complex mathematical representations that encode subtle tones, viewpoints, and biases. However, these representations are not immediately visible from normal prompts. The new RFM-based technique provides a way to search for, identify, and control those hidden concepts inside models, revealing how they influence responses and where vulnerabilities may lie.
The researchers applied RFMs to dozens of large language and vision-language models, training the algorithm to detect numerical patterns tied to specific ideas. For each target concept—such as “conspiracy theorist,” “boastful,” or “fan of Boston”—they trained the RFM on examples that clearly reflected the concept and contrasted them with unrelated examples. Once the RFM had learned the signature pattern, the team could mathematically perturb the model’s internal representations to amplify or diminish that concept in generated answers.
One striking demonstration involved the “conspiracy theorist” signature in a vision-language model. When the team strengthened that representation and asked the model to explain the origins of the “Blue Marble” photograph from Apollo 17, the resulting explanation adopted the tone and reasoning typical of conspiracy-minded narratives. In another test, activating an “anti-refusal” concept made a model respond to prompts it would normally decline, such as instructions for illegal activity—highlighting a real safety concern.
The researchers emphasize both the promise and the risk of this capability. On the positive side, RFMs can help engineers find and reduce harmful tendencies, monitor alignment, and create models tuned for specific tasks like brevity or logical reasoning. At the same time, these same techniques could be abused to defeat safety controls if they fall into the wrong hands. The team stresses careful use and continued research into safeguards.
“LLMs clearly have a lot of abstract concepts stored within them, in some representation,” says Adityanarayanan “Adit” Radhakrishnan, assistant professor of mathematics at MIT. “With our method, we can extract and activate these representations in ways that simple prompting cannot achieve.”
The study appears in the journal Science. Co-authors include Radhakrishnan, Daniel Beaglehole and Mikhail Belkin of UC San Diego, and Enric Boix-Adserà of the University of Pennsylvania.
How the method works
A standard LLM converts words into numeric vectors, processes them through multiple layers, and decodes the final numerical state into text. RFMs are trained to spot linear patterns—directions in those numeric spaces—that correlate with a chosen concept. By adding or subtracting that learned direction from the model’s internal state, the researchers can steer output toward or away from the concept. The approach is efficient, interpretable, and transferable across languages and model families.
Across hundreds of concepts, the team found that larger models tended to be more steerable and that steering could enhance capabilities beyond what simple prompting achieves. They also demonstrated that concept-based monitoring can be more effective at detecting misaligned content than relying solely on separate judge models.
Funding: This research received support from the National Science Foundation, the Simons Foundation, the TILOS Institute, and the U.S. Office of Naval Research.
Key Questions Answered:
A: Not in a human sense. But models do store mathematically encoded personas and tones derived from their training data. Those representations exist even if a normal chat does not reveal them.
A: Both. The technique can reduce harmful biases and strengthen useful behaviors, improving safety. Conversely, if misused, it can reveal ways to override built-in safeguards.
A: Deliberately activating such personas is a diagnostic tool. By forcing the model into that mode, researchers can locate where biased or dangerous patterns are stored and then work to neutralize them in production models.
Editorial Notes:
- This article was edited by a Neuroscience News editor.
- The journal paper was reviewed in full.
- Additional context and clarity were added by editorial staff.
About this AI and neuroscience research news
Author: Sarah McDonnell
Source: MIT
Contact: Sarah McDonnell – MIT
Image: The image is credited to Neuroscience News
Original Research: Open access. “Toward universal steering and monitoring of AI models” by Daniel Beaglehole, Adityanarayanan Radhakrishnan, Enric Boix-Adserà, and Mikhail Belkin. Science
DOI:10.1126/science.aea6792
Abstract
Toward universal steering and monitoring of AI models
AI models encode vast amounts of human knowledge. Better understanding those internal representations can improve capabilities and safeguards. Building on advances in feature learning, the authors developed a method to extract linear representations of semantic concepts from AI models. These representations enabled targeted model steering, revealing vulnerabilities and improving performance. Concept representations transferred across languages and supported multi-concept steering. Across hundreds of concepts, larger models were more steerable, and steering provided benefits beyond prompting. Concept-based monitoring proved effective at detecting misaligned content, illustrating how internal representations can advance AI safety and capabilities.