The Alignment Project是一项规模超过1500万英镑的全球性基金，致力于加速AI控制与对齐研究的进展。该项目汇聚了国际政府、产业、风险投资及慈善基金的支持。本文作为该项目一系列研究方向的介绍，重点关注了可解释性、欺骗检测、未来模型的可解释性、干预模型行为以及探究不良情境等方面。研究旨在提升AI系统的透明度、诚实性，并使其行为更可控，以应对未来AI发展带来的潜在风险。项目鼓励全球研究人员参与，共同推进AI安全。

💡 可解释性技术旨在揭示AI系统的内部运作机制，使我们能够深入了解其信息处理和决策过程。通过检查神经网络的内部表征，可以识别出诸如欺骗等隐藏属性，并能直接干预以调整模型行为，这对于理解日益复杂的AI系统至关重要。

🤥 欺骗检测的研究核心在于能否识别AI系统何时说谎，并确保针对当前模型开发的技术能有效应用于未来模型。若能确保AI系统的诚实性，将极大缓解其带来的风险，这不仅有助于衡量AI系统的倾向性，也能规避欺骗性系统带来的威胁。

🚀 针对未来模型的解释性研究，旨在克服当前技术在成本、有效性及对未来AI发展轨迹适应性方面的挑战。探索更具成本效益的解释性方法，并研究如何在新兴的AI架构（如具有潜在推理能力的模型）中应用现有技术，对于保持对AI行为的理解至关重要。

🔧 利用白盒访问对模型进行有用的干预，是直接编辑模型“信念”或引导其输出（如导向真实性）的关键。结合可扩展的可解释性策略，这些干预方法有望用于评估和缓解未来AI系统带来的风险，例如精细控制模型的拒绝倾向或诱导沙盒化模型的能力。

🔍 通过“探究不良情境”的方法，旨在确定AI系统中是否包含足够的信息来构建其可能恶意行为的情境。这有助于生成能够有效模拟部署过程的数据集，从而提高训练方法在部署时的泛化能力，并抵御对抗性优化压力。

Published on August 1, 2025 10:26 AM GMT

The Alignment Project is a global fund of over £15 million, dedicated to accelerating progress in AI control and alignment research. It is backed by an international coalition of governments, industry, venture capital and philanthropic funders. This post is part of a sequence on research areas that we are excited to fund through the The Alignment Project.

Apply now to join researchers worldwide in advancing AI safety.

Interpretability

Interpretability provides access to AI systems' internal mechanisms, offering a window into how models process information and make decisions. These tools let us examine neural representations to detect hidden properties like deception or latent capabilities, while also enabling direct intervention to modify model behaviour. As models grow more sophisticated, interpretability may be crucial for maintaining our understanding of their behaviour.

Lie Detection

Problem summary: Can we determine when AI systems are lying to us? How can we make arguments that techniques developed in one context or on current models will work on future models?

Why this matters: Many risks posed by future AI systems would be significantly mitigated if we could ensure that these systems are honest and forthcoming. Success here would contribute both to measuring AI system propensities as well as mitigating risks from deceptive systems.

Examples of subproblems:

Established subproblems:

Are there ‘universal lying mechanisms’ such that a transformative AI system could not outwardly exhibit behaviour we would characterise as deceptive without utilising these mechanisms?How might lie detectors become less effective if models are trained against them? This might be intentionally (honesty training) or unintentionally (monitor label leakage) (Cundy et al., 2025).Is it possible to disentangle representations of ‘intent to deceive’ from truthfulness, validity or credence? Are we finding latents that track ‘the model believes that a user believes x?’ when we mean to find latents representing a model’s belief?Do lie detection techniques scale well from weaker to more capable models?

New questions where we are not aware of prior work:

Is it possible to combine lie detection techniques such as activation monitoring with deliberate interrogation techniques?  This may combine benefits of both black and white box methods. Black box methods can ensure that we target specific propositions (Chalmers, 2025) while whitebox methods might work even if a model can convincingly lie.Are there trade-offs between the sophistication or robustness of deceptive behaviours and the ease by which we might detect them? For example, does reinforcement learning for secrecy as described in Marks et al., 2025 have a side effect of making white box methods more effective?Can ensembles of detectors outperform single probe based lie detection strategies?

Suggestions to learn more: We’d recommend reading Goldowsky-Dill et al., 2025, Marks et al., 2023 and Azaria et al., 2023 for context on simple approaches to lie detection with linear probes. Chowdury et al., 2025 may also be useful for further context on truthfulness in frontier AI systems. Farquhar et al., 2023 provides some insights into the limitations of prior unsupervised LLM knowledge discovery techniques. 

Interpretability for Future Models

Problem summary: Some novel interpretability strategies that require significant resource investments may substantially advance our insight into neural network internals (Bricken et al., 2023; Templeton et al., 2024; Lindsey et al., 2025). There are many opportunities to advance this area of interpretability. 

Why this matters: Current interpretability techniques may not work for future models. Possible challenges include the cost of current techniques, known issues with current methods and unclear robustness to future AI development trajectories which may involve superhuman AI. 

Examples of subproblems:

Established subproblems:

Improving the training efficiency of recently developed techniques like sparse autoencoders. Are there ways to get similar levels of value with a fraction of the cost or effort?Current sparse decomposition techniques have known issues which may prevent us from realising their value from AI safety. For example, they may learn classifiers with ‘holes’ in them due to the sparsity objective (Chanin et al., 2025)Can the use of computer-use agents to perform behavioural audits (mentioned in Anthropic's Claude 4 system card) be extended to ‘interpretability agents’ that can not only locate but diagnose and debug anomalous behaviours? 

New questions where we are not aware of prior work:

Demonstrating usefulness of sparse network decomposition (Bricken et al., 2023) in the context of super-intelligent capabilities. Future AI systems may have alien concepts that are far from interpretable to humans (Hewitt et al., 2025). Stress-testing features and circuit finding in domains like superhuman chess or biological foundation models (Simon et al., 2024) may teach us how to continue to get value out of these techniques.Demonstrating how sparse decomposition techniques might work even in models with extensive latent reasoning. Feature characterisation currently relies heavily on token-highlighting, but if future models reason in latent space (Hao et al., 2024, Geiping et al., 2025) then it’s unclear whether we could continue to use the same approaches. 

Leveraging White-Box Access for Useful Interventions

Problem summary: Although some early studies hint at methods that could intervene in a model’s internal representations during inference, none have yet been shown to be a reliable, central technique (Abdelnabi et al., 2025, Turner et al., 2023, Li et al., 2023). 

Why this matters: Combined with scalable interpretability strategies for tracking run-time information, the possible applications of being able to directly edit model ‘beliefs’ or steer completions (such as towards truthfulness) could be used for both estimating and mitigating risks posed by future AI systems. 

Examples of subproblems:

Established subproblems:

Can we detect and prevent evaluation awareness in AI systems (Abdelnabi et al., 2025)?Can we exercise fine-grained control over refusal tendencies (Arditi et al., 2024, Lee et al., 2024)? For example, can we use this to prevent residual refusal tendencies in models that have had their harmfulness training removed to enable CBRN dangerous capability evaluations (Anthropic, 2025)?Can activation steering be used to elicit model capabilities in sandbagging models (van der Weij et al., 2025, Hofstatter et al., 2025)?

New questions where we are not aware of prior work:

We’d like to see more exploration of methods of optimising text to minimise the activations of latents (Thompson et al., 2024), such as those tracking evaluation awareness, as an alternative to activation steering. For example, we could use this to control what information is ‘leaked’ to AI systems through prompts.We’re interested in whether we can better test AI systems for undesirable propensities by directly modifying their beliefs. Such techniques might enable us to probe for behaviours that are usually very difficult to elicit via prompting or other ‘honey-potting’ strategies. For example, there is some prior work intervening on a latent representation for test awareness (Abdelnabi et al., 2025). 

Interpretability Approaches to Eliciting Bad Contexts

Problem summary: We like to determine whether sufficient information is contained within AI systems to construct contexts in which they might behave maliciously (the problem of 'eliciting bad contexts'; Irving et al., 2025).

Why this matters: In theory, a solution to eliciting bad contexts might be useful for generating datasets that mimic deployment sufficiently effectively that we can minimise distribution shift from training to deployment. This means that training methods to produce aligned systems are more likely to generalise to deployment and withstand adversarial optimisation pressure to move away from the aligned state via prompting.

Examples of subproblems:

Are there approaches for mechanistically eliciting latent behaviours based on a single prompt (Mack and Turner, 2024)? How could we use these approaches to locate latent representations and find bad contexts?Can the method in Thompson et al., 2024 for optimising an input to maximally activate a latent representation be extended to SAE latents or steering vectors? Showing these techniques can be applied whilst preserving other properties of a prompt may be useful. For example, can we optimise an eval question to preserve the content of the question but against the model being aware that the question comes from an evaluation?

Suggestions to learn more: We’d recommend reading Irving et al., 2025.

 

View The Alignment Project website to learn more or apply for funding here.

Discuss