Anthropic AI Safety Fellow, London

Responsibilities:

The Anthropic Fellows Program is a 6-month external collaboration program focused on accelerating progress in AI safety research by providing promising talent with an opportunity to gain research experience. Our goal is to bridge the gap between industry engineering expertise and the research skills needed for impactful work in AI safety.

Fellows will use external infrastructure (e.g. open-source models, public APIs) to work on an empirical project aligned with our research priorities, with the goal of producing a public output (e.g. a paper submission). Fellows will receive substantial support - including mentorship from Anthropic researchers, funding, compute resources, and access to a shared workspace - enabling them to develop the skills to contribute meaningfully to critical AI safety research.

We are piloting this program with a cohort of 10-15 new collaborators. We aim to onboard our first cohort of Fellows in March 2025, with the possibility of more cohorts depending on applicant interest and logistical needs.

What To Expect

• Direct mentorship from Anthropic researchers
• Connection to the broader AI safety research community
• Weekly stipend of 1300 GBP & access to benefits
• Funding for compute and other research expenses
• Shared workspaces in Berkeley, California and London, UK

This role will be employed by our third-party talent partner, and may be eligible for benefits through the employer of record.

Mentors & Research Areas

Fellows will undergo a project selection & mentor matching process in March 2025. Potential mentors include

• Ethan Perez
• Jan Leike
• Andi Peng
• Samuel Marks
• Joe Benton
• Akbir Khan
• Fabien Roger
• Alex Tamkin
• Kyle Fish
• Nina Panickssery
• Mrinank Sharma
• Evan Hubinger

Our mentors will lead projects in select AI safety research areas, such as:

• Scalable Oversight: Developing techniques to keep highly capable models helpful and honest, even as they surpass human-level intelligence in various domains. 
• Adversarial Robustness and AI Control: Creating methods to ensure advanced AI systems remain safe and harmless in unfamiliar or adversarial scenarios.
• Model Organisms: Creating model organisms of misalignment to improve our empirical understanding of how alignment failures might arise.
• Model Internals / Interpretability: Advancing our understanding of the internal workings of large language models to enable more targeted interventions and safety measures.
• AI Welfare: Improving our understanding of potential AI welfare and developing related evaluations and mitigations.

For a full list of representative projects for each area, please see our blog post.

You may be a good fit if you:

• Are motivated by reducing catastrophic risks from advanced AI systems
• Are excited to transition into full-time empirical AI safety research and would be interested in a full-time role at Anthropic
  • Please note: We do not guarantee that we will make any full-time offers to Fellows. However, strong performance during the program may indicate that a Fellow would be a good fit here at Anthropic, and external collaborations have historically provided our teams with substantial evidence that someone might be a good hire.
• Have a strong technical background in computer science, mathematics, physics, or related fields
• Have strong programming skills, particularly in Python and machine learning frameworks
• Can work full-time on the fellowship for 6 months
• Have US or UK work authorisation, and are able to work full-time out of Berkeley or London.
  • We may be able to support Fellows based in other locations on a case-by-case basis.
• Are comfortable programming in Python
• Thrive in fast-paced, collaborative environments
• Can execute projects independently while incorporating feedback on research direction

We’re open to all experience levels and backgrounds that meet the above criteria – you do not, for example, need prior experience with AI safety or ML. We particularly encourage applications from underrepresented groups in tech.

Strong candidates may also have:

• Experience with empirical ML research projects
• Experience working with Large Language Models
• Experience in one of the research areas (e.g. Interpretability)
• Experience with deep learning frameworks and experiment management
• Track record of open-source contributions

Candidates need not have:

• 100% of the skills needed to perform the job
• Formal certifications or education credentials

Interview process:

To ensure we can start onboarding Fellows in March 2025, we will conduct interviews on a rolling basis but set hard cut-off dates for each stage. If you are not able to make that stage’s deadline, we unfortunately will not be able to proceed with your candidacy.

We aim to onboard our first cohort of Fellows in March 2025, with the possibility of more cohorts depending on Fellow interest. Please note that if you are accepted into the March cohort, we expect that you will be available for several hours of mentor matching in March, although you may start the full-time program later.

1. Initial Application and References (Complete by Jan 20, 2025)
   • Submit your application below!
   • In the application, we’ll also ask you to provide references who can speak to what it’s like to work with you.
2. Technical Assessment (Complete by Feb 3, 2025)
   • You will complete a 90-minute coding screen in Python
     • As a quick note - we know most auto-screens are pretty bad. We think this one is unusually good and for some teams, give as much signal as an interview. It’s a bunch of reasonably straightforward coding that involves refactoring and adapting to new requirements, without any highly artificial scenarios or cliched algorithms you’d gain an advantage by having memorized.
   • We'll simultaneously collect written feedback from your references during this stage.
3. 15-minute Recruiter Screen + Final Interviews (Complete by Feb 17, 2025)
   • Before the final interviews, you will chat with a recruiter about your background and logistics.
   • We will complete interviews on a rolling basis until February 10, after which we will conduct interviews at specific timeslots on pre-specified days.
   • The final interviews consist of three interviews: 
     • Take-Home Project (5 hours work period + 30 minute review) – You'll work on a research-focused project that demonstrates your technical and analytical abilities
     • Culture Interview (40 minutes) – Discussion of your values, motivations, and reasoning as they relate to your work
     • Research Discussion (15 minutes) – Brainstorming session with Ethan Perez to explore research ideas and approaches
   • In parallel, we will conduct reference calls.
4. Offer decisions (Complete by Mar 10, 2025)
   • We aim to extend all offers by March 10, and finalise our cohort shortly after.
   • We will extend offers on a rolling basis and set an offer deadline of 1 week. However, if you need more time for the offer decision, please feel free to ask for it!
   • After we select our initial cohort, we will kick off mentor matching and project selection in mid-March. This will involve several-hour project brainstorming session in the week of March 17, and follow-up discussions.

At each stage, you'll receive more detailed instructions via email. While we have hard deadlines for each stage, we will be assessing candidates and making offer decisions on a rolling basis, so we encourage you to complete each stage as soon as possible.

Please note: The salary and logistics below this section does not apply to this job posting (for example, we will pay Fellows a weekly stipend of £1300 instead of an annual salary of £1300).