AI Lab Watch

AI Lab Watch

Categories Companies Resources Blog About

			*
Weighted score	27%	21%	17%	5%	3%	3%	1%
Risk assessment	44%	29%	32%	1%	1%	1%	0%	27% weight
Scheming risk prevention	3%	8%	2%	2%	2%	2%	2%	21% weight
Boosting safety research	68%	56%	37%	28%	0%	15%	8%	14% weight
Misuse prevention	12%	4%	5%	0%	0%	0%	0%	12% weight
Prep for extreme security	3%	5%	0%	0%	0%	0%	0%	12% weight
Risk info sharing	35%	13%	32%	0%	28%	0%	0%	8% weight
Planning	14%	26%	0%	0%	0%	1%	0%	6% weight

Up to date as of July 13

Overall score

Anthropic

27

%

DeepMind

21

%

OpenAI

17

%

Meta

5

%

xAI

3

%

Microsoft

3

%

DeepSeek

1

%

Risk assessment

AI companies should do model evals and uplift experiments to determine whether models have dangerous capabilities or how close they are. They should also prepare to check whether models will act well in high-stakes situations.

44<br><br>%	29<br><br>%	32<br><br>%	1<br><br>%	1<br><br>%	1<br><br>%	0<br><br>%

Scheming risk prevention

AIs show signs that if they were more capable, they would sometimes scheme, i.e. fake alignment and subvert safety measures in order to gain power. AI companies should prepare for risks from models scheming, especially during internal deployment: if they can’t reliably prevent scheming, they should prepare to catch some schemers and deploy potentially scheming models safely.

3<br><br>%	8<br><br>%	2<br><br>%	2<br><br>%	2<br><br>%	2<br><br>%	2<br><br>%

Boosting safety research

AI companies should do (extreme-risk-focused) safety research, and they should publish it to boost safety at other AI companies. Additionally, they should assist external safety researchers by sharing deep model access and mentoring.

68<br><br>%	56<br><br>%	37<br><br>%	28<br><br>%	0<br><br>%	15<br><br>%	8<br><br>%

Misuse prevention

AI companies should prepare to prevent catastrophic misuse for deployments via API, once models are capable of enabling catastrophic harm.

12<br><br>%	4<br><br>%	5<br><br>%	0<br><br>%	0<br><br>%	0<br><br>%	0<br><br>%

Prep for extreme security

AI companies should prepare to protect model weights and code by the time AI massively boosts R&D, even from top-priority operations by the top cyber-capable institutions.

3<br><br>%	5<br><br>%	0<br><br>%	0<br><br>%	0<br><br>%	0<br><br>%	0<br><br>%

Risk info sharing

AI companies should share information on incidents, risks, and capabilities — but not share some capabilities research.

35<br><br>%	13<br><br>%	32<br><br>%	0<br><br>%	28<br><br>%	0<br><br>%	0<br><br>%

Planning

AI companies should plan for the possibility that dangerous capabilities appear soon and safety isn’t easy: both for evaluating and improving safety of their systems and for using their systems to make the world safer.

14<br><br>%	26<br><br>%	0<br><br>%	0<br><br>%	0<br><br>%	1<br><br>%	0<br><br>%

I’m Zach Stein-Perlman. I’m worried about future powerful Als causing an existential catastrophe. Here at AI Lab Watch, I track what AI companies are doing in terms of safety.

In this scorecard, I collect actions AI companies can take to improve safety and public information on what they’re doing.

Click on a column or cell to see details about a company; click on a row to see details about a category.

Criteria are grouped into categories; both are weighted by how important they currently are for safety and how much signal the criteria provide/capture. These criteria are not exhaustive; some important variables are hard to measure. I endorse the words more than the numbers; the scores on particular criteria and the weights are largely judgment calls.

In addition to the scorecard, I write blogposts on what AI companies should do and what they are doing, and I maintain resources with information on these topics.

I’m Zach Stein-Perlman. I’m worried about future powerful Als causing an existential catastrophe. I track what AI companies are doing in terms of safety.

For some details on what AI companies should do and what they are doing in terms of safety, click around this scorecard. Or check out the articles below, the rest of my blog, or the resources I maintain.

Featured posts

What AI companies should do → AI companies’ eval reports mostly don’t support their claims → AI companies’ commitments → AI companies’ integrity incidents →

Subscribe to the blog

Direct Link