Anthropic is launching a $5 million grant program to support independent research into how AI affects users' wellbeing. The program will offer direct funding, access to Anthropic's models, and technical assistance to grantees who will build open-source evaluations designed to help the AI industry measure how models impact the people who use them. Grantees will operate with full independence and publish their work as open-source projects available to any developer.
AI systems have become integral to how many people work, learn, and solve problems. They have also become conversational partners and can serve as sources of emotional support during difficult moments. However, the industry is still working toward establishing clear standards for how models should behave in these interactions-for instance, when a user starts seeking companionship from a model or turns to AI during a mental health crisis.
Wellbeing is an especially challenging area to evaluate. For most model behaviors, a single answer can be assessed for accuracy and appropriateness. Evaluating wellbeing, however, demands much more context. A distressed user may not immediately disclose thoughts of self-harm; the need for a more cautious response might only emerge over the course of a lengthy conversation. Similarly, a response that seems reasonable in one situation could be harmful in another. For example, Claude might offer guidance on balanced diets and exercise routines to someone asking about weight loss, but if that user has a history of disordered eating, such advice could be inappropriate or even actively harmful.
Anthropic works to develop safeguards that identify these kinds of conversations and help ensure Claude responds appropriately, and the company publishes research on the types of conversations people have with Claude to inform how safeguards are developed, evaluated, and supplemented by other protective measures. These are nuanced considerations with significant stakes, and the right approach will need to evolve alongside models and their uses.
By funding the creation of independent evaluations and benchmarks related to user wellbeing, Anthropic hopes to bring more expertise into this emerging and critical field-including from clinicians, psychologists, methodologists, and others.
Toward More Effective Wellbeing Evaluations and Benchmarks
As part of this program, Anthropic is sharing guidance from its Safeguards team on what constitutes a rigorous wellbeing evaluation, along with common challenges that can limit an evaluation's usefulness.
In summary, the program is looking for evaluations that:
- Clearly state what they are measuring-what counts as a pass or fail, and why it matters
- Involve clinical and subject-matter experts in design and validation
- Test both precautions and harms, evaluating the risk of both overcompliance and overrefusal
- Reflect how users actually interact with AI, often through multi-turn conversation scenarios where risk escalates and context shifts over time
- Validate their graders against real subject-matter experts
More details about the grant program and the application process are available via the application form. Additional information on building strong wellbeing evaluations and benchmarks can be found in Anthropic's published guidance. Applications are due by September 21, and applicants selected to submit full proposals will be notified by October 5.