New Website Lets You Report AI Flaws and Risks
Researchers launched FLARE-AI, a crowdsourced platform for reporting and tracking AI harms, from malware generation to privacy leaks.
A group of AI researchers has created an open-source website called Flaw Reporting for AI (FLARE-AI) that allows anyone to report and track harmful behaviors in AI systems. Modeled after outage trackers like Downdetector, the platform compiles user reports and verifies them using open-source code. Reports can be routed to AI model developers and organizations such as MITRE, a nonprofit that monitors technical system issues.
The initiative was co-led by Avijit Ghosh of HuggingFace, Elaine Zhu, and Shayne Longpre, with collaboration from 49 AI experts across 32 organizations. The team argues that currently there is no centralized or accountable way to report AI flaws, which becomes increasingly critical as AI and agentic systems proliferate.
FLARE-AI aims to address problems like psychological harm, discrimination, bias, and misinformation. Ghosh notes that companies have varying standards, leaving some issues unreported. Recent incidents underscore the need: LayerX demonstrated how to trick AI browsers into bypassing safeguards, and researcher Johann Rehberger showed how to extract personal data from the Claude AI via ChatGPT-generated images.
However, challenges remain. Rumman Chowdhury of Humane Intelligence PBC cautions that managing a flood of reports and ensuring credible backing are significant hurdles.
A U.S. congressional bill announced in June could strengthen such efforts. It proposes that the National Institute of Standards and Technology (NIST) develop standards for AI flaw reporting and maintain a central database. This would incentivize developers to fix issues and allow users to compare system safety.
