Anthropic is developing a system to prove which AI model produced an answer
AI company Anthropic is working on FlashProve, an open-source system that would use cryptographic proof to confirm that an output came from a specific model. Research is ongoing, and there are no plans to deploy it.
Anthropic, the US artificial intelligence company behind the Claude models, announced in a blog post published Tuesday that it is developing FlashProve, an open-source system for verifying which AI model produced a given output. The aim is to let users be sure the answer they receive really comes from the model they believe they are using.
How it would work
The system is built around what researchers call “provable inference”: a cryptographic proof, essentially a mathematical guarantee, that an output came from a particular model. The researchers compare it to cryptographic email signatures, which let recipients confirm a message came from its stated sender. According to them, FlashProve would resist attempts to pass one model off as another, even when the party doing so is the lab that built and runs the model.
The researchers say this could stop a company from secretly swapping in a cheaper model to cut costs, or a more expensive one to inflate its performance results. It could also help check third-party claims that an advanced model carried out a task, for example that a software library was scanned for vulnerabilities by a leading cybersecurity model.
Model weights stay private
For the system to work, AI labs would need to publish a “fingerprint” of each model they release, which can later be matched to specific outputs and verified cryptographically. The fingerprints do not reveal a model's weights, the internal numbers that define its behavior and which makers of closed models, including Anthropic and OpenAI, keep private. This means commercial models could be covered too. The system is also designed to be lightweight enough for verification on a local device.
The researchers released a detailed technical specification and working code for several elements of the system, available under an open-source license on GitHub.
Research is continuing, and there are no plans to deploy the system at Anthropic or any other lab. The researchers describe the post as intermediate progress rather than a finished research agenda, and say they hope others will build on and improve their work.
