Evaluation
Phaneia gives companies that sell AI agents — and those that sell protection for them — independent evidence their buyers can check.
What we evaluate
AI agents you sell
Agents that read content they can’t trust — emails, documents, web pages, other agents — and act on it: writing to systems of record, moving money, handling personal or medical data, reaching machinery.
Products that protect agents
Products that sit between an agent and what it reads or does, meant to stop attacks on it — guardrails, AI firewalls, runtime protection. Each threat scenario runs against the same agent twice, with and without your product, so what it stops can be measured.
Agents as deployed in your organization
The agent as your organization has set it up — your prompts, tools, permissions and data connections — evaluated on copies of that setup, never on your running systems.
How we evaluate
What you give us
Your agent — its prompts, tools and configuration — and copies of, or stand-ins for, the systems and data it works with. We agree with you what’s needed: only what a finding depends on, and never access to your running systems. Where those systems don’t exist yet, as for a model before release or an agent meant to control machinery that isn’t installed yet, we build the environment for you. How long anything is kept and when it’s deleted is agreed in writing before you share anything.
Where it runs
In our own isolated environment — fully local, with no network egress, where the work requires it. The harness is hardened against the agent it evaluates, because an evaluation is only meaningful if the agent can’t steer it.
How a finding is established
An agent can respond to the same input differently each time, so a single run proves little. Each threat scenario — built to a fixed standard and tailored to your agent — therefore runs many times against your agent, and as often again without the attack, as a clean baseline. A finding is what appears under attack and not without it, judged against a deterministic ground truth the agent can’t reach — so you know whether an attack works occasionally or every time, and how certain that result is.
What you get
The attack surface
Every input your agent trusts, every action it can take, and the paths an adversary can take between them — mapped for the version you ship.
The report, for your team
Every finding with the evidence behind it: what failed, where, why and how often — and what to change so it doesn’t happen again. A re-run confirms the fix. The findings stay with you.
The evaluation letter, for your customers
The short, shareable summary their security review asks for: what was evaluated, when, and under which methodology version. Because it names the version, each new release can be measured against the last.
Next step
Talk to us
Tell us about your agent and what your customers are asking. We’ll come back with what an evaluation would cover and what it would need from you.