Research and perspectives

Perspective / Forthreason AI

Safer agents need boundaries beyond the model.

An agent can be capable without being entitled to act. That distinction is where our approach to secure AI implementation begins.

Our approach and research questions.

Capability is a research question. Authority is a design decision.

A system may be able to write a convincing message, interpret a document or prepare an update. Whether it should send that message, see that document or commit that update is a separate question. Connecting tools makes this distinction concrete.

Our starting point is the task: who requested it, which information it needs, which actions are allowed and who owns the outcome. A model instruction can describe those boundaries; the application must enforce them. OWASP identifies unnecessary functionality, permissions and autonomy as causes of excessive agency.

Separate a proposal from its consequences.

Consider a synthetic supplier-record workflow. An assistant reads an incoming request and proposes a change. A process owner reviews the exact proposed update. A separate service checks current permission before committing it. Reading, proposing, approving and executing are distinct steps.

This is a design example, not a claim that we have deployed a payment-security product. It makes the questions testable: can an old approval authorize a changed action, does revocation take effect, and can a request escape its intended scope?

Test what actually happens.

A reassuring response is not enough to establish that a prohibited action was prevented. Our proposed agent-permissions study would measure the resulting state of a controlled workflow, alongside completion of legitimate tasks. A system that rejects everything has not solved the business problem.

Prior founder work through Bento gives this agenda a practical foundation: Queenie's application-control verification covered roles and conversation ownership, while confidential gaming software explored accountable operational records. These are relevant engineering experiences, not completed agent-safety experiments.

A defensive layer that earns its place.

We want to understand which controls reduce harmful possibilities while preserving useful work. That requires bounded tests, explicit failure criteria and records another person can inspect. The ambition is a stronger defense around increasingly capable systems, supported by evidence rather than confidence alone.