# AI penetration testing: how our agents test web apps and APIs, with human review

Source: https://penetrationtestservices.com/ai-penetration-testing/
Publisher: Excession Labs (penetrationtestservices.com)
Last updated: 2026-11-09

How Excession Labs uses AI agents in a web app or API pentest: planner and workers, allow, ask and deny tool rules, the evidence rule and human review.

> **Short answer:** AI penetration testing is a pentest of a web app or API in which AI agents run much of the testing work. At Excession Labs, a security lab, a planner splits the scope into tasks, workers run them in parallel as each user role, and every tool has an allow, ask or deny rule. A finding needs linked evidence. Excession Labs agrees the scope in writing, checks the evidence and writes the report.

AI penetration testing is a pentest of a web application or API in which AI agents run much of the testing work: mapping routes, sending requests as each user role, comparing responses and collecting evidence. Excession Labs, a security lab, uses its AI testing agents this way in its web app and API tests. People keep the decisions that carry risk. Excession Labs agrees the scope with you in writing, sets the tool rules, checks the evidence for each finding and writes the report.

Scope of this page: AI used to test your web app and API. To have an AI application assessed (LLM apps, RAG, agents, MCP tools), see [AI red teaming services](https://airedteamingservices.com/).

## What kinds of AI penetration testing are there?

The label covers three different things. They differ in who chooses the next test and who decides that a result is real.

| Kind | Who chooses the next test | Who judges the result | Main risk |
|---|---|---|---|
| Scanner with AI features | A fixed check list. A model ranks, groups or explains the alerts | The tool, with a confidence score | Misses flows that need state or a signed-in session |
| Agentic tester | An AI agent, from what it saw in earlier responses | The agent, then a person in a controlled design | Claims an effect it did not prove |
| Human tester with AI tools | The tester | The tester | Limited by the tester's time |

OWASP describes dynamic scanners as black-box tools that inject payloads into a running application ([OWASP DevSecOps Guideline](https://owasp.github.io/www-project-devsecops-guideline/latest/02b-Dynamic-Application-Security-Testing)). The scan still follows a defined set of checks when a model is added. Our service is the second kind, with a person checking every finding. [AI pentest tools](/guides/ai-pentest-tools/) sorts the market into categories, and [automated penetration testing](/guides/automated-penetration-testing/) places each one on a ladder from scanner to human-led test.

## How does an AI-assisted pentest run?

A test runs in nine steps. Nothing runs against your system until the system owner authorizes the test in writing.

| Step | What happens | Who acts |
|---|---|---|
| 1. Scoping call | We learn which apps, APIs and roles matter and what the test is for | Excession Labs and you |
| 2. Written scope and authorization | The scope names hosts, API base paths, environments, roles, bounded tests and what we will not touch | Excession Labs writes, the system owner signs |
| 3. Access setup | Test accounts for each role, two in the same role for read-back, the environment and a stop contact | You |
| 4. Planning | A planner turns the scope into objectives and tasks | AI agent (coordinator) |
| 5. Parallel testing | Workers run tasks at the same time under the tool rules | AI agents, with a person approving "ask" actions |
| 6. Evidence-linked findings | A task is done only when each finding links to its evidence | AI agents |
| 7. Evidence check | Each finding is kept, downgraded or dropped | Excession Labs |
| 8. Report | HTML, PDF and Markdown, with proof-of-concept scripts | Excession Labs |
| 9. Fixes and retest | The original proof runs again against your fix | You fix, Excession Labs can retest |

The [sample report](/sample-report/) shows the report from step 8 on a fictional app.

## How do the planner, workers and tool rules work?

The planner (coordinator) splits the engagement into tasks such as "map the API", "compare what each role can read" or "test the upload endpoint within the stated bound". A known test structure, such as the areas in the [OWASP Web Security Testing Guide](https://wstg.owasp.org/), helps the plan avoid gaps. If a task needs a target outside the written scope, it does not run.

Workers take the tasks and run them in parallel. Each worker runs a loop: decide the next step, call a tool, read the result. One worker might compare what an anonymous caller and a signed-in caller receive while another reads the JavaScript bundles for routes. Each worker keeps its own notes and findings, so a reviewer can follow how a finding came about.

A tool is any action an agent can call: an HTTP client, a browser, a code search, a script runner. Each tool has one rule, and Excession Labs sets the rules for each scope.

| Rule | What it means | Example |
|---|---|---|
| Allow | The tool runs on its own | Read-only requests to in-scope hosts |
| Ask | A person approves each use | A state-changing request on a test-owned record |
| Deny | The tool is blocked | Any host outside the written scope |

The rule is attached to the tool, so the model cannot talk itself past it. Text in your app's pages or API responses is untrusted input, and it can try to steer an agent. Deny rules for out-of-scope hosts and approval for state-changing calls do not depend on the model resisting that text. The full rule table and the stop conditions are in our [methodology](/methodology/).

## Where do people review the work?

People act at five points. The agents do the repetitive testing between them.

1. Scope and authorization. The system owner authorizes the scope in writing before any test.
2. Rule review. Before the run, a person checks the allow, ask and deny rules against the scope.
3. Live approvals. During the run, a person approves or rejects each "ask" call.
4. Evidence review. Excession Labs checks the evidence for each finding and writes the report.
5. Report review and retest. You read the report. We can retest fixed findings.

Each approval request shows the exact request, its limits and its side effect, so the person can judge it:

> **Sample approval request:** ```text
> ASK: http_request
> Target:   https://staging.harbor.example.com/api/notes/import   (in scope)
> Method:   POST, 25 requests, 1 per second
> Body:     test-owned CSV, 10 KB, marked "TEST DATA"
> Purpose:  check that import enforces the 1 MB size limit
> Side effect: creates up to 25 notes in account test-a
> Cleanup:  delete the notes after the check
> Approve / Reject / Edit limits
> ```
Too many prompts cause approval fatigue: people stop reading them. We keep safe, repeated reads on "allow" and group related calls into one approval with a stated cap.

## How does a candidate become a finding?

A candidate is something an agent suspects. It becomes a finding only when it has evidence of a real effect and the controls behave as expected. A write counts when the changed record is read back through another session. A bare "success" response is not proof.

> **Demonstration:** This is a demonstration on Pinecart, a fictional e-commerce API. No real customer system was tested. The identifiers are invented.
| What the agent claimed | Check | Verdict |
|---|---|---|
| buyer_A reads `/v2/orders/ord_test_B1` and gets 200 | A nonexistent order returns 404. buyer_B's own session shows the same order and total | Confirmed, High |
| `/v2/me/orders?user=buyer_B` returns 200 | The response holds buyer_A's own orders. The API ignores the parameter | Dropped |
| The refund endpoint accepts a negative amount | State-changing request, so the rule was "ask". Approved by Excession Labs and run on a test order | Confirmed, Medium |
| Password guessing on `/login` | The rule was "deny" for this scope. Not run | Out of scope |

The second row is the common failure of a language model: a 200 response read as a breach when the server ignored a parameter. The controls caught it, so it never reached the report. The [methodology](/methodology/) walks through a kept candidate and a dropped one request by request.

## Which tasks do agents lead, and which need a person?

Agents lead on work that is wide, repeated or comparative. A person leads wherever "wrong" depends on how your business should work.

| Task type | What the agents do | Where a person decides |
|---|---|---|
| Broad coverage of many endpoints | The same checks on every route, without tiring | Which routes matter most to your business |
| Differential checks across roles | The same request as anonymous, user, admin and a second account | What each role should be allowed to do (you supply the access model) |
| Cross-account authorization (IDOR) | Read-back of a changed record from a second session | Whether a shared record is meant to be shared |
| Client-side JavaScript review | Fast lists of candidate routes and keys | Which candidates are real exposure |
| Retesting fixed findings | Replays the original proof and its controls | Whether the fix matches the intended design |
| Business logic (price, quota, workflow order) | Limited without your rules written down | Needed |
| New multi-step chains | Can combine known steps | Needed for a new chain |
| Severity and risk acceptance | Can propose | Needed: your team decides what risk to accept |

Social engineering, phishing and physical tests are outside this service. If your main risk sits in custom rules for money movement, approvals or entitlements, tell us in the scoping call and give us the intended rules for each role. The written scope states what the test covers and what it does not. [Where AI pentesting falls short](/guides/where-ai-pentesting-falls-short/) lists each limit and how we handle it.

## Is an agent just a scanner with a chat window?

It is, if the tool replays a fixed check list and a model only words the report. Three tests tell the two apart. Ask any provider, including us, for evidence of each.

| Test | Scanner with a chat window | Agentic tester |
|---|---|---|
| Does the next request depend on the last response? | No. The check list sets the order | Yes. The agent reads the response, then picks the next request |
| Is an effect proven by read-back? | Usually a status code or a response pattern | A changed record is read back through a second account |
| Do controls run next to each exploit request? | Rarely | Yes: a nonexistent ID, no token, another account's token |

If a tool fails the second and third tests, treat its findings as leads. Our [DAST guide](/guides/dast-dynamic-application-security-testing/) compares scanners and agents in more depth.

## What does research say about AI agents in pentests?

A December 2025 arXiv study put ten cybersecurity professionals, six existing AI agents and a new agent called ARTEMIS against a university network of about 8,000 hosts across 12 subnets. ARTEMIS found 9 valid vulnerabilities with an 82% valid submission rate and outperformed 9 of the 10 humans. The authors report that agents did well at systematic enumeration and parallel work, but had higher false-positive rates and struggled with GUI-based tasks. In one case an agent reported a default-credentials login after "200 OK" responses that were redirects to the login page ([Lin et al., arXiv 2512.09882](https://arxiv.org/abs/2512.09882)).

A second paper found that LLM agents struggle to keep coherent long-horizon plans and added classical planning to compensate ([arXiv 2512.11143](https://arxiv.org/abs/2512.11143)). Both papers studied network or lab targets, not web application authorization, so they do not predict results on your app. They support two choices in our design: a planner that holds the plan outside a single model call, and an evidence check against false positives. We publish no benchmark scores for our agents. The agents do not find every vulnerability, and a system is not "secure" after a test.

## What should you ask any AI pentest provider?

Ask these four questions before you compare prices or results:

1. Who authorizes the scope, and how is it written?
2. Which actions run without approval, which wait for approval, and which are blocked?
3. What proves a finding: a status code, or a read-back of the changed state?
4. Who reviewed each finding before it reached me?

A benchmark score answers none of them. To check a report you already have, use the scorecard in [how to evaluate an AI pentest report](/guides/evaluating-an-ai-pentest-report/). To compare providers of AI pentests, see [AI penetration testing companies on pentestcompare.com](https://pentestcompare.com/lists/ai-penetration-testing-companies/), a comparison that Excession Labs publishes and discloses. To see the method on your own app first, apply for the [free pentest pilot](/free-pentest/).

## Questions

### Is this the same as testing an AI chatbot?

No. This page is about AI agents that test ordinary web applications and APIs. To have your own LLM application, RAG system, agent or MCP tool assessed for prompt injection and data leaks, see airedteamingservices.com.

### Does AI penetration testing need my source code?

No. The tests are black-box or grey-box. The agents work through your running app and API as each role you provide, and they read the client-side JavaScript bundles your app ships. Code review is a separate kind of engagement.

### Do I need to give the agents production access?

No. You choose the environment during access setup. Staging with test accounts carries the lower risk. If production is in scope, the written scope says so, and stricter approval rules and tighter request rates apply.

### Can I see the agents' work?

You see the evidence for each finding: saved requests and responses with timestamps, the controls that ran and the steps to reproduce. The report has an evidence index, and key evidence can carry a SHA-256 manifest.

### Can I use the report for an audit?

Ask your auditor. We cannot say that any auditor, customer or regulator will accept a report. The report states the scope, the method and the limits of the test.

## Sources

- [Comparing AI Agents to Cybersecurity Professionals in Real-World Penetration Testing (arXiv 2512.09882)](https://arxiv.org/abs/2512.09882)
- [Automated Penetration Testing with LLM Agents and Classical Planning (arXiv 2512.11143)](https://arxiv.org/abs/2512.11143)
- [OWASP Web Security Testing Guide](https://wstg.owasp.org/)
- [OWASP DevSecOps Guideline: Dynamic Application Security Testing](https://owasp.github.io/www-project-devsecops-guideline/latest/02b-Dynamic-Application-Security-Testing)