QAForge Agent

AI assisted development has dramatically increased software development speed. A solo builder can now design and ship functionality that previously required a larger engineering team. But faster development creates another problem: testing capacity does not automatically increase at the same rate.

The person building the product can easily become the same person writing requirements, implementing functionality, debugging, testing, and deciding whether the implementation is acceptable. That creates blind spots.

Traditional automated testing remains important, but many product quality problems require more than deterministic unit or integration tests. Someone still needs to experience the product and ask whether a flow actually makes sense, whether errors are understandable, whether permissions are enforced, and whether the workflow can be broken.

Modern AI agents can operate browsers and computers. I wanted to determine whether those capabilities could become a structured product QA system rather than simply an agent randomly clicking through a website.

Browser capable AI agents should not simply be told to test a website. Effective AI QA needs context, test objectives, defined perspectives, environment information, expected behavior, structured reporting, independent evaluation, cross-model comparison, and human controls.

The intelligence of the underlying model matters, but the quality of the QA methodology surrounding the model matters just as much. QAForge was designed around that principle.

A user forks or clones the repository, provides project context such as the application URL, expected functionality, test environment, available user roles, and test objectives, then runs the workflow locally. The agents receive that context before they ever touch the application.

QAForge intentionally maintains human boundaries. Sensitive information should not be handed to an autonomous agent simply because the agent is technically capable of navigating the interface.

One important example is payment information. The system stops before entering real card or sensitive payment credentials, the user completes that interaction manually, and the agent then continues testing the resulting application state.

This demonstrates an important design principle: agentic automation should not mean unrestricted autonomy. Human intervention should exist at appropriate trust boundaries.

The first perspective evaluates the product like an experienced Product Manager, covering user journeys, onboarding, navigation, feature discoverability, product logic, empty and error states, and role behavior. The objective is not whether a button works, it is whether the product behaves coherently for the intended user.

The second perspective evaluates the application like a QA Engineer, covering functional testing, negative and boundary testing, form validation, authentication and authorization flows, role-based access, error handling, data persistence, and regression risks. The agent documents each issue with severity, steps to reproduce, expected and actual behavior, and evidence.

The third perspective examines externally observable security and abuse risks, including authentication behavior, authorization boundaries, session handling, exposed information, and obvious application-level security weaknesses. This is defensive product-security evaluation, not a replacement for penetration testing, security audits, or specialist security engineering. It adds another perspective to the QA workflow.

A key part of the experiment is model diversity. Rather than relying on a single AI agent to determine whether the product works, the same product is evaluated independently by strong models across OpenAI Codex and Claude Code. Each agent receives the relevant project context and testing methodology, performs its own evaluation, and generates its own report.

The reports are then compared. Agents examine issues identified by both systems, issues identified by only one system, severity disagreements, different interpretations, and potential false positives. The objective is not to assume agreement automatically means truth, it is to use disagreement as an evaluation signal, and disputed findings can be re-tested where necessary.

The final workflow produces a consolidated QA report with an executive summary, test coverage, issues grouped by severity, product experience findings, functional QA findings, security observations, cross-agent agreements and disagreements, evidence, and recommended actions. The document becomes an actionable input for the next development cycle.

A single model can miss an issue. A single model can also incorrectly identify one. Using multiple independent evaluations creates another layer of scrutiny: Agent A tests the product, Agent B tests the product independently, Agent A evaluates B, Agent B evaluates A, and disputed findings are examined before a final report consolidates the strongest supported findings.

This transforms QA from a single agent producing a report into a pipeline of independent evaluation, comparison, critique, synthesis, and human review.

Where the target application belongs to the user and repository access is intentionally provided, the agent can use repository context to better understand implementation, routes, expected workflows, and API relationships, so browser observations and implementation context can complement each other. This assumes repository access is not available for third-party products.

Can AI agents meaningfully increase QA capacity for solo builders? Can browser agents evaluate product quality beyond deterministic tests? Does multi-model evaluation identify more problems than a single model, and can model disagreement become useful QA evidence? Where should humans remain inside autonomous testing workflows, and how much product context does an agent need before testing becomes useful?