QAForge

AI assisted development has increased how fast a product can be built, but testing capacity has not grown at the same rate. The person building a product can easily end up writing the requirements, implementing them, testing them and deciding whether they are good enough.

Most AI assisted testing uses one agent. That agent plans the tests, runs them and grades itself, which leaves the same blind spots a solo builder already has.

QAForge uses two agents from two different providers, each on the most capable model that provider offers. The reference setup is Codex on OpenAI's strongest model and Claude Code on Anthropic's strongest model.

Each agent creates its own runs with its own test identities, records factual results through the command line tool, then reviews the other agent's high risk findings and a sample of its critical passes. Neither agent can edit the other's original evidence.

The rule the whole workspace enforces is simple. Evidence determines a result, not agent agreement. A passing reachability check is not a passing product assessment, and agreement between the two agents cannot waive a mandatory check or a missing piece of evidence. The product owner makes the final release decision.

QAForge is a filesystem and a small command line tool built on the Python standard library. It holds the standards, templates, per product test catalogs, run records and evidence for an assessment. It does not launch agents and it does not run in the cloud. You open one agent session per product and point it at its folder, and the filesystem becomes the durable record of what was actually done.

There are two ways in. Without writing code, you tell Claude Code or Codex to clone the repository and describe what you want to test, and the assistant runs the bootstrap itself. From a terminal, you run an init command, then start a session for either agent with the session brief preloaded.

Every session starts with the agent stating the goal and a short plan in chat, then waiting for an explicit confirmation before it runs anything. A new run refuses to start without a goal and the name of the person who confirmed the plan, and each confirmed session is logged with a date.

Passwords never go into a session. You authenticate in the browser yourself when needed, and the runner refuses to record a result note that looks like it contains a secret.

Target URLs start unset on purpose. Configuration records who authorized the assessment and how, and it expires after fourteen days. Production defaults to read only, and fixture writes, controlled messaging and sandbox payments are separate permissions that must be added to the saved scope.

Plain HTTP targets and private or loopback addresses are rejected unless you explicitly allow them for a local staging host, and cloud metadata endpoints are always rejected.

Core user flows end to end, UI checks including layout, responsiveness, keyboard access and visible focus, form and input checks, saved state after reloads and edits, error, loading and empty states, basic access and permission boundaries, and basic sensitive data exposure in what the browser and the application's own responses reveal.

QAForge is not penetration testing or detailed cybersecurity testing. It is not a vulnerability scanner, so there is no automated crawling or exploitation. It is not a certification, and it is not a release gate on its own, because a person decides whether to release.

Result history is append only through the command line tool, but the files stay editable by hand. That is workflow preservation, not a tamper proof audit vault.

Both agents work inside the ECC plugin, an open source agent harness by Affaan Mustafa, which provides planning, test driven development, fresh context code review and security scanning as reusable workflows. ECC is a separate dependency that is not bundled in the repository. QAForge still works without it, but you then run that review and scanning discipline yourself.

Can two agents from two providers give a solo builder more QA capacity than one agent can? Does requiring evidence for every result keep AI testing honest? Where should a person stay inside the loop, and how much written scope and authorization does an agent need before testing becomes safe to run?