The problem
Most automated tests either mock the database, which tests the mock and not the system, or run against a
shared database and leave test rows behind for the next person to trip over. Neither tells you whether the
real application, on a real database, behaves when someone types something wrong or something hostile.
Why it is hard
- Cleaning up. The usual trick, rolling back a test transaction, cannot undo what the application
writes through its own database connections.
- Testing the server, not the browser. A form's own checks hide server bugs, because an attacker
simply skips them.
- Safety. A SQL injection test that works really does drop a table, so it must never touch a real database.
- Scale. Hundreds of tests are only useful if anyone can add one without writing code.
What it does
- Two database modes. External connects to an approved test database with credentials and restores it.
Internal creates a throwaway PostgreSQL in Docker (Testcontainers), starts the app against it, and deletes it afterwards.
- Snapshot and exact restore. Every row and id sequence is recorded first. Afterwards, added rows are
deleted children first, changed rows set back, removed rows put back parents first, sequences reset, all in one
transaction, and a second snapshot must match the first exactly.
- Checks: connection (resolve, port,
SELECT 1, route trace), HTTP, happy path, wrong input
with browser checks switched off, SQL injection (throwaway database only), and a race test that fires many
requests at the same instant.
- Workflows as tables. Multi-step macros written by hand or recorded by clicking through the app,
repeated as often as you like with fresh Latin dummy data.
- Log analysis. Stack traces reduced to root causes with secrets hidden, and an optional Claude
explanation of each failure.
The proof
- It found four real bugs on its first run. The demo app saved a customer with an empty name, saved
"not-an-email" as an email, crashed on an order for a customer that does not exist, and saved an order with
an empty product. All four were hidden by the browser's own checks. They are fixed, and four regression
tests pin them; those tests were run against the original code, where they failed, to prove they catch the bugs.
- 143 of 143 passed in one internal-mode run with each workflow repeated 50 times, including 25 requests
fired at once (exactly 25 rows appeared) and 15 SQL injection attacks (every table intact).
- The database is left exactly as found. In external mode, a real customer and order were untouched,
and the next id was still the next id.
- 54 unit, controller, regression and error-path tests. The error-path tests break things on purpose:
a seed that cannot succeed, an unreachable database, a restore blocked partway through (it rolls back
completely), an app that dies at start.
Technologies and skills
- Java 21, Maven, one class per job, plain JDBC for the database work.
- Spring Boot 3.5 demo app with embedded Tomcat, Spring Data JPA, Thymeleaf, and MockMvc controller tests with Mockito.
- PostgreSQL 16 schema discovery from
information_schema, foreign-key ordering, sequence reset.
- Selenium WebDriver for page scanning, form filling, macros and a macro recorder.
- Testcontainers and Docker for throwaway databases; GitHub Actions for CI.
- Anthropic Java SDK for the optional failure analysis, sent only redacted evidence.
- AI-assisted engineering. I set the design, the rules and every check; the AI wrote and tested the code
under my direction. I am now rewriting three of its parts by hand as practice drills.
How it was built: about two hours, from my prompts
This is a featured AI-assisted build. I directed it; Claude wrote and ran the code. The times come from the
file system and the git history, measured, not estimated. They are wall-clock time with me directing,
including every round of changes.
- 9:30 PM. First file written.
- 10:06 PM, 36 minutes in. First working version committed: seeding, HTTP and browser tests, exact
restore, 22 tests passing, and a run that left the database untouched.
- 11:31 PM, about 2 hours in. The full QA-in-a-box version: two database modes, wrong input, SQL
injection, race and connection checks, macros and a recorder, log and AI analysis, 54 tests, CI and documentation.
The AI and tools
- AI: Claude Code (Anthropic's coding agent) running Claude Opus 5.5, inside VS Code on Windows 11.
- Build and run: Java 21 (Eclipse Temurin), Maven 3.9, Docker Desktop, PostgreSQL 16, Google Chrome.
- Libraries: Spring Boot 3.5 with embedded Tomcat 10.1, Spring Data JPA and Thymeleaf; Selenium WebDriver 4.27;
Testcontainers 1.21; JUnit 5, Mockito and MockMvc; the Anthropic Java SDK.
- Delivery: Git and GitHub, GitHub Actions.
The prompts I gave, in order
Exactly as I typed them, with spelling tidied. Most came in while the AI was already working; it
finished the step in hand, then took up the next.
- "I want to rebuild TestThisPuppy with a tester that sends HTTP requests, opens the database, fills it with dummy
data, runs the tests and sends the results, and have it integrated with Selenium and WebDriver, where it scans
the page, finds all the elements and populates them. Then when it's complete it cleans up the database as if it
was never there. This way, real database testing on the code base."
- Asked three questions, I chose: write it as a mix (the AI builds the plumbing, I write three core parts as
practice drills), Java 21 with Maven, and PostgreSQL.
- "https://github.com/kenbin64/testthispuppy.git"
- "Just use Latin words to fill the forms and data. Create a filler word CSV file."
- "It can connect to the actual database with credentials, or create an internal DB just for testing."
- "Have a happy path option, a wrong input type test, and test your try-catches to throw errors."
- "Also test for SQL injection, using the internal database only, for safety."
- "Test ping and source route to check the database connection, test forms in the UI, test backend controllers
and objects, test race conditions, regression tests and unit tests."
- "Do we have Tomcat?" Then: "That's right, Spring Boot has it."
- "I also AI-assisted to analyze logs and stack traces."
- "Also include complete documentation."
- "Workflows."
- "Ways to create macros and automated tests using WebDriver."
- "Able to do hundreds of tests, all automated."
- "Your QA in a box."
- "Yes, let's put it in GitHub and on butterflyfx.us, my portfolio page."
- "I am going to do a featured AI-assisted program. Say how long it took to build and add the prompts I used to create it."
- "And the AI and tools used, VS Code, etc."
What the prompts do not show is the checking. Every claim on this page was run before it was written down, and
when a check found a problem (a test that passed for the wrong reason, a safety notice that could be skipped,
a regression test that needed proof it could fail), it was fixed before moving on.
What it does not claim
- It reads PostgreSQL's public schema; other databases need their own schema handling.
- Every table needs a primary key.
- The AI analysis is built and tested up to the API call, but needs your own Anthropic credentials and costs money per run.
4real bugs found, first run
143/143checks in one run
54unit and error-path tests
0rows left behind