Testing evidence
Get a test report with screenshots and a pass, fail, or mixed result before you merge.
A test report records what was tested, how it was tested, what happened, and what remains unverified. Screenshots and recordings should support the results beside them. Alan saves it as an artifact on the task or session, where reviewers can read and comment on it.
How a test report gets made
- The agent tests its own work. When an agent changes a user-facing flow, it runs the app, exercises the change, and writes a test report, using Alan's built-in alan-test-feature skill when it's installed. If testing is blocked, it still files a partial report that names the blocker.
- You ask for one. For example: "Test the new filter on the Pull Requests page in the browser and create a test report with screenshots."
- You write one. In the task's Artifacts panel choose New, then Test report.
The report lands in Artifacts on the task, or on the session if it isn't part of a task, and appears as a card in the conversation.
Read a test report
| Part | What it tells you |
|---|---|
| Title and summary | What was tested and the outcome, in a sentence or two |
| Result | Pass, Fail, or Mixed |
| Report body | How it was tested, what happened, and what is still unverified, with screenshots and recordings placed where they prove a point |
| Branch | The branch that was tested, when the agent records it |
Older reports can also show Steps (with a count such as 4/5 steps passed), Issues, Recording, and Screenshots sections.
Use Mixed when some checks passed and others failed or couldn't run. A report that says what is unverified is more useful than one that claims a clean pass.
Like other artifacts, a test report has a download button, more actions (…), and Comments. See Task artifacts.
Match evidence to risk
| Change | Enough evidence |
|---|---|
| Docs or copy | The rendered page |
| Logic with unit tests | Test and type-check output |
| A user-facing flow | A test report with screenshots or a recording of the real app |
| Data, migrations, or runtime behavior | A test report, plus the checks that cover the risky path |
For browser evidence, capture the real route in the state that changed, not a nearby page or a mock.
Before you merge
- The linked task has a test report whose result matches what the pull request claims.
- The pull request's Required checks pass. See Pull requests.
- Anything marked unverified has an owner or a follow-up task.