Visual Regression
Swarm QA includes a visual regression engine that compares screenshots across runs using pixelmatch — a pixel-level comparison library with zero external dependencies.
How It Works
- During a scan with screenshots enabled, each visited page is captured as a PNG.
- When a new run targets the same URL, xyva retrieves the baseline screenshot from the previous run.
- Pixelmatch compares the two images pixel-by-pixel and generates a diff image highlighting changes.
- If the percentage of changed pixels exceeds the configured threshold, a visual regression finding is created.
Baseline (Run #14) -> Current (Run #15) -> Diff (red = changed pixels)
+--------------+ +--------------+ +--------------+
| | | ## | | ## |
| Page as | | Page with | | Changed |
| expected | | layout | | regions |
| | | shift | | highlighted |
+--------------+ +--------------+ +--------------+View Modes
The visual diff viewer (VisualDiffViewer.tsx) offers two display modes:
| Mode | Description |
|---|---|
| Side-by-Side | Baseline and current screenshots displayed next to each other with the diff image below. Best for spotting layout shifts. |
| Diff Only | Shows only the diff image with changed pixels highlighted in red. Best for quickly scanning many pages. |
TIP
Use side-by-side mode when investigating a specific regression. Use diff-only mode when reviewing a full scan with dozens of pages.
Threshold Configuration
The pixel difference threshold determines how sensitive the comparison is:
| Threshold | Sensitivity | Use case |
|---|---|---|
| 0.01 | Very strict | Pixel-perfect designs, brand pages |
| 0.05 | Standard | General UI regression detection |
| 0.10 | Relaxed | Dynamic content, ads, date-dependent text |
| 0.20 | Loose | Structural changes only |
Configure the threshold in Expert Mode > Visual Regression > Threshold. The default is 0.05 (5% pixel difference).
WARNING
Pages with dynamic content (timestamps, ads, user avatars) will trigger false positives at low thresholds. Use exclude regions or increase the threshold for these pages.
Cached Diffs
Diff images are cached locally to avoid recomputation. The cache is stored alongside run data in .xyva/agent-memory/swarm-runs/. Clearing a run from history also removes its cached diffs.
Baseline Management
The baseline for each URL is automatically set to the most recent run's screenshot. To manually set a baseline:
- Open the History tab and expand the desired run.
- Click Set as Baseline on a specific page screenshot.
- Future comparisons for that URL will use the selected screenshot.