Visual Qa

Visual QA

A visual regression testing pipeline for web applications. Capture baseline screenshots of your app, compare new screenshots against baselines using pixel-level diffing, and pass/fail based on configurable similarity thresholds.

When to Use

Quick Start

1. Capture Baselines

# Capture single URL
python scripts/capture.py http://localhost:3000 --output .visual-qa/baselines
Capture multiple viewports
python scripts/capture.py http://localhost:3000 --viewports desktop mobile tablet --output .visual-qa/baselines
Capture multiple pages from config
python scripts/capture.py --config .visual-qa.json --output .visual-qa/baselines

2. Compare Against Baselines

# Compare new screenshots
python scripts/diff.py --baseline .visual-qa/baselines --current .visual-qa/current --threshold 99

3. All-in-One Gate

# Capture + diff in one command
python scripts/gate.py --baseline .visual-qa/baselines --url http://localhost:3000 --threshold 99
With local dev server
python scripts/gate.py --baseline .visual-qa/baselines --server "npm run dev" --port 3000 --threshold 99
Using config file
python scripts/gate.py --config .visual-qa.json

Config File Pattern

Create .visual-qa.json in your project root:

{
  "urls": ["/", "/about", "/pricing"],
  "baseUrl": "http://localhost:3000",
  "viewports": ["desktop", "mobile"],
  "threshold": 99,
  "server": "npm run dev",
  "port": 3000,
  "baselineDir": ".visual-qa/baselines",
  "ignore": [".visual-qa/diffs", ".visual-qa/current"]
}

Scripts

All scripts support --help for detailed usage.

capture.py

Capture screenshots using Playwright (headless Chromium).

Features:

Usage:

python scripts/capture.py  --output  [options]
python scripts/capture.py --config  --output 

diff.py

Compare screenshots using pixel-level diffing (Pillow).

Features:

Usage:

python scripts/diff.py --baseline  --current  --output  --threshold 

gate.py

All-in-one gate: capture + diff in a single command.

Features:

Usage:

python scripts/gate.py --baseline  --url  --threshold 
python scripts/gate.py --baseline  --server  --port  --threshold 
python scripts/gate.py --config 

Workflow Examples

Initial Baseline Capture

# Start your app
npm run dev
Capture baselines (desktop + mobile)
python scripts/capture.py http://localhost:3000 --viewports desktop mobile --output .visual-qa/baselines

CI/CD Integration

# In your CI pipeline after build
python scripts/gate.py --baseline .visual-qa/baselines --server "npm start" --port 3000 --threshold 99
Exit code 0 = pass, 1 = fail
if [ $? -eq 0 ]; then
  echo "Visual QA passed ✓"
else
  echo "Visual QA failed ✗"
  exit 1
fi

Review Workflow

# 1. Developer makes UI changes
2. Capture new screenshots
python scripts/capture.py http://localhost:3000 --output .visual-qa/current
3. Generate diff images
python scripts/diff.py --baseline .visual-qa/baselines --current .visual-qa/current --output .visual-qa/diffs
4. Review diff images in .visual-qa/diffs/
5. If changes are intentional, update baselines:
rm -rf .visual-qa/baselines
mv .visual-qa/current .visual-qa/baselines

Multi-Page Testing

Create .visual-qa.json:

{
  "urls": ["/", "/products", "/about", "/contact"],
  "baseUrl": "http://localhost:3000",
  "viewports": ["desktop", "mobile"],
  "threshold": 99,
  "baselineDir": ".visual-qa/baselines"
}

# Capture all pages
python scripts/capture.py --config .visual-qa.json --output .visual-qa/baselines
Gate all pages
python scripts/gate.py --config .visual-qa.json

Dependencies

Scripts require Playwright and Pillow:

pip install playwright pillow
python -m playwright install chromium

Scripts will check for dependencies and print install instructions if missing.

Thresholds

The --threshold parameter controls similarity percentage (0-100):

Experiment to find the right threshold for your app. Start strict (99%) and loosen if you get false positives.

Ignoring Dynamic Content

For pages with dynamic content (dates, user-specific data):

1. Use data attributes to hide dynamic elements during testing:

   [data-test-hide] { visibility: hidden !important; }
   

2. Capture specific viewport regions (future enhancement)

3. Loosen threshold for pages with acceptable dynamic content

Troubleshooting

"Command not found: python"

"Playwright not installed"

"Similarity below threshold but images look the same"

"Server not starting"

"Images not found"

Tips

Integration with Other Skills

For detailed script options, run:

python scripts/capture.py --help
python scripts/diff.py --help
python scripts/gate.py --help

1. Capture Baselines

# Capture single URL
python scripts/capture.py http://localhost:3000 --output .visual-qa/baselines
Capture multiple viewports
python scripts/capture.py http://localhost:3000 --viewports desktop mobile tablet --output .visual-qa/baselines
Capture multiple pages from config
python scripts/capture.py --config .visual-qa.json --output .visual-qa/baselines

2. Compare Against Baselines

# Compare new screenshots
python scripts/diff.py --baseline .visual-qa/baselines --current .visual-qa/current --threshold 99

3. All-in-One Gate

# Capture + diff in one command
python scripts/gate.py --baseline .visual-qa/baselines --url http://localhost:3000 --threshold 99
With local dev server
python scripts/gate.py --baseline .visual-qa/baselines --server "npm run dev" --port 3000 --threshold 99
Using config file
python scripts/gate.py --config .visual-qa.json