Generic Quality Gateways for Unattended Agent Development

openClaw Skill: Quality Gateways (Generic Web + API Applications)

Purpose

This skill defines and applies 6 universal quality gateways for typical application projects that include:

Backend API services (any stack)

Web frontends (any stack)

CI/CD pipelines (any provider)

The gateways are written in LLM-friendly operational language: how to check, calculate, evaluate, and document results consistently.

This skill is language-agnostic and can be used on any repository. It relies on a central configuration file:

.defs/quality-gateway-definition.json (MUST be stored in the repository, not the workspace)

Non-Negotiable Storage Rules (openClaw)

The gateway definition file MUST be placed in: REPO_ROOT/.defs/quality-gateway-definition.json

Temporary files MUST go to: REPO_ROOT/.tmp/quality-gates/ (do not create or delete other workspace directories)

Reports MUST be written to repository paths defined in the JSON config (default suggested below)

Inputs

Repository root path (REPO_ROOT)

Optional CI artifacts path (if provided by the runtime)

Optional commit range (for PR-focused evaluation)

Optional environment notes (target load, environments, risk level)

Outputs

1. A human-readable report (Markdown) 2. A machine-readable report (JSON) containing raw metrics + per-check scores 3. Evidence references (paths, snippets, CI links if available)

Recommended default output paths (override via JSON config):

docs/quality/quality-gate-report.md

docs/quality/quality-gate-report.json

Evidence directory: docs/quality/evidence/

---

The 6 Quality Gateways

Each gateway produces:

Score: 0–100

Status: PASS / WARN / FAIL

Blocking behavior: some gateways are “blocking” (FAIL blocks release)

All gateway thresholds and weights come from:

.defs/quality-gateway-definition.json

---

Gateway 1 — Build & Dependency Health

Goal

Ensure the system can be built and packaged reliably, and dependencies are manageable and safe to ship.

What to Check (typical checks)

CI pipeline status (green on default branch / PR)

Reproducible build or deterministic packaging indicators

Dependency freshness (stale/outdated dependencies)

License policy compliance (allowlist/denylist)

SBOM presence (if required)

How to Measure / Calculate

Boolean checks: PASS=100, FAIL=0

Ratio checks (e.g., “outdated deps %”): scale 0–100 using thresholds

Policy checks: hard FAIL if a forbidden license is detected (if enabled)

Evidence to Collect

CI job summary (or local build logs)

Dependency list report output (tool-specific, but keep the report file)

SBOM artifact path (if present)

License scan output (if used)

How to Document

In the report, include:

Build command/pipeline name

Artifact identifiers / versions

Summary of dependency deltas and policy results

---

Gateway 2 — Automated Testing & Coverage

Goal

Prove correctness through automated tests and prevent regression.

What to Check

Unit tests pass

Integration/API tests pass (or contract tests)

E2E/smoke tests pass (for web apps)

Code coverage meets thresholds (overall + critical components)

Flaky test rate is controlled (if CI provides retries/flakes)

How to Measure / Calculate

Test pass: boolean

Coverage: numeric percentage

- Score mapping example: - >= target: 100 - between warn and target: linear 70–99 - below warn: linear 0–69

Optional “critical path coverage” gets extra weight

Evidence to Collect

Test run outputs (JUnit/TRX/etc.)

Coverage summary files

List of failed tests (if any) + links

How to Document

Test suites executed

Coverage numbers (overall + key areas)

Notes on skipped tests (if allowed) and rationale

---

Gateway 3 — Security & Supply-Chain

Goal

Prevent known vulnerabilities, secrets leakage, insecure configs, and supply-chain risks.

What to Check

Dependency vulnerabilities (Critical/High/Medium counts)

Secret scanning results (must be zero leaked secrets)

Basic secure configuration checks (CSP, TLS, auth boundaries) where applicable

SAST findings severity counts (if tooling exists)

Container image scan (if containers exist)

How to Measure / Calculate

Vulnerability gating (typical):

- Critical = 0 required (FAIL otherwise) - High = 0 required (or <= allowedHigh) - Medium allowed up to a budget (WARN if above warn)

Secrets: any secret finding => FAIL (blocking)

Score: start at 100 and subtract penalties by severity and count (config-driven)

Evidence to Collect

Vulnerability scan report files

Secret scan output (including file paths and fingerprint IDs, not actual secrets)

SAST report snippet/summary

How to Document

Severity counts and whether exceptions exist

Any exception MUST include: reason, owner, expiry date (if your org uses waivers)

---

Gateway 4 — Performance & Efficiency (API + Web)

Goal

Ensure the system meets baseline performance and user experience targets.

What to Check

API (typical):

p95 latency under target

Error rate under target

Throughput meets expected load (if known)

Web (typical):

Core Web Vitals (LCP, CLS, INP) on a reference device/profile

Bundle size / asset weight thresholds (optional)

How to Measure / Calculate

Latency score:

- p95 <= target: 100 - between target and warn: linear 70–99 - > warn: 0–69 (linear), with hard FAIL if beyond “max”

Error rate:

- <= target: 100 - <= warn: 70–99 - > warn: 0–69, FAIL if beyond max

Web vitals:

- Each metric scored independently; weighted into a single web score

Evidence to Collect

Load test or benchmark outputs (k6/JMeter/etc.)

APM snapshots (if available)

Lighthouse or Web Vitals report exports

How to Document

Test conditions: environment, dataset size, concurrency, device profile

Key p95 / error rate / vitals values

Notable regressions vs baseline

---

Gateway 5 — Maintainability & Code Health

Goal

Keep the codebase understandable, changeable, and reviewable over time.

What to Check

Static analysis quality (lint errors, rule violations)

Complexity thresholds (cyclomatic complexity, large functions/classes)

Duplication rate

“Change risk” signals (hotspots: frequent churn + complexity)

Documentation coverage for public APIs (e.g., endpoint docs, component docs)

How to Measure / Calculate

Issue density: findings per KLOC (or per file for smaller repos)

Complexity score: percentage of units exceeding complexity threshold

Duplication: % duplicated lines

Score: weighted average of normalized sub-scores (config-driven)

Evidence to Collect

Static analysis summaries

Complexity and duplication reports (any tool is fine; store outputs)

List of top hotspots and why (files + metrics)

How to Document

Top 10 problems by impact

Concrete refactoring suggestions only if asked; otherwise just findings

---

Gateway 6 — Release Readiness & Operability (Observability + Runbooks)

Goal

Make sure the system can be operated safely in production.

What to Check

Health endpoints exist and are meaningful

Logging is structured and includes correlation IDs

Metrics and dashboards exist for key signals (latency, error rate, saturation)

Alerts configured for SLO breaches / error budget burn (if applicable)

Runbooks for major failure modes exist (deploy rollback, incident triage)

Versioning and changelog/release notes exist

How to Measure / Calculate

Mostly “presence + completeness” scoring:

Each required artifact is a boolean check

Optional maturity rubric: 0 (missing), 50 (partial), 100 (complete)

Blocking if “minimum operability” is not met (config-driven)

Evidence to Collect

Paths to runbooks, dashboards-as-code, alert configs

Sample log/metric/tracing docs

On-call/ops notes (if present)

How to Document

List missing operational artifacts

Minimum go-live checklist status

---

Standard Evaluation Algorithm (LLM-Executable)

Step 1: Load configuration

Read REPO_ROOT/.defs/quality-gateway-definition.json

Validate it against the schema description (see below)

If fields are missing, use documented defaults from the JSON

Step 2: Collect metrics per check

For each gate:

For each check:

- Identify data source: - Prefer CI artifacts if provided - Otherwise use repository files and local commands (if allowed by runtime) - Produce a metric value (number/boolean/string) and evidence references

Step 3: Score each check (0–100)

Use the scoring method defined per check:

boolean: pass => 100, fail => 0

threshold_range: linear scoring between warn and target

penalty_by_count: start at 100 and subtract per issue

rubric: map {missing/partial/complete} to {0/50/100}

Step 4: Score each gateway

Compute weighted average of its checks

Determine gateway status using configured thresholds:

- Score >= passScore => PASS - Score >= warnScore => WARN - else => FAIL

If gateway is marked blockingOnFail=true, any FAIL blocks release

Step 5: Produce reports

Write: 1) Markdown report (human) 2) JSON report (machine) Include:

per-gateway score/status

per-check metrics + evidence paths

overall score and overall status

explicit “BLOCKERS” list if any

---

Report Template (Markdown)

Use this outline in docs/quality/quality-gate-report.md unless JSON overrides paths:

Summary

Overall Score:

Overall Status:

Blocking Failures:

Date/Commit:

Gateway Results

| Gateway | Score | Status | Key Metrics | Evidence | |---|---:|---|---|---|

Details (per Gateway)

Score/Status

Checks:

- : metric=..., score=..., evidence=...

Notes / Exceptions

---

quality-gateway-definition.json — JSON Schema Description

The configuration file is a normal JSON document with:

Root

schemaVersion (string) — version of this config layout

projectProfile (object) — context used for defaults

scoring (object) — global pass/warn thresholds and aggregation rules

reporting (object) — output paths and evidence folder

gates (array) — list of gateway definitions (exactly 6 for this skill)

projectProfile (object)

applicationType (string) — e.g. "web_api_and_web"

riskLevel (string) — "low"|"medium"|"high"

releaseCadence (string) — e.g. "daily"|"weekly"|"monthly"

expectedLoad (object, optional)

- apiRps (number) - concurrency (number)

scoring (object)

passScore (number 0–100)

warnScore (number 0–100)

overallAggregation (string) — "weighted_average"

blockIfAnyBlockingGateFails (boolean)

reporting (object)

markdownReportPath (string, repo-relative)

jsonReportPath (string, repo-relative)

evidenceDir (string, repo-relative)

tempDir (string, repo-relative; MUST be inside .tmp/quality-gates/)

gates (array of objects)

Each gate:

id (string) — stable identifier

name (string)

description (string)

weight (number) — relative importance in overall score

blockingOnFail (boolean)

checks (array)

checks (array of objects)

Each check:

id (string)

name (string)

description (string)

weight (number)

metricType (string) — "boolean"|"percentage"|"count"|"duration_ms"|"rubric"

scoringMethod (string) — "boolean"|"threshold_range"|"penalty_by_count"|"rubric"

thresholds (object) — depends on scoringMethod:

- for threshold_range: - target (number) - warn (number) - max (number, optional hard-fail) - direction (string) — "higher_is_better"|"lower_is_better" - for penalty_by_count: - allowed (number) - warnAbove (number) - failAbove (number) - penaltyPerUnit (number)

evidenceHints (array of strings) — where to find evidence in a generic repo/CI

notes (string, optional)

---

Operational Notes

If a metric cannot be measured, do NOT invent numbers.

- Mark the check as "unknown" in the JSON report and score it using the config’s fallback rule (recommended: treat unknown as WARN with score 70 unless the check is security/secrets, where unknown should be FAIL).

Always include evidence references (paths or CI artifact names).

Keep all temp work inside .tmp/quality-gates/.

JSON references

templ/quality-gateway-definition-template.json (template settings file. Can be copied to REPO_ROOT/.defs/quality-gateway-definition.json if missing)