Data Science CV Repro Reviewer
by @zack-dev-cm
Review computer-vision experiment reproducibility evidence, dataset readiness, metric gates, and launch risk. Use when a user asks for a cautious CV experime...
clawhub install data-science-cv-repro-labπ About This Skill
name: data-science-cv-repro-lab description: Data Science CV Repro Lab is a public ClawHub CV repro-lab skill. Use it when the user says "cv repro lab", "computer vision reproducibility", "CV experiment evidence", or wants execution-grade CV experiments and evidence capture across Colab, Kaggle, browser automation, and GPU VMs. version: 1.9.3 homepage: https://zack-dev-cm.github.io/ user-invocable: true metadata: {"openclaw":{"homepage":"https://zack-dev-cm.github.io/","skillKey":"data-science-cv-repro-lab","requires":{"anyBins":["python3","python"]}}}
Data Science CV Repro Lab
Search intent: cv repro lab, computer vision reproducibility, cv experiment evidence, colab kaggle cv workflow
Goal
Turn CV work into a reproducible decision loop:
This skill is the execution and evidence layer, not the full score-improvement stack.
If the real ask is "beat the baseline", "escape a plateau", or "find a better recipe",
pair it with sota-agent and a fixed improvement harness before spending more compute.
Use This Skill When
If the primary goal is benchmark improvement rather than clean execution, route the search loop
through sota-agent and use this skill as the execution lane.
Quick Start
1. Lock the objective before touching code. - Write the product problem in one sentence. - Name the primary metric. - Name the non-regression surfaces. - State what blocks promotion.
2. Initialize the durable records immediately.
- Use python3 {baseDir}/scripts/init_cv_dataset_manifest.py --out .
- Use python3 {baseDir}/scripts/init_cv_run_card.py --out .
- If the task is plateau recovery or benchmark improvement, use python3 {baseDir}/scripts/init_cv_improvement_harness.py --out .
- If the workflow mixes runtime sweeps, QA runs, benchmark panels, or synced VM artifacts, use python3 {baseDir}/scripts/init_cv_review_dashboard_manifest.py --out .
- If a browser lane matters, use python3 {baseDir}/scripts/init_cv_browser_run_card.py --out .
- If browser-visible overlays or prompt variants are part of the hypothesis, use python3 {baseDir}/scripts/init_cv_validation_scorecard.py --out .
- If a long VM run is involved, use python3 {baseDir}/scripts/init_cv_vm_bootstrap_manifest.py --out .
3. Capture the current state immediately.
- Use python3 {baseDir}/scripts/capture_cv_run_context.py --repo-root .
- Record git state, module versions, GPU state, and experiment params before launch.
- Use the dataset, artifact, and browser manifest helpers for any additional evidence instead of broad host inspection.
4. Pick the right orchestration lane. - Local debug lane: tiny overfit, transform audits, shape and dtype checks. - Browser notebook lane: Colab or Kaggle steps that must happen in a real browser or notebook UI. - Colab GPU lane: runtime selection, smoke validation, artifact export, and browser evidence. - Custom VM or cluster lane: long runs with heartbeats, watchdogs, stall detection, sync, and auto-stop. - Review dashboard lane: one local or synced surface for runtime sweeps, QA runs, curated comparisons, and benchmark panels. - Promotion lane: fixed benchmark matrix plus customer-facing surface checks.
5. Work the debug ladder in order.
- Harness review: fixed split, primary metric, slice table, rerun rule, and stop condition.
- Validation scorecard: browser or notebook visual QA with per-image pass or fail notes when the UI is part of the release story.
- Data audit: split integrity, label normalization, image-mask pairing, resize geometry.
- Preview audit: at least one augmentation preview and one transformed batch preview.
- Failure-set review: keep 20-50 representative overlays with short notes instead of trusting one scalar metric.
- Tiny overfit: 4-16 shared samples with no_aug.
- Short resumed run: continue from the best trusted checkpoint.
- Long run: only after the short loop is healthy.
6. Keep agentic work bounded. - External browser LLM output is hypothesis generation, not release evidence. - Keep the main thread on the benchmark contract and improvement harness. - Use bounded Codex subagents for scouting, data audits, patch proposals, and per-case review. - For repeated case review, batch over a manifest or CSV instead of free-form chat drift. - Browser steps must emit screenshots, machine-readable scores, and explicit success markers. - Hard-fail on unavailable browser modes, dead CDP sessions, or ambiguous notebook state. - Keep planner, executor, reviewer, and promoter responsibilities distinct even if one agent performs more than one role.
7. Promote only on full-surface wins.
- Raw checkpoint quality
- Exported or runtime quality
- User-facing render, service, or product surface
- Runtime cost or throughput if deployment matters
- Adjacent-seed or rerun stability if the claimed delta is small
- Generate a promotion bundle with python3 {baseDir}/scripts/init_cv_promotion_bundle.py --out before the final decision.
Operating Rules
Research before edits
research, plan, journal, and evidence.Agentic orchestration rules
Browser automation rules
Colab GPU rules
Custom VM and cluster rules
Review dashboard rules
CV training rules
Plateau recovery rules
Derm and segmentation rules
Codex and auth rules
OPENAI_API_KEY, other vendor API keys, or paid inference APIs as the default runtime path.Promotion rules
Public distribution rules
{baseDir} when pointing at bundled scripts or references.References
Read only the reference that matches the task:
references/official-repro-guidance.mdreferences/agentic-research-patterns.mdkarpathy/autoresearch style loops to DS and CV work.
references/improvement-harness-and-oauth-stack.mdreferences/openclaw-browser-lane.mdreferences/colab-vm-operations.mdreferences/kaggle-2026-practices.mdreferences/cross-repo-cv-patterns.mdreferences/publication-security.mdreferences/runtime-serving-change-gates.mdBundled Scripts
scripts/capture_cv_run_context.pyscripts/init_cv_task_scaffold.pyscripts/init_cv_run_card.pyscripts/init_cv_improvement_harness.pyscripts/init_cv_review_dashboard_manifest.pyscripts/init_cv_dataset_manifest.pyscripts/init_cv_browser_run_card.pyscripts/init_cv_validation_scorecard.pyscripts/render_cv_run_summary.pyscripts/init_cv_artifact_manifest.pyscripts/init_cv_vm_bootstrap_manifest.pyscripts/init_cv_promotion_bundle.pyπ‘ Examples
1. Lock the objective before touching code. - Write the product problem in one sentence. - Name the primary metric. - Name the non-regression surfaces. - State what blocks promotion.
2. Initialize the durable records immediately.
- Use python3 {baseDir}/scripts/init_cv_dataset_manifest.py --out .
- Use python3 {baseDir}/scripts/init_cv_run_card.py --out .
- If the task is plateau recovery or benchmark improvement, use python3 {baseDir}/scripts/init_cv_improvement_harness.py --out .
- If the workflow mixes runtime sweeps, QA runs, benchmark panels, or synced VM artifacts, use python3 {baseDir}/scripts/init_cv_review_dashboard_manifest.py --out .
- If a browser lane matters, use python3 {baseDir}/scripts/init_cv_browser_run_card.py --out .
- If browser-visible overlays or prompt variants are part of the hypothesis, use python3 {baseDir}/scripts/init_cv_validation_scorecard.py --out .
- If a long VM run is involved, use python3 {baseDir}/scripts/init_cv_vm_bootstrap_manifest.py --out .
3. Capture the current state immediately.
- Use python3 {baseDir}/scripts/capture_cv_run_context.py --repo-root .
- Record git state, module versions, GPU state, and experiment params before launch.
- Use the dataset, artifact, and browser manifest helpers for any additional evidence instead of broad host inspection.
4. Pick the right orchestration lane. - Local debug lane: tiny overfit, transform audits, shape and dtype checks. - Browser notebook lane: Colab or Kaggle steps that must happen in a real browser or notebook UI. - Colab GPU lane: runtime selection, smoke validation, artifact export, and browser evidence. - Custom VM or cluster lane: long runs with heartbeats, watchdogs, stall detection, sync, and auto-stop. - Review dashboard lane: one local or synced surface for runtime sweeps, QA runs, curated comparisons, and benchmark panels. - Promotion lane: fixed benchmark matrix plus customer-facing surface checks.
5. Work the debug ladder in order.
- Harness review: fixed split, primary metric, slice table, rerun rule, and stop condition.
- Validation scorecard: browser or notebook visual QA with per-image pass or fail notes when the UI is part of the release story.
- Data audit: split integrity, label normalization, image-mask pairing, resize geometry.
- Preview audit: at least one augmentation preview and one transformed batch preview.
- Failure-set review: keep 20-50 representative overlays with short notes instead of trusting one scalar metric.
- Tiny overfit: 4-16 shared samples with no_aug.
- Short resumed run: continue from the best trusted checkpoint.
- Long run: only after the short loop is healthy.
6. Keep agentic work bounded. - External browser LLM output is hypothesis generation, not release evidence. - Keep the main thread on the benchmark contract and improvement harness. - Use bounded Codex subagents for scouting, data audits, patch proposals, and per-case review. - For repeated case review, batch over a manifest or CSV instead of free-form chat drift. - Browser steps must emit screenshots, machine-readable scores, and explicit success markers. - Hard-fail on unavailable browser modes, dead CDP sessions, or ambiguous notebook state. - Keep planner, executor, reviewer, and promoter responsibilities distinct even if one agent performs more than one role.
7. Promote only on full-surface wins.
- Raw checkpoint quality
- Exported or runtime quality
- User-facing render, service, or product surface
- Runtime cost or throughput if deployment matters
- Adjacent-seed or rerun stability if the claimed delta is small
- Generate a promotion bundle with python3 {baseDir}/scripts/init_cv_promotion_bundle.py --out before the final decision.