Parameter Golf Monitor

---

name: parameter-golf-monitor description: > Monitor the openai/parameter-golf competition leaderboard by fetching PR data from GitHub. Use this skill whenever the user asks about the parameter-golf competition, leaderboard standings, competitor scores, PR rankings, who's winning, what the current SOTA is, or wants to track competition progress. Also trigger when the user mentions "parameter golf", "val_bpb scores", "competition PRs", or asks to check/watch the leaderboard. Works for any competitor — not tied to a specific GitHub account.

---

Parameter Golf Monitor

A skill for tracking the openai/parameter-golf competition leaderboard in real time by fetching open and merged PRs from GitHub's public API.

What this skill does

The monitor script at scripts/monitor.py queries the GitHub API for PRs on openai/parameter-golf, extracts val_bpb scores from titles and bodies, classifies submissions (record vs non-record), and displays a ranked leaderboard table.

It requires zero authentication — it uses the unauthenticated GitHub API (60 requests/hour rate limit).

When to use this skill

User asks "what's the current leaderboard?" or "who's winning parameter golf?"

User wants to check their ranking: "where do I stand?" or "how's my PR doing?"

User asks about competitor scores or techniques

User wants to monitor for new submissions or changes

User says "check the competition" or "show me the standings"

User wants to compare their score against the field

How to run

The script is at scripts/monitor.py relative to this skill's directory. Run it with Python 3 (no dependencies beyond stdlib).

One-shot leaderboard

python3 /scripts/monitor.py

Shows all open PRs ranked by val_bpb (lower is better).

Highlight a specific user

python3 /scripts/monitor.py --me 

Marks the user's PR(s) with <-- and shows their rank + gap to #1 in the summary. Ask the user for their GitHub username if they want highlighting and you don't already know it.

Common flag combinations

| Goal | Command | |------|---------| | Full leaderboard | python3 scripts/monitor.py | | Top 10 records only | python3 scripts/monitor.py --top 10 --records-only | | Highlight a user | python3 scripts/monitor.py --me USERNAME | | Merged records (SOTA history) | python3 scripts/monitor.py --merged | | Everything (open + merged) | python3 scripts/monitor.py --all | | Today's PRs only | python3 scripts/monitor.py --since YYYY-MM-DD | | JSON for piping | python3 scripts/monitor.py --json | | Live polling every 5 min | python3 scripts/monitor.py --watch 5 |

All flags

--me USER — Highlight a GitHub username. Shows rank and gap to #1.

--top N — Show only the top N scored entries (highlighted user always shown even if outside top N).

--records-only — Exclude non-record submissions.

--merged — Show only merged PRs.

--all — Show both open and merged PRs.

--since YYYY-MM-DD — Filter to PRs created on or after this date.

--json — Output as JSON instead of a table (useful for piping to jq or other tools).

--watch MIN — Poll every N minutes continuously (Ctrl+C to stop).

Interpreting results

Rank is by val_bpb ascending (lower = better). Only scored PRs get a numeric rank.

Status shows open, merged, or non-rec (non-record submission).

Scores marked ? mean no val_bpb could be extracted from the PR title or body.

Some very low scores (e.g. < 1.10) may be evaluation-only or use non-standard setups — check the PR for details.

The gap to #1 in the summary tells the highlighted user how far they are from the top.

Rate limits

The GitHub API allows 60 unauthenticated requests per hour. Each run_once call makes 1 request (open PRs) or 2 requests (if --merged or --all). The --watch mode respects this — polling every 5 minutes uses ~12-24 requests/hour.

If you hit the rate limit, the script will show an error and retry on the next interval.

Example output

[2026-03-20 02:17:15] Parameter Golf Leaderboard (open PRs)
Rank   val_bpb     PR  Status    Author              Date        Title
----------------------------------------------------------------------------------------------------
   1   1.15390  #135   open      unnir               2026-03-19  Record: OrthoInit + Int6 MLP3x + BigramHash + SmearGate
   2   1.15800  #106   open      krammnic            2026-03-19  record: 1.158
   3   1.15850  #122   open      mtybadger           2026-03-19  Record: Sliding Window Eval, 2048 Vocab Size
   4   1.16019  #156   open      dexhunter           2026-03-20  Int6 STE + NorMuon + SWA + Sliding Window  <--
Best: val_bpb=1.15390 (#135 by unnir)
Total: 56 PRs, 40 with scores
You (dexhunter): rank #4, val_bpb=1.16019, gap to #1: +0.00629