Pa Eval

---

name: pa-eval description: "Evaluate PA performance through structured scoring, owner feedback analysis, and behavioral benchmarking. Use when: conducting a weekly/monthly PA performance review, owner gives feedback (positive or negative), assessing response quality, or identifying areas for improvement."

---

PA Eval Skill

Minimum Model

Any model for filling in templates. Use a medium model for trend analysis and recommendations.

---

When to Run

Weekly self-eval: Every 7 days. Run automatically.

On owner correction: Log the correction immediately, then re-score the affected dimension.

Monthly report: At the end of each month, aggregate all weekly evals.

On demand: If owner asks "how am I doing?" → generate current eval on the spot.

---

Scoring Dimensions

Score each 1–5:

| Dimension | What to Measure | |---|---| | Execution | Tasks completed without reminders | | Accuracy | Results are correct and complete | | Speed | Response time is fast | | Proactivity | Acts without being asked | | Communication | Concise and context-appropriate | | Memory | Remembers context across sessions | | Tool Use | Tools used correctly and efficiently | | Judgment | Knows when to act vs. when to ask |

Score meanings:

5 = Consistently exceeds expectations

4 = Meets expectations with minor gaps

3 = Acceptable but basic

2 = Frequent gaps or errors

1 = Fails basic expectations

Total: Max 40 points. Grade: A (36–40), B (28–35), C (20–27), D (<20)

---

Weekly Self-Evaluation

Save to .learnings/eval/YYYY-MM-DD.md.

# PA Weekly Eval — YYYY-MM-DD
Scores
| Dimension | Score | Notes |
|---|---|---|
| Execution | /5 | |
| Accuracy | /5 | |
| Speed | /5 | |
| Proactivity | /5 | |
| Communication | /5 | |
| Memory | /5 | |
| Tool Use | /5 | |
| Judgment | /5 | |
| TOTAL | /40 | |
Owner Feedback This Week
Positive:
Corrections:
Complaints:
Tasks Completed
-
Tasks Failed or Incomplete
-
What Went Well
-
What to Improve
-
Actions for Next Week
[ ]

Create the File

#!/bin/bash
set -e
Set the output directory
EVAL_DIR="$HOME/.openclaw/workspace/.learnings/eval"
mkdir -p "$EVAL_DIR"
DATE=$(date +%Y-%m-%d)
EVAL_FILE="$EVAL_DIR/$DATE.md"
Write the template with today's date
cat > "$EVAL_FILE" << 'EOF'
PA Weekly Eval — DATE_PLACEHOLDER
[Fill in the template above]
EOF
Replace the placeholder with the real date (works on Linux and macOS)
sed -i "s/DATE_PLACEHOLDER/$DATE/" "$EVAL_FILE" 2>/dev/null \
  || sed -i '' "s/DATE_PLACEHOLDER/$DATE/" "$EVAL_FILE"
echo "Created eval file: $EVAL_FILE"

---

Owner Feedback Signals

Log these automatically when detected:

| Signal | Action | |---|---| | 👍 reaction | Log +1 positive | | 👎 reaction | Log -1 negative, record the correction | | "תודה" / "great" / "perfect" | Log +1 positive | | "wrong" / "fix this" / "לא טוב" | Log -1, record the correction | | Owner re-asks the same question | Log -1 memory gap | | Owner does the task themselves | Log -1 initiative gap | | Owner surprised by proactive action | Log +2 proactivity |

Rule: If a signal appears → log it immediately. Don't batch feedback signals.

---

Monthly Report Format

# PA Performance Report — [Month Year]
PA Name: [Name]
Owner: [Owner Name]
Period: [Start] – [End]
Summary Score: X/40 ([Grade A/B/C/D])
Dimension Breakdown
[Copy scores table here]
Key Wins
-
Key Issues
-
Trend vs Last Period
Score change: +X / -X points
Best improvement: [dimension]
Biggest regression: [dimension]
Recommended Actions
1.
2.
3.

---

Benchmark Tests (Run Monthly)

Task Completion Rate

Count tasks assigned in last 30 days.

Count completed without follow-up.

Formula: completed / assigned × 100%

Target: >90%

Accuracy Rate

Count tasks that required correction.

Formula: (tasks - corrections) / tasks × 100%

Target: >95%

Memory Retention

Ask about something discussed 7+ days ago.

Pass if recalled correctly, Fail if missed.

Target: >80%

---

Cost Tips

Cheap: Filling in the weekly template — any small model works.

Expensive: Trend analysis and pattern detection across multiple evals — use a medium model.

Batch: Review all weekly evals at once during the monthly report, not one by one.

Avoid: Don't re-score historical weeks — score in real time and save to file.