Structured Multi-Agent Deliberation
by @tatsuko-tsukimi
Provides a structured multi-agent deliberation framework with role schemas (action/guardian/observer/critic), verification protocols, and stopping criteria....
clawhub install structured-deliberationπ About This Skill
name: structured-deliberation description: Provides a structured multi-agent deliberation framework with role schemas (action/guardian/observer/critic), verification protocols, and stopping criteria. Activate when designing multi-agent systems for non-trivial deliberation, instrumenting agent debates against sycophancy, or seeking an evaluable alternative to free-form group prompting. version: 0.1.0
Structured Multi-Agent Deliberation Framework
A protocol for running multi-agent deliberation that produces *evaluable* output β claims with explicit lifecycle, verifications with cross-agent evidence, decisions that cite specific support, and stopping criteria that prevent both premature consensus and infinite loops.
The central premise: free-form multi-agent prompting produces consensus that looks robust but isn't. Without structural cross-validation, agents drift toward agreement (sycophancy cascade); without explicit claims, decisions can't be traced to evidence; without stopping criteria, deliberations either stop too early (premature consensus) or run forever (artifact accumulation).
This framework provides:
1. 4 role schemas (Action / Guardian / Observer / Critic) β each contributing a perspective the others can't substitute 2. 4 cross-validation checks per round β artifacts are not siloed; they must contact each other 3. Claims + verifications infrastructure β disagreements become testable claims with status lifecycles 4. 6 goal-driven stopping criteria β explicit signals for when the deliberation should end 5. Stress test protocol β forced agent absence reveals which roles are load-bearing
When to use
Activate this skill when:
claims.jsonl / verifications.jsonl / decisions.jsonl audit trailsDon't activate when:
The 4-role structure
| Role | Question | Artifact | |---|---|---| | Action | "Where's the verb?" | Behavioral Trace Set | | Guardian | "What's the invariant?" | Identity Invariant Check | | Observer | "Can a real operator navigate this?" | Operator Walkthrough | | Critic | "What's the strongest argument against this?" | Adversarial Test Case |
Full role definitions: references/role-schemas.md.
Why these 4: each contributes a perspective the others *can't substitute*. Action grounds in state transitions; Guardian formalizes invariants; Observer tests usability; Critic forces adversarial scrutiny. Without one, the deliberation has a structural blind spot.
Round structure (8 phases)
Each round runs through:
A. Lead Assignment β who leads this round
B. Lead Proposal β 600-800 word position
C. Supplements β other agents' takes (200-400 words each)
D. Task Phase β 4 structured artifacts
E. Cross-Validation β 4 verification checks
F. Claims Update β new claims, status changes, validate
G. Assessment β decisions, tensions, architecture impact
H. (every K rounds) Drift Check
Full template: templates/round-template.md.tmpl.
Cross-validation: 4 checks
After artifacts are produced (Phase D), 4 verification checks run before claims update:
1. Critic attack vs Action trace β does Critic's attack break Action's traces? 2. Guardian invariant vs Observer walkthrough β does Operator path violate the invariant? 3. Observer friction vs Critic attack β friendly-fire overlap (most valuable signal) 4. All artifacts vs prior decisions β continuity / drift detection
Each check produces a verifications.jsonl entry with evidence_refs β₯2 and a verdict (BROKEN / ROBUST / UNCLEAR / NOT_APPLICABLE).
Full protocol: references/verification-protocol.md.
Claims infrastructure
Disagreements become testable claims:
{
"id": "C-{round}-{sequence}",
"round": N,
"raised_by": "action | guardian | observer | critic",
"text": "the claim itself, 1-3 sentences",
"testable_as": "single observable outcome",
"status": "pending | tested_confirmed | tested_refuted | partially_refuted | tested_unclear | superseded"
}
Status transitions are driven by verification verdicts. Decisions cite claims and verifications.
Full schema and validator: references/claims-infrastructure.md, scripts/claims-validator.py.
Stopping: 6 signals
The deliberation should consider stopping when β₯4 of 6 signals are satisfied:
1. Claim refutation rate stabilizes 2. Disagreement slope flat or rising (no sycophancy) 3. All agents have led at least one non-stress round 4. Stress tests have been executed (β₯N-1 agents stress-tested) 5. Drift checks pass 6. Pending claim fraction < 30%
Full criteria + detector script: references/stopping-criteria.md, scripts/stopping-detector.py.
Stress tests
Forced agent absence reveals load-bearing-ness. If a stress round looks identical to a normal round, the absent agent's role is decorative.
Recommended schedule for a 13-round deliberation:
In stress rounds, the absent agent's artifact is NOT produced. Verifications involving the absent agent's artifact return NOT_APPLICABLE with stress reason in notes.
Failure modes to watch
8 common failure modes with detection signals:
Full catalog with detection signals and responses: references/failure-modes.md.
How to use this skill
When triggered, walk the user through 5 stages:
Stage 1 β Domain elicitation
Ask:
Stage 2 β Role configuration
For each of the 4 roles, instantiate for the user's domain:
Use templates/role-prompt.md.tmpl as starting prompts.
Stage 3 β Round template configuration
Configure:
Use templates/round-template.md.tmpl as the per-round structure.
Stage 4 β Run rounds
For each round:
Use scripts/round-controller.py as orchestration skeleton (adapt to your LLM API).
Stage 5 β Synthesis and retrospective
When stop signals satisfied (β₯4/6) OR round budget hit:
Quick example
User: *"I want to evaluate 3 candidate architectures for our agent system, but free-form discussion keeps converging too fast β I think we're sycophanting."*
Stage 1 reveals:
Stage 2 instantiates 4 roles:
Stage 3 sets:
Stage 4 runs 13 rounds. After R7, sycophancy detector triggers (Signal 2 negative slope). Response: tighten Critic schema, R8 produces a BROKEN verdict on Action's R3 trace.
Stage 5 synthesis: Architecture B emerges as winner not because of consensus, but because:
Full walkthrough: examples/condensed-deliberation.md.
How the skill behaves at each turn
claims-validator.py) after every Phase F. Errors here cascade.References
Templates
Scripts
Examples
What this skill does NOT do
round-controller.py is a skeleton; you integrate with your LLM APISource
This skill captures a methodology developed across multiple multi-agent deliberation experiments, frozen as v1 after running on multiple calibration cases. The 4-role structure, 4 cross-validation checks, 6 stopping signals, and stress test patterns all emerged from calibration findings.
Companion skill: multi-dim-eval-framework β for *evaluating* deliberation quality (MADEF). The two skills compose: this skill produces deliberation; that skill evaluates it.
License
MIT