🎁 Get the FREE AI Skills Starter Guide β€” Subscribe β†’
BytesAgainBytesAgain
BytesAgain β€Ί Skills β€Ί Canary Deploy
πŸ¦€ ClawHub

Canary Deploy

by @lolaopenclaw

Safe system changes with automatic baseline capture, canary testing, and rollback for critical infrastructure modifications. Use when making changes to SSH c...

Versionv1.0.0
Downloads660
#clawhub
TERMINAL
clawhub install canary-deploy

πŸ“– About This Skill

name: canary-deploy description: Safe system changes with automatic baseline capture, canary testing, and rollback for critical infrastructure modifications. Use when making changes to SSH config, firewall rules, network settings, systemd services, kernel parameters, or any system change that could break remote access. Prevents lockouts by validating connectivity before and after changes. Born from a real incident where AllowTcpForwarding=no killed VPN tunnel access.

Canary Deploy

Safe system changes with pre-flight checks, validation, and automatic rollback.

The Problem

System changes can lock you out:

  • SSH hardening breaks remote access
  • Firewall rules block needed ports
  • Kernel parameters cause instability
  • Service restarts break dependencies

    • Recovery without physical access is painful or impossible.

    Quick Start

    Before any critical change

    # Capture baseline (connectivity, services, ports)
bash scripts/canary-test.sh baseline
    Make your change
    
sudo nano /etc/ssh/sshd_config
    Validate change didn't break anything
    
bash scripts/canary-test.sh validate
    If validation fails:
    
bash scripts/canary-test.sh rollback

    For automated changes

    # Full pipeline: baseline β†’ apply β†’ validate β†’ rollback-if-failed
bash scripts/critical-update.sh \
  --name "SSH hardening" \
  --backup "/etc/ssh/sshd_config" \
  --command "sudo sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config && sudo systemctl reload sshd" \
  --validate "ssh -o ConnectTimeout=5 localhost echo ok"

    Protocol A+B (Manual Workflow)

    For interactive sessions where you want human-in-the-loop:

    Protocol A: Test interactively

    1. Tell the human: "Open a second SSH session as backup" 2. Apply change in the first session 3. Ask: "Test connectivity from the second session" 4. If it works β†’ confirm 5. If it fails β†’ rollback from the backup session

    Protocol B: Backup first

    1. Run bash scripts/canary-test.sh baseline 2. Verify backup is valid 3. Apply change 4. Run bash scripts/canary-test.sh validate 5. If validation fails β†’ bash scripts/canary-test.sh rollback

    Always use both A + B together for maximum safety.

    What Gets Checked

    Baseline capture

  • SSH connectivity (local + remote)
  • Open ports (ss -tlnp)
  • Running services (systemctl)
  • Firewall rules (ufw/iptables)
  • Network routes
  • DNS resolution
  • Config file checksums

    • Validation

  • All baseline checks re-run
  • Diff against baseline
  • Any regression = FAIL

    • Critical Change Categories

    | Category | Risk | Example | Recovery | |----------|------|---------|----------| | SSH config | πŸ”΄ HIGH | sshd_config changes | Backup session | | Firewall | πŸ”΄ HIGH | UFW/iptables rules | Pre-change snapshot | | Network | πŸ”΄ HIGH | Interface/routing changes | Console access | | Services | 🟑 MEDIUM | systemd unit changes | systemctl restart | | Kernel params | 🟑 MEDIUM | sysctl changes | Reboot to defaults | | Packages | 🟒 LOW | apt install/upgrade | apt rollback |

    References

    See references/incident-report.md for the real incident that inspired this skill.

    πŸ’‘ Examples

    Before any critical change

    # Capture baseline (connectivity, services, ports)
bash scripts/canary-test.sh baseline
    Make your change
    
sudo nano /etc/ssh/sshd_config
    Validate change didn't break anything
    
bash scripts/canary-test.sh validate
    If validation fails:
    
bash scripts/canary-test.sh rollback

    For automated changes

    # Full pipeline: baseline β†’ apply β†’ validate β†’ rollback-if-failed
bash scripts/critical-update.sh \
  --name "SSH hardening" \
  --backup "/etc/ssh/sshd_config" \
  --command "sudo sed -i 's/PermitRootLogin yes/PermitRootLogin no/' /etc/ssh/sshd_config && sudo systemctl reload sshd" \
  --validate "ssh -o ConnectTimeout=5 localhost echo ok"

    πŸ”’ Constraints

  • All baseline checks re-run
  • Diff against baseline
  • Any regression = FAIL