🎁 Get the FREE AI Skills Starter Guide β€” Subscribe β†’
BytesAgainBytesAgain
BytesAgain β€Ί Skills β€Ί Super Lobster
πŸ¦€ ClawHub

Super Lobster

by @mrherojack

Performs aggressive web research and data extraction via local scripts, browser rendering, crawling, and command execution on a China-networked gateway.

Versionv1.0.0
Downloads290
#research#writing#legal
TERMINAL
clawhub install super-lobster

πŸ“– About This Skill

Super Lobster

Use this skill when tasks require aggressive web research, browser rendering, local scripting, crawling, extraction, and command execution on this gateway.

Operating model

  • This host is in a China network environment. Do not assume western search engines or API endpoints are reachable or stable.
  • Prefer first-party sites, direct URLs, mirrors, and browser rendering over public search APIs.
  • Prefer local execution on the gateway for scraping, parsing, coding, and automation.

    • Default workflow

    1. If a direct URL is known, fetch it with fetch_url.py. 2. If only the readable article body matters, use extract_main_text.py. 3. If the page is JS-heavy or renders differently in browsers, use render_url.py or chrome_dump_dom.sh. 4. If you need to discover more pages inside the same site, use crawl_site.py. 5. For multi-step processing, write a Python script under /root/.openclaw/workspace/memory/tmp and run it.

    Tools

  • /root/.openclaw/workspace/skills/super-lobster/bin/fetch_url.py
    • Returns metadata plus a text and HTML preview.
  • /root/.openclaw/workspace/skills/super-lobster/bin/extract_main_text.py
    • Extracts the readable main content.
  • /root/.openclaw/workspace/skills/super-lobster/bin/chrome_dump_dom.sh
    • Dumps browser-rendered DOM with Chrome headless.
  • /root/.openclaw/workspace/skills/super-lobster/bin/render_url.py
    • Python wrapper around Chrome headless DOM rendering.
  • /root/.openclaw/workspace/skills/super-lobster/bin/crawl_site.py --limit 20
    • Same-site link discovery crawl.

    Browser rules

  • Prefer fetch_url.py or extract_main_text.py first for static pages.
  • Use browser rendering only when static fetch is incomplete or JS-dependent.
  • Do not depend on OpenClaw's built-in browser RPC in this environment unless explicitly verified working in the current session.

    • Coding and execution rules

  • You may write and execute Python or shell programs locally on the gateway.
  • Prefer Python for scraping, parsing, and data cleanup.
  • Keep scratch outputs in /root/.openclaw/workspace/memory/tmp.
  • Remove clearly temporary files after use unless they are likely to be reused.

    • Network rules

  • Avoid default reliance on DuckDuckGo, Tavily, Exa, or similar public search providers unless reachability is verified first.
  • If provider keys are added later and connectivity is stable, treat them as accelerators, not hard dependencies.