🎁 Get the FREE AI Skills Starter Guide β€” Subscribe β†’
BytesAgainBytesAgain
πŸ¦€ ClawHub

Data Harvester Pro

by @windy-001-crypto

Batch web scraping for competitor analysis, price monitoring and market research

Versionv1.0.0
Downloads667
TERMINAL
clawhub install data-harvester-v2

πŸ“– About This Skill


name: data-harvester-v2 description: Batch web scraping for competitor analysis, price monitoring and market research command-dispatch: tool command-tool: data_harvester command-arg-mode: raw metadata: {"openclaw": {"emoji": "πŸ§‘β€πŸŒΎ", "requires": {"bins": ["python3"]}}}

πŸ§‘β€πŸŒΎ Data Harvester - Batch Scraping & Competitor Analysis

Professional-grade batch web scraping tool for competitor monitoring, price tracking, and market research.

Features

  • Batch URL Scraping: Process multiple URLs from file
  • Competitor Comparison: Compare products across platforms
  • Stock Data: Real-time stock quotes and analysis
  • Sector Trends: Hot sectors and market trends
  • News Aggregation: Collect news by keywords
  • Commands

    | Command | Description | Example | |---------|-------------|---------| | stock | Query stock data | /data-harvester stock 600519 | | block | Hot market sectors | /data-harvester block | | fund | Money flow tracking | /data-harvester fund | | news | News search | /data-harvester news AI | | compare | Competitor comparison | /data-harvester compare laptop | | batch | Batch URL list | /data-harvester batch urls.txt | | export | Export data | /data-harvester export data.json excel |

    Usage Examples

    Stock Query

    /data-harvester stock 000001
    
    Returns: Real-time stock price, change %, volume

    Competitor Comparison

    /data-harvester compare "wireless earphones"
    
    Returns: Price comparison across Taobao, JD, Pinduoduo

    Sector Analysis

    /data-harvester block
    
    Returns: Top 7hot sectors with leader stocks

    Fund Flow

    /data-harvester fund
    
    Returns: Top 5 stocks with main fund inflow

    Batch Processing

    Create a text file with one URL per line:

    https://example1.com
    https://example2.com
    https://example3.com
    

    Then run:

    /data-harvester batch urls.txt
    

    Technical Details

  • Language: Python 3
  • Output Formats: JSON, CSV, Excel
  • Rate Limiting: Configurable delay between requests
  • Legal Notice

  • Respect target website's robots.txt
  • Use reasonable request intervals (3-5 seconds)
  • For educational/research purposes only
  • Commercial use requires proper authorization