๐ŸŽ Get the FREE AI Skills Starter Guide โ€” Subscribe โ†’
BytesAgainBytesAgain
๐Ÿฆ€ ClawHub

Smart Web Search

by @davidme6

Smart Web Search v3.1 - Intelligent search with Baidu (China) and Google (International) as primary engines. Real-time news, ad filtering, content de-toxicat...

Versionv3.1.0
Downloads567
Installs3
Starsโญ 1
TERMINAL
clawhub install smart-web-search

๐Ÿ“– About This Skill


name: smart-web-search description: Smart Web Search v3.1 - Intelligent search with Baidu (China) and Google (International) as primary engines. Real-time news, ad filtering, content de-toxication, multi-engine aggregation. Auto-detects query language for accurate results. version: 3.1.0 author: Jarvis triggers: - search - find - web search - real-time news - latest news - today news - latest - verified search - safe search - filter ads - no ads - search with Baidu - search with Google

๐ŸŒ Smart Web Search v3.1

Primary Engines: Baidu (China) + Google (International)

Core Features:

  • โœ… Real-time Search - Today/24h/7d/30d time filtering
  • โœ… Time Filtering - Specify search time range
  • โœ… Multi-engine Aggregation - Search 3-5 engines simultaneously
  • โœ… Smart De-duplication - Auto-filter duplicate content
  • โœ… AI Summaries - Auto-generated key insights
  • โœ… Ad Filtering - Auto-detect and filter ads/promotions
  • โœ… Content De-toxication - Filter spam, fake news, misinformation
  • โœ… Baidu Integration - China's largest search engine (Primary)
  • โœ… Google Integration - World's most accurate engine (Primary)
  • Auto-selects optimal search engine based on query content.


    ๐Ÿง  Intelligent Detection Logic

    Use Baidu Search (China) When:

  • ๐Ÿ‡จ๐Ÿ‡ณ Query contains Chinese characters
  • ๐Ÿ‡จ๐Ÿ‡ณ Searching China-related topics (WeChat, Taobao, Baidu, etc.)
  • ๐Ÿ‡จ๐Ÿ‡ณ Searching Chinese companies/people/events
  • ๐Ÿ‡จ๐Ÿ‡ณ Searching Chinese policies/news/regulations
  • Use Google Search (International) When:

  • ๐ŸŒ Query is pure English
  • ๐ŸŒ Searching international topics (GitHub, Stack Overflow, Reddit, etc.)
  • ๐ŸŒ Searching foreign companies/people/events
  • ๐ŸŒ Searching technical docs/academic papers/international news

  • ๐Ÿ”ง Tool Calls

    Domestic Search (Chinese Queries)

    Primary Engine (Baidu):

    web_fetch(url="https://www.baidu.com/s?wd=query", extractMode="text", maxChars=15000)
    

    Advanced Search:

    # Exact match
    web_fetch(url="https://www.baidu.com/s?wd=\"exact+keyword\"", extractMode="text", maxChars=15000)

    Exclude site

    web_fetch(url="https://www.baidu.com/s?wd=keyword+-site:zhihu.com", extractMode="text", maxChars=15000)

    File type

    web_fetch(url="https://www.baidu.com/s?wd=keyword+filetype:pdf", extractMode="text", maxChars=15000)

    Real-time News:

    # Latest 24 hours
    web_fetch(url="https://www.baidu.com/s?wd=query+2026-03-17&src=news", extractMode="text", maxChars=15000)

    Latest 7 days

    web_fetch(url="https://www.baidu.com/s?wd=query+this+week&src=news", extractMode="text", maxChars=15000)

    Backup Engines (by priority): 1. 360 Search: https://m.so.com/s?q=query 2. Sogou WeChat: https://weixin.sogou.com/weixin?type=2&query=query 3. Bing CN: https://cn.bing.com/search?q=query 4. Sogou Web: https://www.sogou.com/web?query=query 5. Toutiao: https://www.toutiao.com/search/?keyword=query

    Baidu Advantages:

  • โœ… Largest Chinese index (trillions of pages)
  • โœ… Best localization for China
  • โœ… Fastest update speed (minute-level)
  • โœ… Advanced search syntax support
  • โœ… Integrated with Baidu Baike, Zhidao, Tieba
  • International Search (English Queries)

    Primary Engine (Google via Startpage):

    web_fetch(url="https://www.startpage.com/do/search?q=query", extractMode="text", maxChars=15000)
    

    Direct Google (if accessible):

    web_fetch(url="https://www.google.com/search?q=query", extractMode="text", maxChars=15000)
    

    Real-time News:

    # Latest 24 hours
    web_fetch(url="https://www.startpage.com/do/search?q=query+2026-03-17", extractMode="text", maxChars=15000)

    Latest 7 days

    web_fetch(url="https://www.startpage.com/do/search?q=query+this+week", extractMode="text", maxChars=15000)

    Backup Engines (by priority): 1. DuckDuckGo: https://lite.duckduckgo.com/lite/?q=query 2. DuckDuckGo News: https://duckduckgo.com/?q=query&ia=news 3. Qwant: https://www.qwant.com/?q=query&t=web 4. Bing EN: https://www.bing.com/search?q=query 5. Brave Search: https://search.brave.com/search?q=query

    Google Advantages:

  • โœ… World's largest index (hundreds of billions of pages)
  • โœ… Most advanced search algorithm
  • โœ… Richest academic resources
  • โœ… Best multi-language support
  • โœ… Powerful advanced search features

  • ๐Ÿ“‹ Usage Examples

    Chinese Queries

    Basic Search:

    User: Search NVIDIA latest financial report
    โ†’ Auto-use: Baidu Search
    โ†’ Time range: Auto-detect "latest" โ†’ 7 days
    

    Real-time News:

    User: Today March 17 latest news
    โ†’ Auto-use: Baidu News Search
    โ†’ Time range: 2026-03-17 (today)
    โ†’ Extra parameter: &src=news
    

    WeChat Articles:

    User: Find WeChat articles about AI
    โ†’ Auto-use: Sogou WeChat Search
    โ†’ Search type: WeChat official accounts
    

    Specified Time:

    User: Search AI news last 24 hours
    โ†’ Auto-use: Baidu Search + time filter
    โ†’ Time range: 2026-03-16 to 2026-03-17
    

    English Queries

    Basic Search:

    User: search for latest AI news
    โ†’ Auto-use: Google (via Startpage)
    โ†’ Time range: Auto-detect "latest" โ†’ 7 days
    

    Real-time News:

    User: today news March 17 2026
    โ†’ Auto-use: Google News
    โ†’ Time range: 2026-03-17 (today)
    

    Technical Content:

    User: find Python tutorials on GitHub
    โ†’ Auto-use: Google
    โ†’ Priority sources: GitHub, Stack Overflow
    

    Specified Time:

    User: latest tech news this week
    โ†’ Auto-use: Google
    โ†’ Time parameter: this week
    

    Mixed Queries

    International Company:

    User: Tesla TSLA stock price today
    โ†’ Priority: Google (international company)
    โ†’ Backup: Baidu (Chinese results)
    

    China Product:

    User: WeChat how to export data
    โ†’ Priority: Baidu (domestic product)
    โ†’ Backup: Google (English tutorials)
    

    Real-time Search (v3.1 Recommended)

    Today's News:

    "Search today March 17 AI news"
    "NVIDIA latest news (24 hours)"
    "latest AI news today March 17"
    "2026-03-17 AI latest news"
    

    This Week:

    "Search this week AI news"
    "AI latest news this week"
    "Last 7 days AI updates"
    


    ๐ŸŽฏ Search Strategy v3.1

    Step 1: Language Detection

  • Contains Chinese characters โ†’ Baidu (China)
  • Pure English/Latin โ†’ Google (International)
  • Mixed language โ†’ Judge by topic
  • Step 2: Topic Detection

  • China companies/products/policies โ†’ Baidu
  • International companies/tech/academic โ†’ Google
  • News/current events โ†’ Dual-engine parallel
  • Step 3: Time Detection

    Auto-recognize time keywords:
  • "Today", "ไปŠๆ—ฅ", "today" โ†’ 2026-03-17
  • "Yesterday", "ๆ˜จๅคฉ" โ†’ 2026-03-16
  • "Last 24h", "ๆœ€่ฟ‘ 24 ๅฐๆ—ถ" โ†’ Past 24 hours
  • "This week", "ๆœฌๅ‘จ" โ†’ Current week
  • "Latest", "ๆœ€ๆ–ฐ" โ†’ 7 days
  • "This month", "ๆœฌๆœˆ" โ†’ 30 days
  • No time keyword โ†’ Default 30 days

    Step 4: Safety Detection

    1. Detect if query involves sensitive topics
       - Medical/health โ†’ Enable strict de-tox
       - Finance/investment โ†’ Enable scam filter
       - News โ†’ Enable fake news filter

    2. Set filter level - Normal search โ†’ Standard filter - "Verified", "safe" โ†’ Strict filter - "Academic", "research" โ†’ Authoritative source priority

    Step 5: Engine Selection

    Domestic Priority: 1. Baidu (Largest Chinese engine) - PRIMARY 2. 360 Search (General) 3. 360 News (Real-time news) 4. Sogou WeChat (Official accounts) 5. Bing CN (Backup) 6. Toutiao (Breaking news)

    International Priority: 1. Google via Startpage (Most accurate) - PRIMARY 2. DuckDuckGo (Privacy) 3. DuckDuckGo News (News) 4. Qwant (European) 5. Bing EN (Backup) 6. Brave Search (Privacy)

    Step 6: Multi-engine Aggregation

    Search 3 engines simultaneously:
    Primary: Baidu/Google
    Backup 1: 360/Qwant
    Backup 2: Bing
    

    Merge Strategy:

  • Collect top 10 results from each
  • De-duplicate (URL + title similarity)
  • Filter ads and toxic content
  • Sort by time descending
  • Keep 5-8 best results
  • Step 7: Search + Fetch

    1. Search 3 engines simultaneously 2. Merge and de-duplicate results 3. Filter ads (auto-detect) 4. De-toxicate (verify sources) 5. Select 3-5 most relevant URLs 6. Fetch detailed content with web_fetch 7. Generate AI summary


    โš ๏ธ Best Practices

    For Latest Information

    โœ… "Search today March 17 XXX news"
    โœ… "XXX latest news (24 hours)"
    โœ… "latest XXX news today"
    โŒ "Search XXX" (no time range, may return old news)
    

    For High Accuracy

    โœ… "March 17 2026 XXX latest news"
    โœ… "XXX + 2026-03-17"
    โœ… "XXX breaking news"
    โŒ "XXX news" (unclear time range)
    

    For Safe Search

    โœ… "Search vaccine info (verified)"
    โœ… "XXX safe search"
    โœ… "verified XXX info"
    โŒ Direct search for medical/finance (may encounter misinformation)
    

    For Baidu Search

    โœ… "Search with Baidu XXX"
    โœ… "XXX site:baidu.com"
    โœ… "XXX filetype:pdf"
    

    For Google Search

    โœ… "Search with Google XXX"
    โœ… "XXX site:github.com"
    โœ… "XXX filetype:pdf"
    


    ๐Ÿ›ก๏ธ Ad Filtering

    Auto-detect Ads

    Ad Characteristics:

    Ad Labels = [
        "ๅนฟๅ‘Š", "ๆŽจๅนฟ", "Sponsored", "Ad", "Promoted",
        "็ซžไปทๆŽ’ๅ", "ๅ“็‰Œๅฑ•็คบ", "ๅ•†ไธšๆŽจๅนฟ"
    ]

    Ad Positions = [ "Top of results", "Right sidebar", "Bottom recommendations", "Related searches", "Guess you want" ]

    Filtering Rules

    1. Label Filtering:

    โœ… Keep: Organic search results
    โŒ Filter: Results marked as "Ad", "Sponsored", "ๅนฟๅ‘Š", "ๆŽจๅนฟ"
    

    2. Position Filtering:

    โœ… Keep: Natural results (middle positions)
    โŒ Filter: Top 3, right sidebar, bottom recommendations
    

    3. Domain Filtering:

    โŒ Filter domains blacklist:
    
  • Known ad farms
  • Low-quality content sites
  • Content scrapers
  • โœ… Priority domains whitelist:

  • Government sites (.gov.cn)
  • Educational institutions (.edu)
  • Mainstream media
  • Known tech companies
  • Usage Examples

    Enable Ad Filtering:

    "Search AI news (filter ads)"
    "search AI news (no ads)"
    "XXX clean search"
    

    Filter Results:

    Raw results: 20 items
    Ad count: 6 items (30%)
    After filter: 14 items (100% organic)
    


    ๐Ÿงช Content De-toxication

    Toxin Detection

    Content Toxin Types:

    | Type | Characteristics | Action | |------|----------------|--------| | ๐Ÿ—‘๏ธ Spam | Duplicate, keyword stuffing | Filter | | ๐Ÿ“ข Clickbait | "Shocking", "Must-see", "#1" | Down-rank | | ๐Ÿ“ฐ Fake News | No source, no author, no date | Filter | | ๐Ÿ’Š Medical Rumors | Folk remedies, miracle cures, 100% cure rate | Filter + Warning | | ๐Ÿ’ฐ Financial Scams | High returns, guaranteed profit, insider info | Filter + Warning | | ๐Ÿ”ž Inappropriate | Adult, violence, gambling | Filter | | ๐ŸŽญ Deepfakes | AI-generated fake content | Mark + Warning |

    De-tox Algorithm

    1. Source Verification:

    Trusted Sources = [
        "Government sites", "Mainstream media", "Academic journals",
        "Known company websites", "Authoritative organizations"
    ]

    if Source not in Trusted Sources: Credibility Score -= 30%

    2. Content Quality Assessment:

    Quality Metrics = [
        "Has author", "Has date", "Has source citations",
        "Clear logic", "Verifiable data"
    ]

    if Quality Score < 60: Mark as "Low Quality"

    3. Fact-checking:

    # Cross-verify multiple sources
    if Only single source reports:
        Mark as "Unverified"

    if Multiple authoritative sources confirm: Credibility += 50%

    4. Sentiment Analysis:

    if Contains extreme emotion words > 10:
        Mark as "Sensational content"
        Credibility -= 40%
    

    Warning System

    Warning Levels:

    | Level | Icon | Trigger | Action | |-------|------|---------|--------| | ๐ŸŸข Trusted | โœ… | Authoritative + verified | Normal display | | ๐ŸŸก Questionable | โš ๏ธ | Single source/medium quality | Mark display | | ๐ŸŸ  Suspicious | โ— | Low quality/clickbait | Down-rank + Warning | | ๐Ÿ”ด Toxic | ๐Ÿšซ | Fake/scam | Filter + Warning |

    Usage Examples

    Enable De-toxication:

    "Search vaccine info (verified)"
    "search vaccine info (de-toxic)"
    "XXX safe search"
    "XXX filter misinformation"
    

    De-tox Results:

    Raw results: 20 items
    Toxic content: 5 items (25%)
      - Fake info: 2 items
      - Clickbait: 2 items
      - Low quality: 1 item
    After filter: 15 items (100% trusted)
    


    ๐Ÿ“Š Engine Comparison

    | Engine | Region | Free | API Key | Accuracy | Features | |--------|--------|------|---------|----------|----------| | Baidu | ๐Ÿ‡จ๐Ÿ‡ณ China | โœ… | โŒ | โญโญโญโญโญ | Largest Chinese index | | Google | ๐ŸŒ Global | โœ… | โŒ | โญโญโญโญโญ | Most accurate globally | | 360 Search | ๐Ÿ‡จ๐Ÿ‡ณ China | โœ… | โŒ | โญโญโญโญ | Safe & clean | | DuckDuckGo | ๐ŸŒ Global | โœ… | โŒ | โญโญโญโญ | Privacy-focused | | Startpage | ๐ŸŒ Global | โœ… | โŒ | โญโญโญโญโญ | Google results + privacy | | Qwant | ๐ŸŒ Europe | โœ… | โŒ | โญโญโญโญ | European engine | | Bing CN | ๐Ÿ‡จ๐Ÿ‡ณ China | โœ… | โŒ | โญโญโญโญ | Stable & reliable | | Bing EN | ๐ŸŒ Global | โœ… | โŒ | โญโญโญโญ | Stable backup |


    ๐Ÿš€ Quick Start

    Basic Search

    "Search XXX"
    "search for XXX"
    "Help me find XXX"
    "find XXX"
    

    Real-time Search (v3.1 Recommended)

    Get Today's Latest:

    "Search today March 17 XXX news"
    "XXX latest news (24 hours)"
    "latest XXX news today March 17"
    "2026-03-17 XXX latest news"
    

    Get This Week:

    "Search this week XXX news"
    "XXX latest news this week"
    "Last 7 days XXX updates"
    

    Safe Search (v3.1 New)

    De-toxic Search:

    "Search vaccine info (verified)"
    "XXX safe search"
    "verified XXX info"
    "XXX filter misinformation"
    

    Filter Ads:

    "Search XXX (filter ads)"
    "XXX no ads"
    "XXX clean search"
    

    Specify Engine

    Use Baidu:

    "Search with Baidu XXX"
    "XXX site:baidu.com"
    

    Use Google:

    "Search with Google XXX"
    "XXX site:github.com"
    

    Advanced Search

    Specify Time Range:

    "Search March 16-17 2026 XXX"
    "XXX news from 2026-03-16 to 2026-03-17"
    

    Specify Source:

    "Search WeChat articles about XXX"
    "find XXX on GitHub"
    "search XXX on Reddit"
    


    ๐Ÿ“Š v3.1 Changelog

    New Features

  • โœ… Baidu as primary China engine (was 360)
  • โœ… Google as primary International engine (via Startpage)
  • โœ… Advanced search syntax support
  • โœ… Real-time search (today/24h/7d/30d)
  • โœ… Multi-engine aggregation (3 engines)
  • โœ… Smart de-duplication (>80% similarity filter)
  • โœ… AI summaries (key insights)
  • โœ… Time keyword auto-recognition
  • โœ… News source priority (real-time news engines)
  • Improvements

  • โœ… Domestic engines: 3 โ†’ 6 (+100%)
  • โœ… International engines: 3 โ†’ 6 (+100%)
  • โœ… Timeout optimization (15s auto-skip)
  • โœ… Cache optimization (30min/real-time exception)
  • โœ… Result ranking (time descending)
  • โœ… Accuracy improvement (Baidu + Google)
  • โœ… Safety improvement (ad filter + de-tox)
  • โœ… All documentation in English
  • Engine Updates

    Domestic New:
  • โœ… Baidu (Primary - Largest Chinese engine)
  • โœ… Toutiao (Breaking news)
  • International New:

  • โœ… Google via Startpage (Primary - Most accurate)
  • โœ… Brave Search (Privacy)

  • ๐Ÿ”ฎ Roadmap (v4.0)

  • [ ] Vertical search (academic/images/videos/products)
  • [ ] AI summary upgrade (multi-language)
  • [ ] Search subscription (RSS push)
  • [ ] Custom search engines
  • [ ] Search history analysis
  • [ ] Batch search (multi-keyword parallel)
  • [ ] Fact-check API integration
  • [ ] Multi-modal search (text+image)

  • *Last updated: 2026-03-17 14:50* *Version: v3.1.0* *Status: โœ… Production Ready* *Primary Engines: Baidu (China) + Google (International)*