Fox Veille

Skill Veille - RSS Aggregator

RSS feed aggregator with URL deduplication and topic-based deduplication for OpenClaw agents. Fetches articles from 20+ configured sources, filters already-seen URLs (TTL 14 days), and deduplicates articles covering the same story using Jaccard similarity + named entities.

No external dependencies: stdlib Python only (urllib, xml.etree, email.utils).

Trigger phrases

Quick Start

# 1. Setup
python3 scripts/setup.py
2. Validate
python3 scripts/init.py
3. Fetch + Score + Send (full pipeline)
python3 scripts/veille.py fetch --filter-seen --filter-topic \
  | python3 scripts/veille.py score \
  | python3 scripts/veille.py send

Setup

Requirements

Installation

# From the skill directory
python3 scripts/setup.py
Validate
python3 scripts/init.py

The wizard creates:

Customizing sources

Edit ~/.openclaw/config/veille/config.json and add/remove entries in the "sources" dict:

{
  "sources": {
    "My Blog": "https://example.com/feed.xml",
    "BleepingComputer": "https://www.bleepingcomputer.com/feed/"
  }
}

Storage and credentials

Files written by this skill

| Path | Written by | Purpose | Contains secrets | |------|-----------|---------|-----------------| | ~/.openclaw/config/veille/config.json | setup.py | Sources, outputs, options | NO | | ~/.openclaw/data/veille/seen_urls.json | veille.py | URL dedup store (TTL 14d) | NO | | ~/.openclaw/data/veille/topic_seen.json | veille.py | Topic dedup store (TTL 5d) | NO |

Files read from outside the skill

| Path | Read by | Key accessed | When | |------|---------|-------------|------| | ~/.openclaw/openclaw.json | dispatch.py | channels.telegram.botToken (read-only) | Only when telegram_bot output is enabled and no bot_token is set in the output config |

This is the only cross-config read. To avoid it entirely, set bot_token explicitly in your output config:

{ "type": "telegram_bot", "bot_token": "YOUR_BOT_TOKEN", "chat_id": "...", "enabled": true }

Output credentials (optional)

Credentials are only used if you enable the corresponding output. None are required for core functionality (RSS fetch + dedup).

| Output | Credential source | What is used | |--------|-----------------|-------------| | telegram_bot | ~/.openclaw/openclaw.json or bot_token in output config | Bot token (read-only) | | mail-client | Delegated to mail-client skill (its own creds) | Nothing read directly | | mail-client (SMTP fallback) | smtp_user / smtp_pass in output config | SMTP login | | nextcloud | Delegated to nextcloud-files skill (its own creds) | Nothing read directly |

Cleanup on uninstall

python3 scripts/setup.py --cleanup

Security model

Credential isolation

Subprocess boundaries

File output safety

Cross-config reads

Autonomous dispatch

CLI reference

fetch

python3 veille.py fetch [--hours N] [--filter-seen] [--filter-topic] [--sources FILE]

Options:

Output (JSON on stdout):

{
  "hours": 24,
  "count": 42,
  "skipped_url": 5,
  "skipped_topic": 3,
  "articles": [...],
  "wrapped_listing": "=== UNTRUSTED EXTERNAL CONTENT ..."
}

seen-stats

python3 veille.py seen-stats

Shows URL seen store statistics (count, TTL, file path).

topic-stats

python3 veille.py topic-stats

Shows topic deduplication store statistics.

mark-seen

python3 veille.py mark-seen URL [URL ...]

Marks one or more URLs as already seen (prevents them from appearing in future fetches with --filter-seen).

score

python3 veille.py score [--dry-run]

Reads a digest JSON from stdin (output of fetch) and scores articles using an OpenAI-compatible LLM. Returns enriched JSON with scored, ghost_picks, and per-article score/reason fields.

Options:

When llm.enabled is false (default), articles pass through unchanged ("scored": false).

Pipeline usage:

python3 veille.py fetch --filter-seen --filter-topic | python3 veille.py score | python3 veille.py send

send

python3 veille.py send [--profile NAME]

Reads a digest JSON from stdin and dispatches to all enabled outputs configured in config.json. Accepts both raw fetch output (articles key) and LLM-processed digests (categories key).

Output types: telegram_bot, mail-client, nextcloud, file.

Configure outputs interactively:

python3 scripts/setup.py --manage-outputs

config

python3 veille.py config

Prints the active configuration (no secrets).

LLM scoring configuration

The llm key in config.json controls the optional LLM-based article scoring:

{
  "llm": {
    "enabled": false,
    "base_url": "https://api.openai.com/v1",
    "api_key_file": "~/.openclaw/secrets/openai_api_key",
    "model": "gpt-4o-mini",
    "top_n": 10,
    "ghost_threshold": 5
  }
}

| Key | Default | Description | |-----|---------|-------------| | enabled | false | Enable LLM scoring (requires API key) | | base_url | https://api.openai.com/v1 | OpenAI-compatible API endpoint | | api_key_file | ~/.openclaw/secrets/openai_api_key | Path to file containing the API key | | model | gpt-4o-mini | Model to use for scoring | | top_n | 10 | Max articles to send to LLM per batch | | ghost_threshold | 5 | Score threshold for ghost_picks (blog-worthy articles) |

Scoring rules:

When disabled, the score subcommand passes data through unchanged.

Nextcloud output mode

The nextcloud output now defaults to append mode with a date separator. Each dispatch adds content below a ## YYYY-MM-DD HH:MM header, preserving previous entries.

Set "mode": "overwrite" in the output config to restore the old behavior:

{ "type": "nextcloud", "path": "/Veille/digest.md", "mode": "overwrite" }

File output configuration

The file output writes digests to the local filesystem. By default, only paths under ~/.openclaw/ are allowed. To authorize additional directories, use config.security.allowed_output_dirs:

{
  "security": {
    "allowed_output_dirs": [
      "~/Documents/veille",
      "/srv/digests"
    ]
  }
}

Blocked paths (always rejected, even if inside an allowed directory): .ssh, .gnupg, .config/systemd, crontab, /etc/, .bashrc, .profile, .bash_profile, .zshrc, .env

Content validation — written content is rejected if it:

All blocked attempts are logged to stderr with the reason.

Templates (agent usage)

Basic digest

# In agent tool call:
result = exec("python3 scripts/veille.py fetch --hours 24 --filter-seen --filter-topic")
data = json.loads(result.stdout)
data["wrapped_listing"] is ready for LLM prompt injection
data["count"] = number of new articles
data["articles"] = list of article dicts

Prompt template

You are a news analyst. Here are today's articles:
{data["wrapped_listing"]}
Please summarize the 5 most important stories, focusing on security and tech.

Agent workflow example

1. Call veille fetch --filter-seen --filter-topic
2. Pipe through veille score (LLM scoring, if enabled)
3. If count > 0: pass wrapped_listing to LLM for analysis
4. LLM produces digest summary
5. Pipe through veille send (dispatches to configured outputs)

Pipeline (CLI)

python3 scripts/veille.py fetch --filter-seen --filter-topic \
  | python3 scripts/veille.py score \
  | python3 scripts/veille.py send

Filtering by keyword (post-fetch)

data = json.loads(fetch_output)
security_articles = [
    a for a in data["articles"]
    if any(kw in a["title"].lower() for kw in ["cve", "vuln", "patch", "breach"])
]

Ideas

Combine with

  veille fetch --filter-seen | ... | mail-client send
  

  veille fetch --filter-seen | jq .wrapped_listing -r > /tmp/digest.md
  nextcloud-files upload /tmp/digest.md /Digests/$(date +%Y-%m-%d).md
  

Troubleshooting

See references/troubleshooting.md for detailed troubleshooting steps.

Common issues:

# 1. Setup
python3 scripts/setup.py
2. Validate
python3 scripts/init.py
3. Fetch + Score + Send (full pipeline)
python3 scripts/veille.py fetch --filter-seen --filter-topic \
  | python3 scripts/veille.py score \
  | python3 scripts/veille.py send

Requirements

Installation

# From the skill directory
python3 scripts/setup.py
Validate
python3 scripts/init.py

The wizard creates:

Customizing sources

Edit ~/.openclaw/config/veille/config.json and add/remove entries in the "sources" dict:

{
  "sources": {
    "My Blog": "https://example.com/feed.xml",
    "BleepingComputer": "https://www.bleepingcomputer.com/feed/"
  }
}

See references/troubleshooting.md for detailed troubleshooting steps.

Common issues: