Local-First LLM
by @joelnishanth
Routes LLM requests to a local model (Ollama, LM Studio, llamafile) before falling back to cloud APIs. Tracks token savings and cost avoidance in a persisten...
clawhub install local-first-llmπ About This Skill
name: local-first-llm description: "Routes LLM requests to a local model (Ollama, LM Studio, llamafile) before falling back to cloud APIs. Tracks token savings and cost avoidance in a persistent dashboard. Use when: (1) user asks to run a task with a local model first, (2) user wants to reduce cloud API costs or keep requests private, (3) user asks to see their token savings or LLM routing dashboard, (4) any request where local-vs-cloud routing should be decided automatically. Supports Ollama, LM Studio, and llamafile as local providers." metadata: { "openclaw": { "emoji": "π ", "requires": { "bins": ["python3"] }, "install": [] } }
Local-First LLM
Route requests to a local LLM first; fall back to cloud only when necessary. Track every decision to show real token and cost savings.
Quick Start
1. Check if a local LLM is running
python3 skills/local-first-llm/scripts/check_local.py
Returns JSON: { "any_available": true, "best": { "provider": "ollama", "models": [...] } }
2. Route a request
python3 skills/local-first-llm/scripts/route_request.py \
--prompt "Summarize this meeting transcript" \
--tokens 800 \
--local-available \
--local-provider ollama
Returns: { "decision": "local", "reason": "...", "complexity_score": -1 }
3. Log the outcome
After executing the request, record it:
python3 skills/local-first-llm/scripts/track_savings.py log \
--tokens 800 \
--model gpt-4o \
--routed-to local
4. Show the dashboard
python3 skills/local-first-llm/scripts/dashboard.py
Full Routing Workflow
βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β 1. check_local.py β is a local provider running? β
β β
β 2. route_request.py β local or cloud? β
β - sensitivity check (private data β local) β
β - complexity score (high score β cloud) β
β - availability gate (no local β cloud) β
β β
β 3. Execute with the chosen provider β
β β
β 4. track_savings.py log β record the outcome β
β β
β 5. dashboard.py β show cumulative savings β
βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
Routing Rules (Summary)
| Condition | Route |
| ----------------------------------------------------------------------------- | -------- |
| No local provider available | βοΈ Cloud |
| Prompt contains sensitive data (password, secret, api key, ssn, etc.) | π Local |
| Complexity score β₯ 3 | βοΈ Cloud |
| Complexity score < 3 | π Local |
For full scoring details, see references/routing-logic.md.
Executing with a Local Provider
Once route_request.py returns "decision": "local", send the request:
Ollama
curl http://localhost:11434/api/generate \
-d '{"model": "llama3.2", "prompt": "YOUR_PROMPT", "stream": false}'
LM Studio / llamafile (OpenAI-compatible)
curl http://localhost:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "local-model", "messages": [{"role": "user", "content": "YOUR_PROMPT"}]}'
Dashboard
The dashboard reads from ~/.openclaw/local-first-llm/savings.json (auto-created).
βββββββββββββββββββββββββββββββββββββββββββ
β π§ Local-First LLM β Dashboard β
βββββββββββββββββββββββββββββββββββββββββββ€
β Local LLM: β
ollama (llama3.2...) β
βββββββββββββββββββββββββββββββββββββββββββ€
β Total requests: 42 β
β Routed locally: 31 (73.8%) β
β Routed to cloud: 11 β
βββββββββββββββββββββββββββββββββββββββββββ€
β Tokens saved: 84,200 β
β Cost saved: $0.4210 β
βββββββββββββββββββββββββββββββββββββββββββ
Reset savings data:
python3 skills/local-first-llm/scripts/track_savings.py reset
Additional References
π‘ Examples
1. Check if a local LLM is running
python3 skills/local-first-llm/scripts/check_local.py
Returns JSON: { "any_available": true, "best": { "provider": "ollama", "models": [...] } }
2. Route a request
python3 skills/local-first-llm/scripts/route_request.py \
--prompt "Summarize this meeting transcript" \
--tokens 800 \
--local-available \
--local-provider ollama
Returns: { "decision": "local", "reason": "...", "complexity_score": -1 }
3. Log the outcome
After executing the request, record it:
python3 skills/local-first-llm/scripts/track_savings.py log \
--tokens 800 \
--model gpt-4o \
--routed-to local
4. Show the dashboard
python3 skills/local-first-llm/scripts/dashboard.py