Web Scraper Trae
by @zhengjia626
Opens browser and scrapes webpage content using Playwright. Invoke when user wants to crawl/scrape a webpage, extract data from a website, or get content fro...
clawhub install web-scraper-traeπ About This Skill
name: "@web-scraper-trae" description: "Opens browser and scrapes webpage content using Playwright. Invoke when user wants to crawl/scrape a webpage, extract data from a website, or get content from a URL." allowed-tools: Bash(node:*)
Web Scraper Trae
Opens a browser using Playwright and scrapes webpage content.
Prerequisites
npm install playwright
npx playwright install chromium
Usage
When user provides a URL, create a Node.js script to scrape the page:
const { chromium } = require('playwright');async function scrape(url) {
const browser = await chromium.launch({
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox']
});
const page = await browser.newPage();
await page.goto(url, { waitUntil: 'networkidle', timeout: 60000 });
const title = await page.title();
const text = await page.textContent('body');
const html = await page.content();
await browser.close();
return { title, text, html, url };
}
const url = process.argv[2];
if (!url) {
console.error('θ―·ζδΎ URL εζ°');
process.exit(1);
}
scrape(url).then(result => {
console.log('=== SCRAPE_RESULT ===');
console.log(JSON.stringify(result, null, 2));
}).catch(err => {
console.error('η¬εε€±θ΄₯:', err.message);
process.exit(1);
});
Execution
Run the script with:
node scrape.js "https://example.com"
Output Format
Return JSON with:
title: Page titletext: Visible text content (HTML stripped)html: Full HTML sourceurl: Original URLNotes
headless: true for server environmentswaitUntil: 'networkidle' to ensure full page loadplaywright-cli skill insteadπ‘ Examples
When user provides a URL, create a Node.js script to scrape the page:
const { chromium } = require('playwright');async function scrape(url) {
const browser = await chromium.launch({
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox']
});
const page = await browser.newPage();
await page.goto(url, { waitUntil: 'networkidle', timeout: 60000 });
const title = await page.title();
const text = await page.textContent('body');
const html = await page.content();
await browser.close();
return { title, text, html, url };
}
const url = process.argv[2];
if (!url) {
console.error('θ―·ζδΎ URL εζ°');
process.exit(1);
}
scrape(url).then(result => {
console.log('=== SCRAPE_RESULT ===');
console.log(JSON.stringify(result, null, 2));
}).catch(err => {
console.error('η¬εε€±θ΄₯:', err.message);
process.exit(1);
});
βοΈ Configuration
npm install playwright
npx playwright install chromium
π Tips & Best Practices
headless: true for server environmentswaitUntil: 'networkidle' to ensure full page loadplaywright-cli skill instead