🎁 Get the FREE AI Skills Starter Guide β€” Subscribe β†’
BytesAgainBytesAgain
BytesAgain β€Ί Skills β€Ί Firecrawl
πŸ¦€ ClawHub

Firecrawl

by @byungkyu

Firecrawl API integration with managed authentication. Scrape, crawl, map, and search web content. Use this skill when users want to extract content from web...

Versionv1.0.2
Downloads2,304
Installs10
Stars⭐ 1
#security#writing#legal#travel#productivity
TERMINAL
clawhub install firecrawl-api

πŸ“– About This Skill

name: firecrawl description: | Firecrawl API integration with managed authentication. Scrape, crawl, map, and search web content. Use this skill when users want to extract content from websites, crawl entire sites, map URLs, or search the web. For other third party apps, use the api-gateway skill (https://clawhub.ai/byungkyu/api-gateway). compatibility: Requires network access and valid Maton API key metadata: author: maton version: "1.0" clawdbot: emoji: 🧠 homepage: "https://maton.ai" requires: env: - MATON_API_KEY

Firecrawl

Access the Firecrawl API with managed authentication. Scrape webpages, crawl entire websites, map site URLs, and search the web with full content extraction.

Quick Start

# Scrape a webpage
python <<'EOF'
import urllib.request, os, json
data = json.dumps({"url": "https://example.com", "formats": ["markdown"]}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/scrape', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Base URL

https://api.maton.ai/firecrawl/{native-api-path}

Maton proxies requests to api.firecrawl.dev and automatically injects your API key.

Authentication

All requests require the Maton API key in the Authorization header:

Authorization: Bearer $MATON_API_KEY

Environment Variable: Set your API key as MATON_API_KEY:

export MATON_API_KEY="YOUR_API_KEY"

Getting Your API Key

1. Sign in or create an account at maton.ai 2. Go to maton.ai/settings 3. Copy your API key

Connection Management

Manage your Firecrawl connections at https://api.maton.ai.

List Connections

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections?app=firecrawl&status=ACTIVE')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Create Connection

python <<'EOF'
import urllib.request, os, json
data = json.dumps({'app': 'firecrawl'}).encode()
req = urllib.request.Request('https://api.maton.ai/connections', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Get Connection

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections/{connection_id}')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Response: 

{
  "connection": {
    "connection_id": "{connection_id}",
    "status": "ACTIVE",
    "creation_time": "2026-03-11T09:49:09.917114Z",
    "last_updated_time": "2026-03-11T09:49:27.616143Z",
    "url": "https://connect.maton.ai/?session_token=...",
    "app": "firecrawl",
    "metadata": {},
    "method": "API_KEY"
  }
}

Delete Connection

python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections/{connection_id}', method='DELETE')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

Specifying Connection

If you have multiple Firecrawl connections, specify which one to use with the Maton-Connection header:

python <<'EOF'
import urllib.request, os, json
data = json.dumps({"url": "https://example.com"}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/scrape', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
req.add_header('Maton-Connection', '{connection_id}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

If you have multiple connections, always include this header to ensure requests go to the intended account.

Security & Permissions

  • Access is scoped to web scraping, crawling, site mapping, structured extraction, and browser sessions within the connected Firecrawl account.
  • All operations require explicit user approval. Scrape, crawl, map, search, extract, and agent operations all consume Firecrawl credits. Before executing any request, confirm the target URLs, scope (e.g., crawl limit, maxDepth), and intended effect with the user.
  • Browser actions and custom headers require extra caution. The actions parameter (click, write, execute JavaScript) and headers parameter can interact with websites beyond passive reading. Always confirm with the user before using these options.
  • Large crawls can consume significant credits. Always set a reasonable limit and confirm with the user before starting crawl or batch operations.

    • API Reference

    Scrape

    POST /firecrawl/v2/scrape

    Extract content from a single webpage.

    Required Parameters:

  • url (string): The webpage URL to scrape

    • Optional Parameters:

  • formats (array): Output formats - "markdown", "html", "json", "screenshot", "links" (default: ["markdown"])
  • onlyMainContent (boolean): Extract only main content, exclude headers/footers (default: true)
  • includeTags (array): HTML tags to include
  • excludeTags (array): HTML tags to exclude
  • waitFor (integer): Milliseconds to wait before scraping (default: 0)
  • timeout (integer): Request timeout in ms (default: 30000, max: 300000)
  • mobile (boolean): Emulate mobile device (default: false)
  • actions (array): Browser actions to perform before scraping
  • headers (object): Custom HTTP headers
  • blockAds (boolean): Block ads and cookie banners (default: true)

    • Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "url": "https://docs.firecrawl.dev",
    "formats": ["markdown", "html"],
    "onlyMainContent": True,
    "waitFor": 1000
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/scrape', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "data": {
    "markdown": "# Example Domain\n\nThis domain is for use in documentation...",
    "metadata": {
      "title": "Example Domain",
      "language": "en",
      "sourceURL": "https://example.com",
      "url": "https://example.com/",
      "statusCode": 200,
      "contentType": "text/html",
      "creditsUsed": 1
    }
  }
}

    Crawl (Start)

    POST /firecrawl/v2/crawl

    Start crawling an entire website. Returns a crawl ID for status polling.

    Required Parameters:

  • url (string): The base URL to start crawling from

    • Optional Parameters:

  • limit (integer): Maximum pages to crawl (default: 10000)
  • maxDepth (integer): Maximum crawl depth
  • includePaths (array): Regex patterns for URLs to include
  • excludePaths (array): Regex patterns for URLs to exclude
  • allowSubdomains (boolean): Enable subdomain crawling
  • allowExternalLinks (boolean): Follow external links
  • scrapeOptions (object): Options for each page scrape (formats, onlyMainContent, etc.)
  • webhook (string): Webhook URL for completion notification

    • Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "url": "https://example.com",
    "limit": 10,
    "scrapeOptions": {
        "formats": ["markdown"]
    }
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/crawl', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "id": "019cdc53-0acf-76ec-a80c-3ead753b2730",
  "url": "https://api.firecrawl.dev/v1/crawl/019cdc53-0acf-76ec-a80c-3ead753b2730"
}

    Crawl (Get Status)

    GET /firecrawl/v2/crawl/{id}

    Get the status and results of a crawl job.

    Path Parameters:

  • id (string): The crawl job ID

    • Example: 

    python <<'EOF'
import urllib.request, os, json
crawl_id = "019cdc53-0acf-76ec-a80c-3ead753b2730"
req = urllib.request.Request(f'https://api.maton.ai/firecrawl/v2/crawl/{crawl_id}')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "status": "completed",
  "completed": 2,
  "total": 2,
  "creditsUsed": 2,
  "expiresAt": "2026-03-12T09:56:00.000Z",
  "data": [
    {
      "markdown": "# Example Domain\n\nThis domain is for use in documentation...",
      "metadata": {
        "title": "Example Domain",
        "sourceURL": "https://example.com",
        "statusCode": 200
      }
    }
  ]
}

    Status Values:

  • scraping - Crawl in progress
  • completed - Crawl finished successfully
  • failed - Crawl failed

    • Crawl (Cancel)

    DELETE /firecrawl/v2/crawl/{id}

    Cancel an in-progress crawl job.

    Path Parameters:

  • id (string): The crawl job ID

    • Example: 

    python <<'EOF'
import urllib.request, os, json
crawl_id = "019cdc53-0acf-76ec-a80c-3ead753b2730"
req = urllib.request.Request(f'https://api.maton.ai/firecrawl/v2/crawl/{crawl_id}', method='DELETE')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "status": "cancelled"
}

    Map

    POST /firecrawl/v2/map

    Get all URLs from a website without scraping content.

    Required Parameters:

  • url (string): The starting URL

    • Optional Parameters:

  • search (string): Query to order results by relevance
  • limit (integer): Maximum links to return (default: 5000, max: 100000)
  • includeSubdomains (boolean): Include subdomains (default: true)
  • sitemap (string): Sitemap handling - "skip", "include", "only" (default: "include")
  • ignoreQueryParameters (boolean): Exclude URLs with query params (default: true)
  • timeout (integer): Timeout in milliseconds

    • Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "url": "https://docs.firecrawl.dev",
    "limit": 100,
    "includeSubdomains": False
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/map', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "links": [
    "https://docs.firecrawl.dev",
    "https://docs.firecrawl.dev/api-reference",
    "https://docs.firecrawl.dev/introduction"
  ]
}

    Search

    POST /firecrawl/v2/search

    Search the web and get full page content for each result.

    Required Parameters:

  • query (string): Search query (max 500 characters)

    • Optional Parameters:

  • limit (integer): Number of results (default: 5, max: 100)
  • sources (array): Search types - "web", "images", "news" (default: ["web"])
  • country (string): ISO country code (default: "US")
  • location (string): Geographic targeting (e.g., "Germany")
  • tbs (string): Time filter - "qdr:d" (day), "qdr:w" (week), "qdr:m" (month), "qdr:y" (year)
  • timeout (integer): Timeout in ms (default: 60000)
  • scrapeOptions (object): Options for content extraction

    • Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "query": "web scraping best practices",
    "limit": 5,
    "scrapeOptions": {
        "formats": ["markdown"]
    }
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/search', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "data": [
    {
      "url": "https://example.com/article",
      "title": "Web Scraping Best Practices",
      "description": "Learn the best practices for web scraping...",
      "markdown": "# Web Scraping Best Practices\n\n..."
    }
  ],
  "creditsUsed": 5
}

    Batch Scrape (Start)

    POST /firecrawl/v2/batch/scrape

    Scrape multiple URLs in a single batch job.

    Required Parameters:

  • urls (array): List of URLs to scrape

    • Optional Parameters:

  • formats (array): Output formats (default: ["markdown"])
  • onlyMainContent (boolean): Extract only main content (default: true)
  • webhook (string): Webhook URL for completion notification

    • Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "urls": ["https://example.com", "https://example.org"],
    "formats": ["markdown"]
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/batch/scrape', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "id": "019cdc59-56b9-7096-a9f9-95fcc92a3a75",
  "url": "https://api.firecrawl.dev/v1/batch/scrape/019cdc59-56b9-7096-a9f9-95fcc92a3a75"
}

    Batch Scrape (Get Status)

    GET /firecrawl/v2/batch/scrape/{id}

    Get the status and results of a batch scrape job.

    Path Parameters:

  • id (string): The batch scrape job ID

    • Example: 

    python <<'EOF'
import urllib.request, os, json
batch_id = "019cdc59-56b9-7096-a9f9-95fcc92a3a75"
req = urllib.request.Request(f'https://api.maton.ai/firecrawl/v2/batch/scrape/{batch_id}')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "status": "completed",
  "completed": 2,
  "total": 2,
  "creditsUsed": 2,
  "expiresAt": "2026-03-12T10:02:54.000Z",
  "data": [
    {
      "markdown": "# Example Domain\n\n...",
      "metadata": {
        "title": "Example Domain",
        "sourceURL": "https://example.com",
        "statusCode": 200
      }
    }
  ]
}

    Batch Scrape (Cancel)

    DELETE /firecrawl/v2/batch/scrape/{id}

    Cancel an in-progress batch scrape job.

    Path Parameters:

  • id (string): The batch scrape job ID

    • Batch Scrape (Get Errors)

    GET /firecrawl/v2/batch/scrape/{id}/errors

    Get errors from a batch scrape job.

    Path Parameters:

  • id (string): The batch scrape job ID

    • Response: 

    {
  "errors": [],
  "robotsBlocked": []
}

    Crawl (Get Errors)

    GET /firecrawl/v2/crawl/{id}/errors

    Get errors from a crawl job.

    Path Parameters:

  • id (string): The crawl job ID

    • Example: 

    python <<'EOF'
import urllib.request, os, json
crawl_id = "019cdc53-0acf-76ec-a80c-3ead753b2730"
req = urllib.request.Request(f'https://api.maton.ai/firecrawl/v2/crawl/{crawl_id}/errors')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "errors": [],
  "robotsBlocked": []
}

    Crawl (Get Active)

    GET /firecrawl/v2/crawl/active

    Get all active crawl jobs.

    Example: 

    python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/crawl/active')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "crawls": []
}

    Extract (Start)

    POST /firecrawl/v2/extract

    Extract structured data from URLs using AI.

    Required Parameters:

  • urls (array): List of URLs to extract from
  • prompt (string): Natural language description of what to extract

    • Optional Parameters:

  • schema (object): JSON schema for structured output
  • scrapeOptions (object): Options for scraping

    • Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "urls": ["https://example.com"],
    "prompt": "Extract the main heading and description"
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/extract', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "id": "019cdc59-977b-774b-b584-af2af45c055b",
  "urlTrace": []
}

    Extract (Get Status)

    GET /firecrawl/v2/extract/{id}

    Get the status and results of an extract job.

    Path Parameters:

  • id (string): The extract job ID

    • Example: 

    python <<'EOF'
import urllib.request, os, json
extract_id = "019cdc59-977b-774b-b584-af2af45c055b"
req = urllib.request.Request(f'https://api.maton.ai/firecrawl/v2/extract/{extract_id}')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "data": [
    {
      "heading": "Example Domain",
      "description": "This domain is for use in documentation..."
    }
  ],
  "status": "completed",
  "expiresAt": "2026-03-11T16:03:05.000Z"
}

    Browser (Create Session)

    POST /firecrawl/v2/browser

    Create an interactive browser session for manual control via CDP.

    Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/browser', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "id": "019cdc5d-5c9d-732e-a7bd-f095a96a2bb1",
  "cdpUrl": "wss://browser.firecrawl.dev/cdp/...",
  "liveViewUrl": "https://liveview.firecrawl.dev/...",
  "interactiveLiveViewUrl": "https://liveview.firecrawl.dev/...",
  "expiresAt": "2026-03-11T10:17:12.409Z"
}

    Browser (List Sessions)

    GET /firecrawl/v2/browser

    List all active browser sessions.

    Example: 

    python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/browser')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "sessions": [
    {
      "id": "019cdc5d-5c9d-732e-a7bd-f095a96a2bb1",
      "status": "active",
      "cdpUrl": "wss://browser.firecrawl.dev/cdp/...",
      "liveViewUrl": "https://liveview.firecrawl.dev/..."
    }
  ]
}

    Browser (Delete Session)

    DELETE /firecrawl/v2/browser/{id}

    Delete a browser session.

    Path Parameters:

  • id (string): The browser session ID

    • Agent (Start)

    POST /firecrawl/v2/agent

    Start an AI agent to autonomously navigate and extract data.

    Required Parameters:

  • prompt (string): Description of what data to extract (max 10,000 chars)

    • Optional Parameters:

  • urls (array): URLs to constrain the agent to
  • schema (object): JSON schema for structured output
  • maxCredits (integer): Maximum credits to use (default: 2500)
  • strictConstrainToURLs (boolean): Only visit provided URLs
  • model (string): "spark-1-mini" (default, cheaper) or "spark-1-pro" (higher accuracy)

    • Example: 

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "prompt": "Find the pricing information",
    "urls": ["https://example.com"],
    "model": "spark-1-mini"
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/agent', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "id": "019cdc5d-a2d4-728c-9c91-e9eae475568f"
}

    Agent (Get Status)

    GET /firecrawl/v2/agent/{id}

    Get the status and results of an agent job.

    Path Parameters:

  • id (string): The agent job ID

    • Example: 

    python <<'EOF'
import urllib.request, os, json
agent_id = "019cdc5d-a2d4-728c-9c91-e9eae475568f"
req = urllib.request.Request(f'https://api.maton.ai/firecrawl/v2/agent/{agent_id}')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Response: 

    {
  "success": true,
  "status": "completed",
  "model": "spark-1-pro",
  "data": {...},
  "expiresAt": "2026-03-12T10:07:30.055Z"
}

    Agent (Cancel)

    DELETE /firecrawl/v2/agent/{id}

    Cancel an in-progress agent job.

    Path Parameters:

  • id (string): The agent job ID

    • Browser Actions

    Use actions parameter to interact with pages before scraping:

    python <<'EOF'
import urllib.request, os, json
data = json.dumps({
    "url": "https://example.com",
    "formats": ["markdown", "screenshot"],
    "actions": [
        {"type": "wait", "milliseconds": 2000},
        {"type": "click", "selector": "#load-more"},
        {"type": "scroll", "direction": "down", "amount": 500},
        {"type": "screenshot"}
    ]
}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/scrape', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Available Actions:

  • wait - Wait for specified milliseconds
  • click - Click an element by CSS selector
  • write - Type text into an input field
  • scroll - Scroll the page
  • screenshot - Take a screenshot
  • execute - Run custom JavaScript

    • Code Examples

    JavaScript

    const response = await fetch('https://api.maton.ai/firecrawl/v2/scrape', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': Bearer ${process.env.MATON_API_KEY}
  },
  body: JSON.stringify({
    url: 'https://example.com',
    formats: ['markdown']
  })
});
const data = await response.json();
console.log(data.data.markdown);

    Python

    import os
import requests
    response = requests.post(
    'https://api.maton.ai/firecrawl/v2/scrape',
    headers={'Authorization': f'Bearer {os.environ["MATON_API_KEY"]}'},
    json={
        'url': 'https://example.com',
        'formats': ['markdown']
    }
)
data = response.json()
print(data['data']['markdown'])

    Notes

  • Scrape uses 1 credit per page (basic proxy)
  • Enhanced proxy for anti-bot sites uses up to 5 credits
  • Crawl results expire after 24 hours
  • Maximum timeout is 300,000ms (5 minutes)
  • Use onlyMainContent: true to get cleaner output without navigation/footer
  • IMPORTANT: When piping curl output to jq or other commands, environment variables like $MATON_API_KEY may not expand correctly in some shell environments

    • Error Handling

    | Status | Meaning | |--------|---------| | 400 | Missing Firecrawl connection or invalid parameters | | 401 | Invalid or missing Maton API key | | 402 | Firecrawl credits exhausted | | 409 | Conflict (e.g., crawl already completed) | | 429 | Rate limited | | 4xx/5xx | Passthrough error from Firecrawl API |

    Troubleshooting: API Key Issues

    1. Check that the MATON_API_KEY environment variable is set:

    echo $MATON_API_KEY

    2. Verify the API key is valid by listing connections:

    python <<'EOF'
import urllib.request, os, json
req = urllib.request.Request('https://api.maton.ai/connections')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    Troubleshooting: Invalid App Name

    1. Ensure your URL path starts with firecrawl. For example:

  • Correct: https://api.maton.ai/firecrawl/v2/scrape
  • Incorrect: https://api.maton.ai/v2/scrape

    • Resources

  • Firecrawl API Documentation
  • Firecrawl Dashboard
  • Maton Community
  • Maton Support

    • πŸ’‘ Examples

    # Scrape a webpage
python <<'EOF'
import urllib.request, os, json
data = json.dumps({"url": "https://example.com", "formats": ["markdown"]}).encode()
req = urllib.request.Request('https://api.maton.ai/firecrawl/v2/scrape', data=data, method='POST')
req.add_header('Authorization', f'Bearer {os.environ["MATON_API_KEY"]}')
req.add_header('Content-Type', 'application/json')
print(json.dumps(json.load(urllib.request.urlopen(req)), indent=2))
EOF

    πŸ“‹ Tips & Best Practices

  • Scrape uses 1 credit per page (basic proxy)
  • Enhanced proxy for anti-bot sites uses up to 5 credits
  • Crawl results expire after 24 hours
  • Maximum timeout is 300,000ms (5 minutes)
  • Use onlyMainContent: true to get cleaner output without navigation/footer
  • IMPORTANT: When piping curl output to jq or other commands, environment variables like $MATON_API_KEY may not expand correctly in some shell environments