Doubleword API

Doubleword Batch Inference

Process multiple AI inference requests asynchronously using the Doubleword batch API with high throughput and low cost.

Prerequisites

Before submitting batches, you need: 1. Doubleword Account - Sign up at https://app.doubleword.ai/ 2. API Key - Create one in the API Keys section of your dashboard 3. Account Credits - Add credits to process requests (see pricing below)

When to Use Batches

Batches are ideal for:

Available Models & Pricing

Pricing is per 1 million tokens (input / output):

Qwen3-VL-30B-A3B-Instruct-FP8 (mid-size):

Qwen3-VL-235B-A22B-Instruct-FP8 (flagship):

Cost estimation: Upload files to the Doubleword Console to preview expenses before submitting.

Quick Start

Two ways to submit batches:

Via API: 1. Create JSONL file with requests 2. Upload file to get file ID 3. Create batch using file ID 4. Poll status until complete 5. Download results from output_file_id

Via Web Console: 1. Navigate to Batches section at https://app.doubleword.ai/ 2. Upload JSONL file 3. Configure batch settings (model, completion window) 4. Monitor progress in real-time dashboard 5. Download results when ready

Workflow

Step 1: Create Batch Request File

Create a .jsonl file where each line contains a complete, valid JSON object with no line breaks within the object:

{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "anthropic/claude-3-5-sonnet", "messages": [{"role": "user", "content": "What is 2+2?"}]}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "anthropic/claude-3-5-sonnet", "messages": [{"role": "user", "content": "What is the capital of France?"}]}}

Required fields per line:

Optional body parameters:

File requirements:

Common pitfalls:

Helper script: Use scripts/create_batch_file.py to generate JSONL files programmatically:

python scripts/create_batch_file.py output.jsonl

Modify the script's requests list to generate your specific batch requests.

Step 2: Upload File

Via API:

curl https://api.doubleword.ai/v1/files \
  -H "Authorization: Bearer $DOUBLEWORD_API_KEY" \
  -F purpose="batch" \
  -F file="@batch_requests.jsonl"

Via Console: Upload through the Batches section at https://app.doubleword.ai/

Response contains id field - save this file ID for next step.

Step 3: Create Batch

Via API:

curl https://api.doubleword.ai/v1/batches \
  -H "Authorization: Bearer $DOUBLEWORD_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "input_file_id": "file-abc123",
    "endpoint": "/v1/chat/completions",
    "completion_window": "24h"
  }'

Via Console: Configure batch settings in the web interface.

Parameters:

Response contains batch id - save this for status polling.

Before submitting, verify:

Step 4: Poll Status

Via API:

curl https://api.doubleword.ai/v1/batches/batch-xyz789 \
  -H "Authorization: Bearer $DOUBLEWORD_API_KEY"

Via Console: Monitor real-time progress in the Batches dashboard.

Status progression: 1. validating - Checking input file format 2. in_progress - Processing requests 3. completed - All requests finished

Other statuses:

Response includes:

Polling frequency: Check every 30-60 seconds during processing.

Early access: Results available via output_file_id before batch fully completes - check X-Incomplete header.

Step 5: Download Results

Via API:

curl https://api.doubleword.ai/v1/files/file-output123/content \
  -H "Authorization: Bearer $DOUBLEWORD_API_KEY" \
  > results.jsonl

Via Console: Download results directly from the Batches dashboard.

Response headers:

Output format (each line):

{
  "id": "batch-req-abc",
  "custom_id": "request-1",
  "response": {
    "status_code": 200,
    "body": {
      "id": "chatcmpl-xyz",
      "choices": [{
        "message": {
          "role": "assistant",
          "content": "The answer is 4."
        }
      }]
    }
  }
}

Download errors (if any):

curl https://api.doubleword.ai/v1/files/file-error123/content \
  -H "Authorization: Bearer $DOUBLEWORD_API_KEY" \
  > errors.jsonl

Error format (each line):

{
  "id": "batch-req-def",
  "custom_id": "request-2",
  "error": {
    "code": "invalid_request",
    "message": "Missing required parameter"
  }
}

Tool Calling in Batches

Tool calling (function calling) enables models to intelligently select and use external tools. Doubleword maintains full OpenAI compatibility.

Example batch request with tools:

{
  "custom_id": "tool-req-1",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "anthropic/claude-3-5-sonnet",
    "messages": [{"role": "user", "content": "What's the weather in Paris?"}],
    "tools": [{
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather for a location",
        "parameters": {
          "type": "object",
          "properties": {
            "location": {"type": "string"}
          },
          "required": ["location"]
        }
      }
    }]
  }
}

Use cases:

Structured Outputs in Batches

Structured outputs guarantee that model responses conform to your JSON Schema, eliminating issues with missing fields or invalid enum values.

Example batch request with structured output:

{
  "custom_id": "structured-req-1",
  "method": "POST",
  "url": "/v1/chat/completions",
  "body": {
    "model": "anthropic/claude-3-5-sonnet",
    "messages": [{"role": "user", "content": "Extract key info from: John Doe, 30 years old, lives in NYC"}],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "person_info",
        "schema": {
          "type": "object",
          "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "city": {"type": "string"}
          },
          "required": ["name", "age", "city"]
        }
      }
    }
  }
}

Benefits:

autobatcher: Automatic Batching

autobatcher is a Python client that automatically converts individual API calls into batched requests, reducing costs without code changes.

Installation:

pip install autobatcher

How it works: 1. Collection Phase: Requests accumulate during a time window (default: 1 second) or until batch size threshold 2. Batch Submission: Collected requests are submitted together 3. Result Polling: System monitors for completed responses 4. Transparent Response: Your code receives standard ChatCompletion responses

Key benefit: Significant cost reduction through automatic batching while writing normal async code using the familiar OpenAI interface.

Documentation: https://github.com/doublewordai/autobatcher

Additional Operations

List All Batches

Via API:

curl https://api.doubleword.ai/v1/batches?limit=10 \
  -H "Authorization: Bearer $DOUBLEWORD_API_KEY"

Via Console: View all batches in the dashboard.

Cancel Batch

Via API:

curl https://api.doubleword.ai/v1/batches/batch-xyz789/cancel \
  -X POST \
  -H "Authorization: Bearer $DOUBLEWORD_API_KEY"

Via Console: Click cancel in the batch details view.

Notes:

Common Patterns

Processing Results

Parse JSONL output line-by-line:

import json
with open('results.jsonl') as f:
    for line in f:
        result = json.loads(line)
        custom_id = result['custom_id']
        content = result['response']['body']['choices'][0]['message']['content']
        print(f"{custom_id}: {content}")

Handling Partial Results

Check for incomplete batches and resume:

import requests
response = requests.get(
    'https://api.doubleword.ai/v1/files/file-output123/content',
    headers={'Authorization': f'Bearer {api_key}'}
)
if response.headers.get('X-Incomplete') == 'true':
    last_line = int(response.headers.get('X-Last-Line', 0))
    print(f"Batch incomplete. Processed {last_line} requests so far.")
    # Continue polling and download again later

Retry Failed Requests

Extract failed requests from error file and resubmit:

import json
failed_ids = []
with open('errors.jsonl') as f:
    for line in f:
        error = json.loads(line)
        failed_ids.append(error['custom_id'])
print(f"Failed requests: {failed_ids}")
Create new batch with only failed requests

Processing Tool Calls

Handle tool call responses:

import json
with open('results.jsonl') as f:
    for line in f:
        result = json.loads(line)
        message = result['response']['body']['choices'][0]['message']
        if message.get('tool_calls'):
            for tool_call in message['tool_calls']:
                print(f"Tool: {tool_call['function']['name']}")
                print(f"Args: {tool_call['function']['arguments']}")

Best Practices

1. Descriptive custom_ids: Include context in IDs for easier result mapping - Good: "user-123-question-5", "dataset-A-row-42" - Bad: "1", "req1"

2. Validate JSONL locally: Ensure each line is valid JSON with no internal line breaks before upload

3. No duplicate IDs: Each custom_id must be unique within the batch

4. Split large files: Keep under 200MB limit by splitting into multiple batches

5. Choose appropriate window: Use 24h for cost savings (50-83% cheaper), 1h only when time-sensitive

6. Handle errors gracefully: Always check error_file_id and retry failed requests

7. Monitor request_counts: Track progress via completed/total ratio

8. Save file IDs: Store batch_id, input_file_id, output_file_id for later retrieval

9. Use cost estimator: Preview expenses in console before submitting large batches

10. Consider autobatcher: For ongoing workloads, use autobatcher to automatically batch individual API calls

Reference Documentation

For complete API details, see:

Two ways to submit batches:

Via API: 1. Create JSONL file with requests 2. Upload file to get file ID 3. Create batch using file ID 4. Poll status until complete 5. Download results from output_file_id

Via Web Console: 1. Navigate to Batches section at https://app.doubleword.ai/ 2. Upload JSONL file 3. Configure batch settings (model, completion window) 4. Monitor progress in real-time dashboard 5. Download results when ready

Before submitting batches, you need: 1. Doubleword Account - Sign up at https://app.doubleword.ai/ 2. API Key - Create one in the API Keys section of your dashboard 3. Account Credits - Add credits to process requests (see pricing below)

1. Descriptive custom_ids: Include context in IDs for easier result mapping - Good: "user-123-question-5", "dataset-A-row-42" - Bad: "1", "req1"

2. Validate JSONL locally: Ensure each line is valid JSON with no internal line breaks before upload

3. No duplicate IDs: Each custom_id must be unique within the batch

4. Split large files: Keep under 200MB limit by splitting into multiple batches

5. Choose appropriate window: Use 24h for cost savings (50-83% cheaper), 1h only when time-sensitive

6. Handle errors gracefully: Always check error_file_id and retry failed requests

7. Monitor request_counts: Track progress via completed/total ratio

8. Save file IDs: Store batch_id, input_file_id, output_file_id for later retrieval

9. Use cost estimator: Preview expenses in console before submitting large batches

10. Consider autobatcher: For ongoing workloads, use autobatcher to automatically batch individual API calls