Agent Lightning

Agent Lightning ⚡

Microsoft Research's agent training framework. Turn your AI agents into optimizable beasts with (almost) zero code changes.

Core Features

Installation

pip install agentlightning

For latest nightly build:

pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning

Quick Start

1. Instrument Your Agent

Option A: Add emit helpers (recommended)

import agentlightning as agl
In your agent's tool calls
response = agl.emit_tool_call(
    model=model,
    messages=messages,
    tools=tools,
    context={"task": "search"}
)

Option B: Use tracer (zero code change)

from agentlightning import tracer
Wrap your agent with tracer
with tracer.trace("my-agent", input_data):
    result = your_agent.run(user_query)

2. Create Training Config

# config.yaml
agent:
  name: "my-agent"
  type: "openai"  # openai, langchain, autogen, crewai
training:
  algorithm: "grpo"  # grpo, apo, sft, rloo
  episodes: 100
  batch_size: 16
  
environment:
  eval_tasks:
    - "math"
    - "coding"
    - "reasoning"

3. Run Training

agent-lightning train --config config.yaml

Algorithms

| Algorithm | Use Case | Description | |-----------|----------|-------------| | GRPO | General RL | Group Relative Policy Optimization — stable, works well for most agents | | APO | Prompt Tuning | Automatic Prompt Optimization — improves system prompts | | SFT | Supervised Fine-tuning | Supervised Fine-tuning with preference data | | RLOO | Long-horizon | RLOO for tasks with sparse rewards |

Usage Commands

agent-lightning train

agent-lightning eval

agent-lightning export

agent-lightning serve

Example: SQL Agent Training

See full example: Train SQL Agent with RL

from agentlightning import Agent, RLConfig, GRPOTrainer
1. Define your agent
sql_agent = Agent(
    name="sql-agent",
    system_prompt="You are a SQL expert...",
    tools=[execute_sql, query_schema]
)
2. Configure RL training
config = RLConfig(
    algorithm="grpo",
    episodes=500,
    learning_rate=1e-4
)
3. Train
trainer = GRPOTrainer(config=config)
trainer.train(sql_agent, eval_tasks=["sql-generation"])

Integration with Clawdbot

Environment Variables

# Required for training
export OPENAI_API_KEY="sk-..."
Optional: for remote storage
export AGL_STORAGE="s3://my-bucket/agent-lightning/"

Python API

from agentlightning import LightningStore, GRPOTrainer
LightningStore keeps tasks, resources, and traces in sync
store = LightningStore()
Read traces, learn, and update prompts
trainer = GRPOTrainer(store=store)
trainer.train(agent=my_agent)

Monitoring Training

# Launch dashboard
agent-lightning dashboard --port 8080
View logs
tail -f ~/.agent-lightning/logs/training.log

Best Practices

1. Start Small: Begin with 10-50 episodes to verify setup 2. Define Clear Rewards: Design reward functions that match your goal 3. Use Evaluation Tasks: Always eval on held-out tasks 4. Checkpoint Frequently: Save model every N episodes 5. Monitor Convergence: Watch loss curves in dashboard

Resources

Citation

If you use Agent Lightning in research:

@misc{luo2025agentlightningtrainai,
  title={Agent Lightning: Train ANY AI Agents with Reinforcement Learning},
  author={Xufang Luo and Yuge Zhang and Zhiyuan He and Zilong Wang and Siyun Zhao and Dongsheng Li and Luna K. Qiu and Yuqing Yang},
  year={2025},
  eprint={2508.03680},
  archivePrefix={arXiv},
  primaryClass={cs.AI}
}

1. Instrument Your Agent

Option A: Add emit helpers (recommended)

import agentlightning as agl
In your agent's tool calls
response = agl.emit_tool_call(
    model=model,
    messages=messages,
    tools=tools,
    context={"task": "search"}
)

Option B: Use tracer (zero code change)

from agentlightning import tracer
Wrap your agent with tracer
with tracer.trace("my-agent", input_data):
    result = your_agent.run(user_query)

2. Create Training Config

# config.yaml
agent:
  name: "my-agent"
  type: "openai"  # openai, langchain, autogen, crewai
training:
  algorithm: "grpo"  # grpo, apo, sft, rloo
  episodes: 100
  batch_size: 16
  
environment:
  eval_tasks:
    - "math"
    - "coding"
    - "reasoning"

3. Run Training

agent-lightning train --config config.yaml

1. Start Small: Begin with 10-50 episodes to verify setup 2. Define Clear Rewards: Design reward functions that match your goal 3. Use Evaluation Tasks: Always eval on held-out tasks 4. Checkpoint Frequently: Save model every N episodes 5. Monitor Convergence: Watch loss curves in dashboard