Doc-to-LoRA

Doc-to-LoRA Skill

Internalize any document into a small model's weights in seconds. No fine-tuning loop, no RAG retrieval at query time. The model "knows" the document.

How It Works (30-second summary)

A trained hypernetwork reads your document and instantly generates LoRA adapter weights for every layer of Gemma 2 2B. The adapter is applied to the base model, which can then answer questions about the document without it being in the prompt.

Document --> Context Encoder --> Perceiver --> HyperLoRA --> LoRA weights
                                                                |
                                                    Apply to Gemma 2 2B
                                                                |
                                                    Answer questions (no doc in prompt)

For architecture details, read references/ARCHITECTURE.md in this skill directory.

Security Notes

Prerequisites

This skill runs inside a clone of the doc-to-lora repository. It is not a standalone tool.

Required before setup:

Run setup once. This installs Python dependencies and downloads model weights (~7GB total).

export HF_TOKEN=hf_your_token_here
bash ${CLAUDE_SKILL_DIR}/scripts/setup.sh

If setup was already completed, skip this step. Check with:

test -d trained_d2l/gemma_demo && echo "Weights present" || echo "Run setup first"

Workflow A: PyTorch Path (simpler, ~10GB RAM)

Use this when the user provides a document and wants answers. The internalize.py script handles both internalization and querying in one call.

Internalize a document and ask questions

python ${CLAUDE_SKILL_DIR}/scripts/internalize.py \
  --input "path/to/document.txt" \
  --question "What is the main finding?" \
  --checkpoint trained_d2l/gemma_demo/checkpoint-80000/pytorch_model.bin

Or pass text directly:

python ${CLAUDE_SKILL_DIR}/scripts/internalize.py \
  --text "Paste the document content here..." \
  --question "What is this about?"

For multiple questions, pass them comma-separated:

python ${CLAUDE_SKILL_DIR}/scripts/internalize.py \
  --input "path/to/document.txt" \
  --question "Question 1?,Question 2?,Question 3?"

For programmatic use, output results as JSON:

python ${CLAUDE_SKILL_DIR}/scripts/internalize.py \
  --input doc.txt --question "Q?" --output-json results.json

Workflow B: MLX Path (faster, ~6GB RAM, recommended for Mac)

Use this for best performance on Apple Silicon. Two-phase: export once, query fast.

Step 1: Export LoRA adapter from document

python scripts/export_d2l_to_mlx_adapter.py \
  --checkpoint trained_d2l/gemma_demo/checkpoint-80000/pytorch_model.bin \
  --context-file "path/to/document.txt" \
  --output-dir adapters_d2l

Step 2: Query with MLX (lightweight, Metal-accelerated)

python ${CLAUDE_SKILL_DIR}/scripts/query_mlx.py \
  --adapter-dir adapters_d2l \
  --question "What is the main finding?"

When to Use Which Path

| Scenario | Path | Why | |----------|------|-----| | Quick one-off question about a doc | PyTorch | Simpler, no export step | | Many questions about the same doc | MLX | Export once, query fast and cheap | | RAM-constrained (16GB Mac) | MLX | ~6GB vs ~10GB at query time | | Multiple documents to compare | MLX | Export each, swap adapters instantly |

Limitations

Troubleshooting

| Problem | Fix | |---------|-----| | ModuleNotFoundError: No module named 'ctx_to_lora' | Run setup: bash ${CLAUDE_SKILL_DIR}/scripts/setup.sh | | FileNotFoundError: trained_d2l/... | Download weights: uv run huggingface-cli download SakanaAI/doc-to-lora --local-dir trained_d2l | | FileNotFoundError: install_mac.sh | This skill must be used inside a doc-to-lora repo clone that contains install_mac.sh | | RuntimeError: MPS backend out of memory | Use MLX path instead, or close other apps | | ImportError: bitsandbytes | Expected on Mac. The scripts auto-disable quantization on non-CUDA. | | Answers seem wrong / generic | Check if LoRA is applied: outputs should differ from baseline. Try rephrasing. |

This skill runs inside a clone of the doc-to-lora repository. It is not a standalone tool.

Required before setup:

Run setup once. This installs Python dependencies and downloads model weights (~7GB total).

export HF_TOKEN=hf_your_token_here
bash ${CLAUDE_SKILL_DIR}/scripts/setup.sh

If setup was already completed, skip this step. Check with:

test -d trained_d2l/gemma_demo && echo "Weights present" || echo "Run setup first"

| Problem | Fix | |---------|-----| | ModuleNotFoundError: No module named 'ctx_to_lora' | Run setup: bash ${CLAUDE_SKILL_DIR}/scripts/setup.sh | | FileNotFoundError: trained_d2l/... | Download weights: uv run huggingface-cli download SakanaAI/doc-to-lora --local-dir trained_d2l | | FileNotFoundError: install_mac.sh | This skill must be used inside a doc-to-lora repo clone that contains install_mac.sh | | RuntimeError: MPS backend out of memory | Use MLX path instead, or close other apps | | ImportError: bitsandbytes | Expected on Mac. The scripts auto-disable quantization on non-CUDA. | | Answers seem wrong / generic | Check if LoRA is applied: outputs should differ from baseline. Try rephrasing. |