Ai Tip From Karpathy

AI Tip from @karpathy

Description

Automatically generated AI learning skill from curated web and social media sources.

Steps

1. Tip: When training neural nets, always start with a tiny dataset (e.g. 10 examples) and overfit it completely before scaling up. This validates your entire pipeline works end-to-end. 2. # Test your training loop 3. def test_overfit(): 4. model = YourModel() 5. tiny_dataset = get_tiny_dataset(10) 6. train(model, tiny_dataset, epochs=1000) 7. assert loss < 0.01

Code Examples

# Test your training loop
def test_overfit():
    model = YourModel()
    tiny_dataset = get_tiny_dataset(10)
    train(model, tiny_dataset, epochs=1000)
    assert loss < 0.01