TRAINING FARM

NOTES FROM THE FARM

Blog.

Research notes on training models. Methods, experiments, and the details needed to reproduce them.

Articles01
  1. Pretraining

    Puro-2B: How Can a Poor Lab Pretrain a Model from Scratch?

    Pretraining Puro-2B on a budget with RTX 5090 GPUs. Explore the hardware, FP8, optimizer, and data choices behind the recipe, plus the open resources for reproducing it.

    Kairong Luo, Jiarui Cui, Yaorui Yin, Shengqi Chen et al.

Starting with one training story.