Book a call

03 / A-200M

Training log

Validation loss at the end of each session. Pretraining finished at step 15,000 with 2.971.

Validation loss by training stepLower is better
3.03.23.43.605k10k15kSession 2 start: step 1,860, validation loss 3.556Session 2: step 3,903, validation loss 3.268Session 3: step 5,697, validation loss 3.199Session 4: step 7,656, validation loss 3.07Session 5: step 9,677, validation loss 3.059Session 6: step 11,241, validation loss 3.06Session 7: step 12,753, validation loss 3.023Session 8: step 14,766, validation loss 2.922Session 9, pretraining complete: step 15,000, validation loss 2.9713.5562.971
201.9Mparameters
15,000steps, over 9 sessions
3.9Btokens seen
₹0cash on compute
PointStepValidation loss
Session 2 start1,8603.556
Session 23,9033.268
Session 35,6973.199
Session 47,6563.070
Session 59,6773.059
Session 611,2413.060
Session 712,7533.023
Session 814,7662.922
Session 9, pretraining complete15,0002.971

Single readings swing by about 0.1, so read the shape, not one row. Perplexity at the end is about 19.5.

Questions about the run?

Write to us or book a short call.