anlp-a2-part2-muon

Dense decoder-only transformer pretrained from scratch on browndw/human-ai-parallel-corpus (1x the training split) with a from-scratch muon optimizer.

• params: 17,011,584
• trained on 41,680,896 tokens
• W&B run: https://wandb.ai/dunkronit-iiit-hyderabad/anlp-a2-part2/runs/muon-dd551d9d

Load with src.part2.model.Transformer.from_pretrained("DunkRonit/anlp-a2-part2-muon") from the assignment repo.