anlp-a2-part2-adamw

Dense decoder-only transformer pretrained from scratch on browndw/human-ai-parallel-corpus (1x the training split) with a from-scratch adamw optimizer.

• params: 17,011,584
• trained on 41,680,896 tokens
• W&B run: https://wandb.ai/dunkronit-iiit-hyderabad/anlp-a2-part2/runs/adamw-40cfbfa5

Load with src.part2.model.Transformer.from_pretrained("DunkRonit/anlp-a2-part2-adamw") from the assignment repo.