How I Study AI - Learn AI Papers & Lectures the Easy Way

ArXiv-to-Model: A Practical Study of Scientific LM Training

Intermediate

Anuj GuptaFeb 19arXiv

This paper shows, step by step, how to train a 1.36-billion-parameter science-focused language model directly from raw arXiv LaTeX files using only 2 A100 GPUs.

#scientific language model#arXiv LaTeX#tokenization

Solar Open Technical Report

Intermediate

Sungrae Park, Sanghoon Kim et al.Jan 11arXiv

Solar Open is a giant bilingual AI (102 billion parameters) that focuses on helping underserved languages like Korean catch up with English-level AI quality.

#Solar Open#Mixture-of-Experts#bilingual LLM

Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learning

Intermediate

Ming Chen, Sheng Tang et al.Dec 6arXiv

The paper shows that making a model write a number as a sequence of digits and then grading the whole number at the end works better than grading each digit separately.

#decoding-based regression#sequence-level reward#reinforcement learning

Papers3

ArXiv-to-Model: A Practical Study of Scientific LM Training

Solar Open Technical Report

Beyond Token-level Supervision: Unlocking the Potential of Decoding-based Regression via Reinforcement Learning