Defeating the Training-Inference Mismatch via FP16
Sea AI Lab
company
Verified
AI & ML interests
None defined yet.
Recent Activity
View all activity
Papers
Rethinking the Trust Region in LLM Reinforcement Learning
Revisiting Parameter Server in LLM Post-Training
-
Understanding R1-Zero-Like Training: A Critical Perspective
Paper • 2503.20783 • Published • 59 -
sail/Qwen2.5-Math-7B-Oat-Zero
Text Generation • 8B • Updated • 102 • • 6 -
sail/Qwen2.5-Math-1.5B-Oat-Zero
Text Generation • 2B • Updated • 136 • • 4 -
sail/Llama-3.2-3B-Oat-Zero
Text Generation • 3B • Updated • 97 • 1
Automatic data mixture method for large language model pre-training
-
RegMix
📚6Generate predictions and visualize regression results from CSV data
-
RegMix: Data Mixture as Regression for Language Model Pre-training
Paper • 2407.01492 • Published • 40 -
sail/data-mixture-human-1b
Text Generation • Updated • 6 • 3 -
sail/data-mixture-pile-cc-1b
Text Generation • Updated • 2 • 3
Self-alignment with DPO Implicit Rewards
Efficient Process Reward Model Training via Active Learning.
Sailing in South-East Asia with Inclusive Multilingual LLMs
-
Sailor2 20B Chat
🔱27Chat with Sailor2 for detailed answers in multiple languages
-
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
Paper • 2502.12982 • Published • 19 -
sail/Sailor2-8B-Chat
Text Generation • 9B • Updated • 208 • 19 -
sail/Sailor2-1B-Chat
Text Generation • 1.0B • Updated • 63 • 16
Increase your vocabulary size when you scale up your language model
-
Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
Paper • 2407.13623 • Published • 56 -
Scaling With Vocab Demo
📊12Predict optimal vocabulary size for models
-
sail/scaling-vocab-3b-43k-overtrain
Text Generation • 3B • Updated • 1 -
sail/scaling-vocab-3b-32k-overtrain
Text Generation • 3B • Updated • 1
Sailor: Open Language Models tailored for South-East Asia (SEA) released by Sea AI Lab.
Defeating the Training-Inference Mismatch via FP16
Efficient Process Reward Model Training via Active Learning.
-
Understanding R1-Zero-Like Training: A Critical Perspective
Paper • 2503.20783 • Published • 59 -
sail/Qwen2.5-Math-7B-Oat-Zero
Text Generation • 8B • Updated • 102 • • 6 -
sail/Qwen2.5-Math-1.5B-Oat-Zero
Text Generation • 2B • Updated • 136 • • 4 -
sail/Llama-3.2-3B-Oat-Zero
Text Generation • 3B • Updated • 97 • 1
Sailing in South-East Asia with Inclusive Multilingual LLMs
-
Sailor2 20B Chat
🔱27Chat with Sailor2 for detailed answers in multiple languages
-
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
Paper • 2502.12982 • Published • 19 -
sail/Sailor2-8B-Chat
Text Generation • 9B • Updated • 208 • 19 -
sail/Sailor2-1B-Chat
Text Generation • 1.0B • Updated • 63 • 16
Automatic data mixture method for large language model pre-training
-
RegMix
📚6Generate predictions and visualize regression results from CSV data
-
RegMix: Data Mixture as Regression for Language Model Pre-training
Paper • 2407.01492 • Published • 40 -
sail/data-mixture-human-1b
Text Generation • Updated • 6 • 3 -
sail/data-mixture-pile-cc-1b
Text Generation • Updated • 2 • 3
Increase your vocabulary size when you scale up your language model
-
Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies
Paper • 2407.13623 • Published • 56 -
Scaling With Vocab Demo
📊12Predict optimal vocabulary size for models
-
sail/scaling-vocab-3b-43k-overtrain
Text Generation • 3B • Updated • 1 -
sail/scaling-vocab-3b-32k-overtrain
Text Generation • 3B • Updated • 1
Self-alignment with DPO Implicit Rewards
Sailor: Open Language Models tailored for South-East Asia (SEA) released by Sea AI Lab.