The following papers were recommended by the Semantic Scholar API
\n- \n
- RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models (2025) \n
- Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning (2025) \n
- Sparse Gradient Compression for Fine-Tuning Large Language Models (2025) \n
- Diffusion Instruction Tuning (2025) \n
- A Multi-Encoder Frozen-Decoder Approach for Fine-Tuning Large Language Models (2025) \n
- Boosting Tool Use of Large Language Models via Iterative Reinforced Fine-Tuning (2025) \n
- Towards Compatible Fine-tuning for Vision-Language Model Updates (2024) \n
Please give a thumbs up to this comment if you found it helpful!
\nIf you want recommendations for any Paper on Hugging Face checkout this Space
\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: \n\n@librarian-bot\n\t recommend
Selective Self-to-Supervised Fine-Tuning for Generalization in Large Language Models
Abstract
Selective Self-to-Supervised Fine-Tuning (S3FT) improves generalization and performance of fine-tuned Large Language Models (LLMs) by leveraging correct responses and reducing specialization.
Fine-tuning Large Language Models (LLMs) on specific datasets is a common practice to improve performance on target tasks. However, this performance gain often leads to overfitting, where the model becomes too specialized in either the task or the characteristics of the training data, resulting in a loss of generalization. This paper introduces Selective Self-to-Supervised Fine-Tuning (S3FT), a fine-tuning approach that achieves better performance than the standard supervised fine-tuning (SFT) while improving generalization. S3FT leverages the existence of multiple valid responses to a query. By utilizing the model's correct responses, S3FT reduces model specialization during the fine-tuning stage. S3FT first identifies the correct model responses from the training set by deploying an appropriate judge. Then, it fine-tunes the model using the correct model responses and the gold response (or its paraphrase) for the remaining samples. The effectiveness of S3FT is demonstrated through experiments on mathematical reasoning, Python programming and reading comprehension tasks. The results show that standard SFT can lead to an average performance drop of up to 4.4 on multiple benchmarks, such as MMLU and TruthfulQA. In contrast, S3FT reduces this drop by half, i.e. 2.5, indicating better generalization capabilities than SFT while performing significantly better on the fine-tuning tasks.
Community
Fine-tuning Large Language Models (LLMs) on specific datasets is a common practice to improve performance on target tasks. However, this performance gain often leads to overfitting, where the model becomes too specialized in either the task or the characteristics of the training data, resulting in a loss of generalization. This paper introduces Selective Self-to-Supervised Fine-Tuning (S3FT), a fine-tuning approach that achieves better performance than the standard supervised fine-tuning (SFT) while improving generalization. S3FT leverages the existence of multiple valid responses to a query. By utilizing the model's correct responses, S3FT reduces model specialization during the fine-tuning stage. S3FT first identifies the correct model responses from the training set by deploying an appropriate judge. Then, it fine-tunes the model using the correct model responses and the gold response (or its paraphrase) for the remaining samples. The effectiveness of S3FT is demonstrated through experiments on mathematical reasoning, Python programming, and reading comprehension tasks. The results show that standard SFT can lead to an average performance drop of up to 4.4 on multiple benchmarks, such as MMLU and TruthfulQA. In contrast, S3FT reduces this drop by half, i.e. 2.5, indicating better generalization capabilities than SFT while performing significantly better on the fine-tuning tasks.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Models (2025)
- Token Cleaning: Fine-Grained Data Selection for LLM Supervised Fine-Tuning (2025)
- Sparse Gradient Compression for Fine-Tuning Large Language Models (2025)
- Diffusion Instruction Tuning (2025)
- A Multi-Encoder Frozen-Decoder Approach for Fine-Tuning Large Language Models (2025)
- Boosting Tool Use of Large Language Models via Iterative Reinforced Fine-Tuning (2025)
- Towards Compatible Fine-tuning for Vision-Language Model Updates (2024)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper