Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /home/zhenxiangba/zhenxiangba.com/public_html/phproxy-improved-master/index.php on line 456
ssz1111 (ssz)
[go: Go Back, main page]
Hugging Face's logo
ssz's picture
5 14 14

ssz

ssz1111
patrickNLP's profile picture AMRuk's profile picture ShuangZeng's profile picture
·

AI & ML interests

None yet

Recent Activity

reacted to vicgalle's post with 🤝 30 days ago
Can you merge models of different sizes? ⚗️ Well, yes, if the models are somewhat compatible. Here is an experiment I did. I wanted to merge two of the best performing models: https://huggingface.co/mlabonne/NeuralBeagle14-7B and https://huggingface.co/jeonsworld/CarbonVillain-en-10.7B-v4 Here is my recipe: 1. Expand the layers of NeuralBeagle to 10.7B ala frankenmerge. 2. DPO-tune the previous model with a high-quality preference dataset, https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs 3. Merge the previous model with CarbonVillain (needs —allow-crimes in mergekit! 🔪) And here is the resulting model, CarbonBeagle-11B, which ranked top in the leaderboard for its size class: https://huggingface.co/vicgalle/CarbonBeagle-11B
reacted to vicgalle's post with 🤯 30 days ago
Can you merge models of different sizes? ⚗️ Well, yes, if the models are somewhat compatible. Here is an experiment I did. I wanted to merge two of the best performing models: https://huggingface.co/mlabonne/NeuralBeagle14-7B and https://huggingface.co/jeonsworld/CarbonVillain-en-10.7B-v4 Here is my recipe: 1. Expand the layers of NeuralBeagle to 10.7B ala frankenmerge. 2. DPO-tune the previous model with a high-quality preference dataset, https://huggingface.co/datasets/argilla/distilabel-intel-orca-dpo-pairs 3. Merge the previous model with CarbonVillain (needs —allow-crimes in mergekit! 🔪) And here is the resulting model, CarbonBeagle-11B, which ranked top in the leaderboard for its size class: https://huggingface.co/vicgalle/CarbonBeagle-11B
upvoted an article 30 days ago
Merge Large Language Models with mergekit
View all activity

Organizations

DeepLang AI's profile picture The BIRD Team's profile picture