hanzceo/sts-en-en
Viewer • Updated • 1.18k • 46
Hanz
hanzceo
AI & ML interests
None yet
Recent Activity
upvoted a paper 1 day ago
Morphing into Hybrid Attention Models repliedto Banaxi-Tech's post 2 days ago
A new model is coming!
Its going to take a long time on my 5070 Ti so expect a release in ~1 month.
We think this model is going to be SOTA For its size.
Our Mini Version will be 25M Parameters and Pro with 140M.
The Pro version has a 3072 Context Window (Extensible to up to 6K with RoPE) And the Mini version has a context window of 4096 (Up to 8K with RoPE)
Meanwhile we are currently working on a Instruct Version of our BananaMind 1.5 Base.
The training will start this weekend
We are very exited to release it when its done! reacted to Banaxi-Tech's post with 🔥 2 days ago
A new model is coming!
Its going to take a long time on my 5070 Ti so expect a release in ~1 month.
We think this model is going to be SOTA For its size.
Our Mini Version will be 25M Parameters and Pro with 140M.
The Pro version has a 3072 Context Window (Extensible to up to 6K with RoPE) And the Mini version has a context window of 4096 (Up to 8K with RoPE)
Meanwhile we are currently working on a Instruct Version of our BananaMind 1.5 Base.
The training will start this weekend
We are very exited to release it when its done!