Will SHINE-R weights be released?

[isyuhaochen]

Great work! SHINE is a very elegant and impressive approach. I’m wondering whether there are any plans to release the SHINE-R weights. It would be extremely helpful for reproducing the results and facilitating further research. Thanks for your excellent work!

[Yewei-Liu]

Thanks!
We don’t plan to release SHINE-R, as we are not fully satisfied with its current results. The architecture could be further improved, and with more training time and data, we believe it could achieve much better performance. So, if you’re interested in long context and multiple LoRA updates, please wait for our upcoming research.
If you would like to conduct further research or discuss related ideas, feel free to contact me by email or WeChat. I’m always happy to talk with others in the community.