upstage/SOLAR-10.7B-v1.0

SOLAR-10.7B-v1.0 by Upstage is a 10.7 billion parameter large language model, developed using a Depth Up-Scaling (DUS) methodology that integrates Mistral 7B weights. This compact yet powerful model demonstrates superior performance, outperforming many models up to 30B parameters, including Mixtral 8x7B, on the H6 benchmark. It is primarily designed as a robust and adaptable base model for fine-tuning, offering significant performance improvements for various natural language processing tasks.