yandex/YandexGPT-5-Lite-8B-pretrain

YandexGPT-5-Lite-8B-pretrain is an 8 billion parameter pre-trained large language model developed by Yandex, featuring a 32k token context length. It was trained on 15T tokens, primarily Russian and English texts, with a significant focus on web pages, code, and mathematics. This model achieves parity or surpasses global SOTA benchmarks in its category for pre-trained models, particularly excelling due to its optimized tokenizer for the Russian language.