Qwen/Qwen2-1.5B

Qwen/Qwen2-1.5B is a 1.5 billion parameter base language model developed by Qwen, part of the new Qwen2 series. This decoder-only Transformer model features SwiGLU activation and Grouped Query Attention, and is designed for a wide range of tasks including language understanding, generation, coding, and mathematics. It demonstrates strong performance across various benchmarks, particularly excelling in MMLU, GSM8K, MATH, C-Eval, and CMMLU compared to similarly sized open-source models.