TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T

TinyLlama-1.1B-intermediate-step-955k-token-2T is a 1.1 billion parameter Llama-architecture language model developed by the TinyLlama project. This intermediate checkpoint has been pretrained on 2 trillion tokens across 995,000 steps, utilizing the same architecture and tokenizer as Llama 2. Its compact size and Llama 2 compatibility make it suitable for applications requiring restricted computation and memory footprints.