astronomer/Llama-3-70B-Special-Tokens-Adjusted

The astronomer/Llama-3-70B-Special-Tokens-Adjusted is a 70 billion parameter Llama 3 model, developed by Astronomer, with its input and output embedding weights adjusted for previously untrained special tokens. This modification addresses potential fine-tuning instabilities like gradient explosions or NaN gradients, making it an ideal and stable base for further fine-tuning. It maintains the original Llama 3 architecture and a context length of 8192 tokens, specifically optimized to prevent issues when adding custom tokens or utilizing existing special tokens.