LLM360 Pretraining Suite

LLM360 Pretraining Suite is a series of step-by-step guides to reproduce each of our models, for tech enthusiasts, AI practitioners, and academic or industry researchers, to transfer knowledge on LLM pretraining techniques.
Our tutorials are currently in active development and will be released in the upcoming weeks. Stay tuned!

📖 Tutorials

K2 Tutorials:

Crystal Tutorials:

Amber Tutorials:

Pretraining Result References

Please review the key results provided for your reference and check if your model's results are correct 🤗

Models Check results
K2 K2 Prompt Gallery
K2 K2 Evaluation Gallery
K2 W&B
Crystal W&B
Amber W&B