LLM
Collection
Llm for Scratch
โข
1 item
โข
Updated
๏ฝ ๐ฅ๏ธ GitHub (Coming soon) | Hugging Face | ๐ค | ๐ Paper (Coming soon) |
Project Philp is an open-source large language model (LLM) development project with a full model of approximately 2x250 million parameters and a scale of 500 million parameters. The project is an experimental and learning-focused effort developed individually. Articles, technical notes, and experimental results related to the model's development process will be shared openly.
The main goal of the project is to produce a simple but functional LLM architecture from scratch, using some configurations from models such as Mistral, and to advance this model over time by developing it with more modern architectures.
The targeted hardware options for training the model include GPUs with high memory capacity. The planned hardware is NVIDIA H100 80GB or A100 80GB GPU systems. There may be delays in the model development process due to limited hardware access.
Project Philp is conducted entirely for hobby and personal learning purposes. There is no corporate R&D support or financial resources within the scope of the project. The developer aims to learn model development from scratch, architectural design, and machine learning infrastructures through this project.
Project outputs, model weights, training notes, and technical documentation will be published as open source. Errors, trials, and unsuccessful results throughout the work process will also be shared transparently.
This project will be carried out entirely from scratch by someone learning Python and PyTorch for the first time.
Project page and publications: Will be shared via Hugging Face.
According to the plan, a simple web interface running on a Raspberry Pi CPU is planned.
Model name philp-1-Medium
The quantized version of the model will also be shared for easier execution. Formats to be shared (.GGUF โQ2_K Q3_K_S Q3_K_M Q4_K_M Q4_K_S Q5_K_M Q5_K_S Q6_K Q8_Kโ FP8 NVFP4)
Planned release date: 4 to 5 months
Translated with DeepL.com