About the Project

This project focuses on fine-tuning a pre-trained large language model, specifically Google's Gemma 2b instruction-tuned model, to enhance its ability to answer questions related to Python programming.

By fine-tuning on a domain-specific dataset, the aim is to improve the model's accuracy, relevance to answering question concerning sensitive topics like mental health (normally return generic answer in pre-trained model)

Technique

• Parameter-Efficient Fine-Tuning (PEFT) - LoRA.
• Utilized trl library (SFTTrainer) for Supervised Fine-Tuning.
• Using unsloth for fast fine-tuning and accessing large language models.

Achievement

• Successfully fine-tuned Gemma 2b
• Configured and executed a standard PEFT fine-tuning pipeline.
• Generated a fine-tuned model capable of responding to Python questions.

Further Development

Potential areas for further development include:

• Increasing Dataset Size and Diversity: Fine-tuning on the full dataset or incorporating data from other relevant Python resources could significantly improve the model's breadth and depth of knowledge.
• Extended Training: Training for more epochs might allow the model to converge better and capture more complex patterns in the data.
• Hyperparameter Tuning: Experimenting with different LoRA parameters (e.g., r, lora_alpha) and training arguments (e.g., learning rate, batch size, gradient accumulation) could lead to improved performance.
• Advanced Evaluation Metrics: Incorporating human evaluation or using metrics specifically designed for code generation or technical question answering could provide a more comprehensive assessment of the model's capabilities.
• Deployment and Inference Optimization: Exploring methods for deploying the fine-tuned model efficiently for inference in various applications.