Fine-Tuning Large Language Models 🧠

This project implements a resource-efficient method for fine-tuning Large Language Models (LLMs) on consumer-grade hardware. It focuses on the technical implementation of Parameter-Efficient Fine-Tuning (PEFT), offering a modular codebase that supports two distinct training pathways: the standard Hugging Face library and the optimized Unsloth framework.

The key to achieving this resource-efficient method lies in the use of Low-Rank Adaptation (LoRA). Instead of retraining the full model parameters, a process that requires massive computational resources, LoRA freezes the pre-trained model and injects trainable rank-decomposition matrices into the transformer layers. For memory optimization, the project also supports QLoRA (Quantized LoRA). This technique quantizes the frozen base model to 4-bit precision to significantly reduce memory usage (VRAM) while maintaining model performance. Apart from standard LoRA and QLoRA implementation, the project also offers an Unsloth mode, which uses optimized low-level GPU kernels to execute the QLoRA fine-tuning up to 5x faster than standard implementations, while providing superior VRAM utilization. These approaches make it possible to fine-tune billion-parameter models on standard GPUs.

The entire implementation is demonstrated using Google's Gemma-3-1B-IT as the base model and serves as a practical reference for developers looking to adapt similar architectures to downstream tasks.

📜 Dataset Description

The project used the Sentiment Analysis for Mental Health dataset for experimentation which, contains user statements labeled with mental health conditions. This dataset is structured in a simple CSV format containing approximately 53,000 rows. Each entry consists of a unique identifier, the raw text statement, and the corresponding ground-truth label. It classifies text into seven distinct categories. It is important to note that the classes are imbalanced, with conditions like "Normal" and "Depression" being significantly more represented than "Personality Disorder" or "Bi-Polar." This imbalance presents a realistic challenge for fine-tuning, requiring the model to learn features for minority classes effectively. The specific labels used in this project are detailed below:

Label	Description
Normal	General conversation, neutral observations, or positive sentiment without distress.
Depression	Statements reflecting persistent sadness, hopelessness, lethargy, or loss of interest.
Suicidal	High-risk content indicating self-harm ideation or intent.
Anxiety	Expressions of excessive worry, nervousness, panic, or unease.
Stress	Reactions to external pressure, tension, burnout, or inability to cope.
Bi-Polar	Text exhibiting rapid mood cycling, manic energy, or depressive lows.
Personality Disorder	Patterns of behavior or inner experience that deviate markedly from expectations.

🛠️Development Workflow

This project follows a three-tier branching strategy with automated deployments:

Branch Structure

• dev - Development branch for active feature work
• tst - Testing/staging environment for validation
• prd - Production-ready stable releases

CI/CD Pipeline

Automatic Deployment (dev → tst):

• Any push to dev automatically triggers a GitHub Actions workflow
• Changes are merged into tst branch for testing
• Workflow: .github/workflows/deploy_tst.yml

Manual Deployment (tst → prd):

• Deployment to prd requires manual approval via GitHub Actions
• Only executable from the tst branch
• Workflow: .github/workflows/deploy_prd.yml

Although this is a personal project, the CI/CD pipeline adheres to professional standards for maintaining a stable codebase and facilitating effective collaboration.

⚙️ Setup Instructions

Prerequisites

• Operating System: A Linux environment (Ubuntu, Debian, etc.) or Windows Subsystem for Linux (WSL 2).
• NVIDIA GPU with CUDA 12.4 support (RTX 30/40 series recommended)
• Python 3.11
• Poetry for dependency management
• HuggingFace Account with a valid User Access Token

Installation

1. Clone the repository:

   git clone https://github.com/Dalageo/fine-tuning-llms.git
   cd fine-tuning-llms

2. Install Poetry (if not already installed):

   curl -sSL https://install.python-poetry.org | python3 -

3. Install dependencies:

   poetry install

   This will install PyTorch 2.6.0 with CUDA 12.4, Unsloth, Transformers, PEFT, TRL, and all required packages.

4. Create .env file with your HuggingFace token:

   "HF_TOKEN=your_huggingface_token_here"

5. Update dataset path in app/configs/config.py:

   DATASET_PATH = 'path/to/your/dataset.csv'

6. Configure training mode in app/configs/config.py:

   UNSLOTH = True        # Use Unsloth (faster) or standard HF
   LORA_MODE = "qlora"   # Choose "lora" or "qlora"

Training

Run the training pipeline:

poetry run python -m app.train

Training artifacts will be saved to ./sft_output/{HF_REPO_ID}/ including:

• Adapter weights (adapter_model.safetensors)
• Tokenizer files
• Configuration files
• Checkpoints (saved according to the predefined save_steps setting)

Inference

Inference can run in two modes:

Evaluation Mode (using test dataset):

poetry run python -m app.inference
# Enter: eval

Interactive Chat Mode:

poetry run python -m app.inference
# Enter: chat

Upload to HuggingFace Hub

After training, you can simply upload your model:

poetry run python -m app.utils.upload

The model will be pushed to the configured HF_PERSONAL_REPO_ID on HuggingFace Hub.

📁 Project Structure

fine-tuning-llms/
├── app/
│   ├── configs/
│   │   ├── config.py           # Main configuration (model, dataset, lora/qlora)
│   │   └── lora_config.py      # LoRA/QLoRA hyperparameters
│   ├── model/
│   │   ├── model.py            # Model loading logic (Unsloth/Standard)
│   │   └── tokenizer.py        # Tokenizer initialization
│   ├── utils/
│   │   ├── data_prep.py        # Dataset loading and preprocessing
│   │   ├── download.py         # Download models from HuggingFace
│   │   └── upload.py           # Upload trained adapters to HuggingFace
│   ├── train.py                # Training pipeline
│   └── inference.py            # Inference and evaluation
├── pyproject.toml              # Poetry dependencies

❓ Troubleshooting

CUDA Out of Memory

• Reduce per_device_train_batch_size in train.py
• Increase gradient_accumulation_steps
• Lower max_length (e.g., from 1024 to 512)
• Use QLoRA instead of LoRA

Slow Training

• Enable UNSLOTH=True in config
• Verify CUDA is being used: torch.cuda.is_available()
• Check GPU utilization: nvidia-smi

Dependency Conflicts

• Use Poetry's lock file: poetry install --sync
• Ensure PyTorch is from CUDA 12.4 source: check pyproject.toml

✨ Acknowledgments

Special thanks to Google for developing and releasing open-source models, to the Hugging Face community for hosting the models and providing the Transformers, PEFT, and TRL libraries, and to Unsloth AI for their optimized training framework that makes it easier for individuals to experiment with LLMs using their own GPUs. Their contributions were essential to this project.

         

⚖️ License

This repository utilizes components with different licenses:

• The Code & Documentation: Licensed under the AGPL-3.0 license.

  The AGPL-3.0 license was chosen to promote open collaboration, ensure transparency, and require that any modifications or improvements must also be shared under the same license, with appropriate acknowledgment.

• The Base LLM Weights: Gemma weights used for fine-tuning are subject to their respective Google's terms Gemma Terms of Use.

• The Unsloth Framework: Unsloth AI is an open-source tool licensed under the Apache License 2.0.

• Dataset: The Sentiment Analysis for Mental Health dataset may have its own license terms on Kaggle.