Small Language Model

This project is focused on building and evaluating a small language model. The repository contains notebooks for data exploration, dataset preparation, modeling, inference, and evaluation.

Folder Structure

.
├── Data
│   ├── raw
│   └── weights
├── Notebooks
│   ├── 01-Exploratory.ipynb
│   ├── 02-Dataset.ipynb
│   ├── 03-Modeling.ipynb
│   ├── 04-Inference.ipynb
│   └── 05-Evaluation.ipynb
├── Reports
│   └── Figures
│       ├── Average_time_inference.png
│       ├── inference_time_per_prompt.png
│       ├── model_size_reduction.png
│       └── performance_summary.png
├── .gitignore
├── README.md
└── requirements.txt

Hardware Requirements

• CPU: Intel Core i7 10th Gen
• GPU: NVIDIA A100
• RAM: 16 GB

Installation

1. Clone the repository:

   git clone https://github.com/your-username/small-language-model.git

2. Install the required packages:

   pip install -r requirements.txt

Performance Analysis

Average Time Inference

Inference Time per Prompt

Model Size Reduction

Performance Summary