Graduate deep learning coursework in PyTorch: MLPs and the Forward-Forward algorithm, CNNs and deformable convolutions, LSTMs on molecules, VAEs and VQ-VAEs, GPT-2 fine-tuning, and a GAN-BERT project that detects machine-generated text in SemEval-2024 Task 8.
Course: Deep Learning (Dr. Fatemizadeh), Sharif University of Technology, 2023.
05-project-gan-bert classifies text into six sources (Human, ChatGPT, Cohere, Davinci, BloomZ, Dolly) using only a small labeled fraction of the SemEval-2024 Task 8 (Subtask B) data:
- A BERT baseline with adapter layers for parameter-efficient fine-tuning.
- GAN-BERT: a (K+1)-class discriminator on BERT representations, trained adversarially against two generator designs. G1 maps noise to representations, and G2 is an independent pretrained BERT.
- Experiments with 1 %, 5 %, 10 % and 50 % labeled data, comparing semi-supervised GAN-BERT with the supervised baseline.
GAN-BERT with the G1 generator beats the BERT baseline at every label fraction. The gap is largest when labels are scarcest: about 38 % vs. 29 % accuracy with 1 % of the labels, and 56 % vs. 52 % with 50 %.
| # | Folder | Topics |
|---|---|---|
| 1 | 01-ml-foundations |
Decision trees, PCA and classification on MNIST and a heart-disease dataset; theory review. |
| 2 | 02-neural-networks-optimization |
Fully connected networks from scratch, optimization and regression, and Hinton's Forward-Forward algorithm on MNIST. |
| 3 | 03-convolutional-networks |
ResNet-style CNNs on CIFAR-10, deformable convolutions (MNIST, COCO), and change detection between image pairs. |
| 4 | 04-sequence-and-generative-models |
LSTM and BiLSTM on SMILES molecular strings (BBBP), VAE and CVAE, GPT-2 fine-tuned for Persian poetry (Ferdowsi), and VQ-VAE on MNIST and colored MNIST. |
| 5 | 05-project-gan-bert |
Final project (above). |
Each assignment folder contains problem-set.pdf, theory-solutions.pdf (where there is a theory part), and the notebooks in code/.
For reference, exams/ holds the midterm and final exams with solutions, and slides/ holds the lecture slides.
Perceptrons and MLPs, backpropagation, regularization and normalization, optimization (SGD and variants), CNN architectures and vision applications, RNNs, LSTMs, GRUs and Transformers, autoencoders, VAEs and CVAEs, GANs and conditional GANs, diffusion models, and self-supervised learning.
The notebooks were developed on Google Colab with GPU runtimes. To run them locally:
pip install torch torchvision transformers scikit-learn pandas matplotlib jupyter
jupyter labDatasets are downloaded inside the notebooks or are included next to them (for example 01-ml-foundations/data/, 04-sequence-and-generative-models/Q1/BBBP.csv).
Mohammad Javad Amin · GitHub
