Skip to content

About

Deep learning coursework in PyTorch: CNNs, Forward-Forward, VAEs and VQ-VAE, GPT-2, and GAN-BERT for machine-generated text detection.

Topics

Resources

Stars

3 stars

Watchers

1 watching

Forks

Repository files navigation

Deep Learning

Graduate deep learning coursework in PyTorch: MLPs and the Forward-Forward algorithm, CNNs and deformable convolutions, LSTMs on molecules, VAEs and VQ-VAEs, GPT-2 fine-tuning, and a GAN-BERT project that detects machine-generated text in SemEval-2024 Task 8.

Python PyTorch Hugging Face Jupyter

Course: Deep Learning (Dr. Fatemizadeh), Sharif University of Technology, 2023.

Highlight: GAN-BERT for machine-generated text detection

05-project-gan-bert classifies text into six sources (Human, ChatGPT, Cohere, Davinci, BloomZ, Dolly) using only a small labeled fraction of the SemEval-2024 Task 8 (Subtask B) data:

  • A BERT baseline with adapter layers for parameter-efficient fine-tuning.
  • GAN-BERT: a (K+1)-class discriminator on BERT representations, trained adversarially against two generator designs. G1 maps noise to representations, and G2 is an independent pretrained BERT.
  • Experiments with 1 %, 5 %, 10 % and 50 % labeled data, comparing semi-supervised GAN-BERT with the supervised baseline.

GAN-BERT with the G1 generator beats the BERT baseline at every label fraction. The gap is largest when labels are scarcest: about 38 % vs. 29 % accuracy with 1 % of the labels, and 56 % vs. 52 % with 50 %.

Accuracy vs. fraction of labeled data for BERT, GAN-BERT-G1 and GAN-BERT-G2

Assignments

# Folder Topics
1 01-ml-foundations Decision trees, PCA and classification on MNIST and a heart-disease dataset; theory review.
2 02-neural-networks-optimization Fully connected networks from scratch, optimization and regression, and Hinton's Forward-Forward algorithm on MNIST.
3 03-convolutional-networks ResNet-style CNNs on CIFAR-10, deformable convolutions (MNIST, COCO), and change detection between image pairs.
4 04-sequence-and-generative-models LSTM and BiLSTM on SMILES molecular strings (BBBP), VAE and CVAE, GPT-2 fine-tuned for Persian poetry (Ferdowsi), and VQ-VAE on MNIST and colored MNIST.
5 05-project-gan-bert Final project (above).

Each assignment folder contains problem-set.pdf, theory-solutions.pdf (where there is a theory part), and the notebooks in code/. For reference, exams/ holds the midterm and final exams with solutions, and slides/ holds the lecture slides.

Syllabus at a glance

Perceptrons and MLPs, backpropagation, regularization and normalization, optimization (SGD and variants), CNN architectures and vision applications, RNNs, LSTMs, GRUs and Transformers, autoencoders, VAEs and CVAEs, GANs and conditional GANs, diffusion models, and self-supervised learning.

Running

The notebooks were developed on Google Colab with GPU runtimes. To run them locally:

pip install torch torchvision transformers scikit-learn pandas matplotlib jupyter
jupyter lab

Datasets are downloaded inside the notebooks or are included next to them (for example 01-ml-foundations/data/, 04-sequence-and-generative-models/Q1/BBBP.csv).

Author

Mohammad Javad Amin · GitHub

About

Deep learning coursework in PyTorch: CNNs, Forward-Forward, VAEs and VQ-VAE, GPT-2, and GAN-BERT for machine-generated text detection.

Topics

Resources

Stars

3 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages