causal-attention

Here are 8 public repositories matching this topic...

msmrexe / pytorch-gpt2-persian-sentiment-generation

A from-scratch implementation of a scaled-down GPT-2 model in PyTorch, trained on the Snappfood dataset for sentiment-controlled Persian text generation.

deep-learning university-project text-generation transformer course-project self-attention review-generation gpt2 positional-embedding decoder-only persian-text-generation causal-attention

Updated Nov 2, 2025
Python

ictup / Build-a-LLM-model-from-scratch-simple

Star

LLM pipeline: data→tokenizer→attention→GPT train/eval→instruction FT→sampling. Reproducible, clean configs, RTX-4060 defaults, ready for AMP/LoRA/DDP.

tokenizer bytes pytorch transformer gpt lora ddp attention-mechanism peft perplexity llm instruction-tuning next-token-prediction causal-attention

Updated Aug 14, 2025
Jupyter Notebook

anurag965 / SentinelVision

Star

VEHANT Causal Temporal Action Detection System: State-of-the-art deep learning for real-time fight/collapse detection in videos. Features causal attention, motion tokenization, MediaPipe skeletons, uncertainty quantification, and multi-task learning (class + bbox + temporal). 95% accuracy, ONNX/Docker-ready, 25ms GPU inference. 🚀

causal-attention temporal-feature-extraction flow4agent skelformer causal-tad ada-tad

Updated Feb 12, 2026
Python

Diatomic-assay511 / pytorch-gpt2-persian-sentiment-generation

Star

deep-learning university-project text-generation transformer course-project self-attention review-generation gpt2 positional-embedding decoder-only persian-text-generation causal-attention

Updated Oct 10, 2026
Python

Hamidreza-Talei / decoder-only-transformer-from-scratch

Star

A decoder-only Transformer implemented from scratch in PyTorch for character-level name generation.

natural-language-processing deep-learning transformers text-generation language-modeling pytorch transformer autoregressive character-level self-attention multi-head-attention positional-encoding transformer-from-scratch scaled-dot-product-attention temperature-sampling causal-attention decoder-only-transformer

Updated Aug 17, 2026
Jupyter Notebook

Ahtesham00 / attention-mechanism

Star

From-scratch, first-principles implementations of every major attention mechanism — from vanilla dot-product attention to multi-head, causal, and modern LLM-scale variants (MLA, GQA, Flash Attention). Every notebook derives the math by hand, verifies it with loops before vectorizing, and builds up to a reusable PyTorch module.

nlp deep-learning transformers pytorch attention-mechanism from-scratch mla self-attention multi-head-attention gqa llm flash-attention causal-attention

Updated Jul 6, 2026
Jupyter Notebook

atandra2000 / GPT-From-Scratch

Star

Decoder-only Transformer language model built from scratch with PyTorch — 4 layers, 8 heads, character-level tokenizer, trained on Tiny Shakespeare (~6M params, loss 8.69→0.83 over 5 epochs on P100). Foundational educational project; see STATUS.md.

pytorch transformer gpt language-model autoregressive from-scratch p100 causal-attention tiny-shakespeare

Updated Oct 8, 2026
Python

peymanpro / How-Attentions-Work

Star

From-scratch attention mechanisms in NumPy: scaled dot-product, self/cross, multi-head, causal/local, MQA/GQA, additive attention, and verified gradients.

Updated Oct 6, 2026
Python

Add this topic to your repo

To associate your repository with the causal-attention topic, visit your repo's landing page and select "manage topics."

Learn more