Overview

From scratch implementation of GPT

Technical Details

• Framework: PyTorch
• Dataset: TinyShakespeare
• Category: Language Models

Implementation Details

I implemented the GPT from scratch using Pytorch on Tinyshakespeare dataset.

Improving Language Understanding by Generative Pre-Training

Datasets

Tineshakespeare: in the /data folder

Frameworks:

Pytorch

Source Code

📁 GitHub Repository: GPT

View the complete implementation, training scripts, and documentation on GitHub.