English-to-ASL Translation

NLP pipeline translating English text into ASL gloss, mapped to sign videos

Semester-long NLP + computer vision project translating English sentences into American Sign Language (ASL) gloss and mapping the output to corresponding sign videos, rather than relying on a rule-based English→ASL mapping.

Pipeline

An eight-step pipeline, runnable either as a single end-to-end notebook or as modular scripts:

  1. Load datasets — ASLG_PC12 (English↔ASL gloss parallel corpus) and WLASL2000 (gloss-to-video mapping).
  2. Preprocess — clean text, build the gloss-to-video mapping.
  3. Tokenize — prepare the parallel corpus for model training.
  4. Build DataLoaders — PyTorch DataLoaders for training/evaluation.
  5. Verify — sanity-check dataset and model setup before committing GPU time to a bad run.
  6. Train — fine-tune a text-to-gloss translation model (PyTorch + Hugging Face).
  7. Evaluate — generate predictions, score against held-out gloss.
  8. Map gloss → video — retrieve and stitch WLASL2000 clips for the predicted gloss sequence.

Stack

Python, PyTorch, Hugging Face Transformers, Jupyter.

Status

Complete.

Code: github.com/awais-de/asl