Digitizing NFL playbooks
Synthetic data and semantic segmentation for converting play diagrams into structured objects.
A client needed to convert proprietary American football play diagrams into structured objects but had no annotated dataset. I designed a computer-vision pipeline that learned to identify and extract routes entirely from synthetic examples.
Project period · 2025 – 2026
The constraint
The target scans contained thin, branching routes, orientation changes, annotations, colors, and domain-specific visual noise. A simple object detector that works on clean synthetic cards fails to generalize to real playbooks.
The system
I built a synthetic play generator and expanded its variability through adversarial augmentation informed by the real, unlabeled examples. I then trained a U-Net-style semantic-segmentation model in TensorFlow with multiple GPUs. Finally I devised an algorithm to trace the routes from the predicted masks to produce structured play objects.
The loss combined focal and Dice terms with topology-aware centerline supervision and hard-example mining. This addressed both the extreme foreground/background imbalance and the operationally important failure mode of broken route segments.
Evaluation
Synthetic validation data supported repeatable model selection, but the real objective was performance on scanned client cards. Subject-matter experts annotated a small real-world evaluation set, on which the system achieved greater than 80% intersection over union despite using no real images for training.