We address the problem of modeling and classifying American Football offense
teams’ plays in video, a challenging example of group activity analysis. Automatic play
classification will allow coaches to infer patterns and tendencies of opponents more ef-
ficiently, resulting in better strategy planning in a game. We define a football play as a
unique combination of player trajectories. To this end, we develop a framework that uses
player trajectories as inputs to MedLDA, a supervised topic model. The joint maximiza-
tion of both likelihood and inter-class margins of MedLDA in learning the topics allows
us to learn semantically meaningful play type templates, as well as, classify different
play types with 70% average accuracy. Furthermore, this method is extended to analyze
individual player roles in classifying each play type. We validate our method on a large
dataset comprising 271 play clips from real-world football games, which will be made
publicly available for future comparisons.