SoccerLens

Explainability benchmark for soccer video event-classification models, evaluating Chefer (transformer attention-based) attribution methods on three vision backbones: SigLIP, MatchVision, and SoccerMaster.

---

Overview

This repo benchmarks how well saliency maps generated by the Chefer attribution method align with ground-truth spatial bounding boxes and temporal event windows in soccer broadcast clips.

Models evaluated

Model	Backbone	Input
SigLIP	SigLIP-base-patch16-224	Frames (spatial only)
MatchVision	VisionTimesformer (SigLIP-base + temporal)	30-frame clips
SoccerMaster	SigLIP2-large-patch16-512 + temporal gates (layer ≥16)	30-frame clips

Attribution methods

• Chefer (Spatial) — attention × gradient relevance propagation across all spatial layers (Chefer et al., ICCV 2021)
• Chefer (Temporal) — same propagation extended to temporal attention layers (MatchVision / SoccerMaster only)

Evaluation metrics

• Energy — fraction of saliency mass inside ground-truth bounding boxes
• Pointing — whether the peak activation falls inside a bounding box
• S-IoU — IoU between the thresholded saliency map and bounding-box mask
• T-IoU — IoU between predicted and ground-truth active frame sets

Annotations are stored in COCO format (annotations-coco.json), with per-second bounding boxes annotated over 13 SoccerNet-v2 event classes.

---

Repository Structure

├── annotations-coco.json          # Ground-truth bounding boxes (COCO format)
├── build_baseline_table.py        # Build LaTeX results table from eval JSONs
├── requirements_environment.txt   # Pinned dependencies
├── run_*.sh                       # SLURM job scripts (one per model × method)
├── config/                        # Model config files
├── dataset/                       # Video loading utilities
├── inference/
│   ├── chefer_matchvision.py      # Chefer spatial — MatchVision
│   ├── chefer_matchvision_temporal.py
│   ├── chefer_siglip.py           # Chefer spatial — SigLIP
│   ├── chefer_soccermaster.py     # Chefer spatial — SoccerMaster
│   ├── chefer_soccermaster_temporal.py
│   ├── coco_attribution_eval.py   # Metric computation (Energy / Pointing / IoU)
│   ├── inference_utils.py         # Shared utilities and attention wrapping
│   └── visualization_video.py
└── model/
    ├── SigLIP_classifier.py
    ├── MatchVision/
    └── SoccerMaster/

---

Setup

conda create -n soccerexplain python=3.10
conda activate soccerexplain
pip install -r requirements_environment.txt

Pre-trained checkpoints

# MatchVision
wget https://huggingface.co/Homie0609/UniSoccer/resolve/main/pretrained_classification.pth

# SoccerMaster
git clone https://huggingface.co/xleprime/SoccerMaster

• Place the MatchVision .pth file under model/MatchVision/pretrained_models/ (or pass --checkpoint_path)
• Place the cloned SoccerMaster directory under model/SoccerMaster/pretrained_models/SoccerMaster/
• SigLIP weights are downloaded automatically from HuggingFace (google/siglip-base-patch16-224 / google/siglip2-large-patch16-512)

---

Running Inference

Each combination of model × method has a corresponding SLURM script. Edit the path variables at the top of each script (SOCCER_DIR, CHECKPOINT_DIR, HF_HOME, --account), then submit:

# Examples
sbatch run_chefer_matchvision.sh
sbatch run_chefer_matchvision_temporal.sh
sbatch run_chefer_soccermaster.sh
sbatch run_chefer_soccermaster_temporal.sh
sbatch run_chefer_siglip.sh

Each script writes per-video attribution visualizations and an *_eval_results.json to its output_*/ directory.

---

Building the Results Table

After all jobs finish, generate the LaTeX table:

python build_baseline_table.py

The script reads output_*/ directories and prints a table of Energy / Pointing / S-IoU / T-IoU across all three label-set tiers (P, P+S, P+S+C).

---

References

• Chefer, H., Gur, S., & Wolf, L. (2021). Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers. ICCV 2021. [arXiv]
• SoccerNet-v2 dataset and event taxonomy: [SoccerNet]