Baselines & Evaluation
We provide strong baseline models to help participants get started. All models are implemented in the official starter pack.
Baseline Models
- Model 1: DistilBERT with token classification head (Marker Extraction)
- Model 2: DistilBERT with sequence classification head (Binary Conspiracy Detection)
- Model 3: Same as Model 2 but includes special marker tokens for end-to-end joint learning
Performance
- Marker Extraction: F1 ≈ 0.20 (macro)
- Conspiracy Detection: F1 ≈ 0.63 (Model 2) / 0.67 (Model 3)
Participants are encouraged to improve on these baselines, especially in underrepresented markers or more difficult subreddits.
Access the starter code and evaluation scripts at:
https://github.com/hide-ous/semeval26_starter_pack