Task Overview
The PsyCoMark task is divided into two synergistic subtasks designed to promote generalizable, interpretable models for conspiracy theory detection.
Subtask 1: Conspiracy Marker Extraction
Identify spans of text that express core conspiracy markers grounded in evolutionary psychology. Each document may include zero or multiple overlapping spans. Evaluation will use an overlap-based macro F1-score for each marker.
- Actor – Mentions of individual or group agents.
- Action – Descriptions of what the actor is doing.
- Effect – Consequences of the actions.
- Victim – Individuals or groups being harmed.
- Evidence – Claims or proof used to support the theory.
Subtask 2: Conspiracy Detection
Classify Reddit comments as either conspiracy-related or not conspiracy-related. The task is designed to benefit from the extracted markers but can be approached independently. Performance will be evaluated using macro-averaged F1-score.
Participants are encouraged (but not required) to build joint models for both subtasks.