Task 1: Persuasion Technique Detection

Definition

For the persuasion technique detection task, we offer the two subtasks, each with two different data sources, which are defined below:

Subtask A: Given a multigenre (tweet and news paragraphs of the news articles) snippet, identify whether it contains content with persuasion technique. This is a binary classification task.

Subtask B: Given a multigenre (tweet and news paragraphs of the news articles) snippet, identify the propaganda techniques used in it. This is a multilabel classification task.

Leaderboard

Subtask 1A

Subtask 1B

Datasets

The dataset consists of tweets and news paragraphs annotated with the persuasion techniques both in binary and multilabel settings. Please check the Task 1 repository.

Evaluation

To measure the performance of the systems, for all subtasks, we use micro-F1 and macro-F1, as these are multiclass/multilabel problems, where labels are imbalanced. The official evaluation measure for all subtasks is micro-F1.

Submission

Scorers, Format Checkers, and Baseline Scripts

All scripts can be found on gitlab at Task 1 repository

Submission

Guidelines

The process consists of two phases:

1. System Development Phase: This phase involves working on the dev set.
2. Final Evaluation Phase: This phase involves working on the test set, which will be released during the evaluation cycle. For each phase, please adhere to the following guidelines:

• Each team should create and maintain a single account for submissions. Please ensure all runs are submitted through this account. Submissions from multiple accounts by the same team could result in your system being not ranked in the overview paper.
• The most recent file submitted to the leaderboard will be considered your final submission.
• The output file must be named task[1/2][A/B]_any_suffix.tsv, where [1/2] is the task number and A/B is your specific subtask (for example, task1A_team_name.tsv or task2B_team_name.tsv). Failure to follow this naming convention will result in an error on the leaderboard. Subtasks include 1A, 1B, 2A and 2B.
• You are required to compress the .tsv file into a .zip file (for example, zip task1B.zip task1B.tsv) and submit it via the Codalab page.
• Please include your team name and a description of your method with each submission.
• You are permitted to submit a maximum of 200 submissions per day for each subtask.

Submission Site

System Development Phase: Please submit your results on the respective subtask tab through this competition on CodaLab.

Final Evaluation Phase: Please submit your results on the respective subtask tab through this competition on CodaLab.