DataFirst

Understanding the Relation between Noise and Bias
in Annotated Datasets

Motivation

Bias in Annotation: Annotator differences in subjective tasks introduce bias in their annotations, especially in sensitive domains like hate speech recognition, stemming from diverse backgrounds and perspectives.
Misinterpretation of Bias as Noise: Minority votes are often considered outliers by models. This causes models to perceive them as noise, leading to biased predictions favoring majority vote.
In this project, we’ll explore if perspectivist classification models effectively utilize valuable insights from instances labeled as noisy by noise-detection techniques.

annotations emoji image

Problem Statement

In the evolving landscape of machine learning, the reliability and fairness of models hinge on the quality of annotated data. However, the presence of bias, particularly in subjective tasks, has become a critical concern. This is especially prominent in sensitive domains like hate speech recognition, where annotators, stemming from diverse backgrounds and perspectives, might introduce bias in their annotations.

Datasets

	SBIC [1]	Kennedy [2]	Agree to Disagree [3]
	Toxicity and Hate Speech
# Annotators	307	7,912	819
# Annotations per annotator	479±829.6	17.1±3.8	63.7±139
# Unique texts	45318	39,565	10,440
# Annotations per text	3.2±1.2	2.3±1.0	5
# Labels	2	3	2

Methodology

Two Regimes of Classification

model 1 model 1

MODEL AND PERFORMANCE

Model

Trained 2 models for each dataset

Majority Label Model	Multi Annotator Model
Model - Roberta-Base [6] Epochs - 5 Learning Rate - 5e-5 Batch Size - 32	Model - DISCO [7] Epochs - 5 Learning Rate - 2e-3 Batch Size - 200

Performance

Dataset	F1 Score (majority)	F1 Score (multi-annotator)
Agree to Disagree	0.78	0.78
Kennedy	0.68	0.75
SBIC	0.80	0.78

Uncertainty in Machine Learning Predictions

Data Cartography summarizes training dynamics for all samples as:

Confidence: Mean of probabilities for gold label across epochs.
Variability: Standard Deviation of probabilities for gold label across epochs.

Results

DATASET CARTOGRAPHY

AGREE TO DISAGREE

agree to disagree

KENNEDY

kennedy

SBIC

sbic

SINGLE GROUND TRUTH MODEL

AGREE TO DISAGREE

agr factor vs conf

KENNEDY

agr factor vs conf

SBIC

agr factor vs conf

MULTI ANNOTATOR MODEL

AGREE TO DISAGREE

low conf mjr disagree vs conf group count]

SBIC

low conf mjr disagree vs conf group count

KENNEDY

low conf mjr disagree vs conf group count

Analysis

High Confidence in Majority model and Low Confidence in Multi-Annotator model

Mislabeled annotations:

Text	Annotation
I thought you were going to win this in a landslide	Offensive
crossing my fingers for you from Germany. Even though I am not an American.	Offensive
boy F**K you	Not Offensive
Fragility at its finest	Offensive

Findings

Single Ground Truth Model

There is correlation between human disagreement on instances and model’s uncertainty/confidence for classifying that instance.
For instances that there is more disagreement among human labelers there is also low confidence from the single-GT model.

Multi-Annotator Model

Like single gold label model, we see significant correlation between agreement between annotators and model confidence in all the datasets, with the confidence decreasing with more disagreement between the annotators.
For low confidence samples in single ground truth model, we observe multi-annotator model having high confidence for labels which disagree with majority, hence learning valuable information from samples that majority vote aggregation discards.
We see high number of annotations per annotator is necessary to model different perspectives effectively.