TwiBot-22

This is the official repository of TwiBot-22 @ NeurIPS 2022, Datasets and Benchmarks Track.

Introduction

TwiBot-22 is the largest and most comprehensive Twitter bot detection benchmark to date. Specifically, TwiBot-22 is designed to address the challenges of limited dataset scale, imcomplete graph structure, and low annotation quality in previous datasets. For more details, please refer to the TwiBot-22 paper and statistics.

Dataset Format

Each dataset contains node.json (or tweet.json, user.json, list.json, and hashtag.json for TwiBot-22), label.csv, split.csv and edge.csv (for datasets with graph structure). See here for a detailed description of these files.

How to download TwiBot-22 dataset

TwiBot-22 is available at Google Drive.

Please apply for access by contacting shangbin at cs.washington.edu and clearly state your institution, your research advisor (if any), and your use case of TwiBot-22.

How to download other datasets

For TwiBot-20, visit the TwiBot-20 github repository.

For other datasets, please visit the Bot Repository.

After downloading these datasets, you can transform them into the 4-file format detailed in "Dataset Format". Alternatively, you can directly download our preprocessed version:

For TwiBot-20, visit the TwiBot-20 github repository, apply for TwiBot-20 access, and there will be a TwiBot-20-Format22.zip in the TwiBot-20 Google Drive link.

For other datasets, you can directly download them from Google Drive. You should adhere to the license of each dataset, the "Content redistribution" section of the Twitter Developer Agreement and Policy, the rules set by the Bot Repository, and only use these datasets for research purposes.

Requirements

• pip: pip install -r requirements.txt
• conda : conda install --yes --file requirements.txt

How to run baselines

1. clone this repo by running git clone https://github.com/LuoUndergradXJTU/TwiBot-22.git
2. make dataset directory mkdir datasets and download datasets to ./datasets
3. change directory to src/{name_of_the_baseline}
4. run experiments under the guidance of corresponding readme.md

Baseline Overview

baseline	paper	acc on Twibot-22	f1 on Twibot-22	type	tags
Abreu et al.	link	0.7066	0.5344	F	random forest
Alhosseini et al.	link	0.4772	0.3810	F G	gcn
BGSRD	link	0.7188	0.2114	F	BERT GAT
Bot Hunter	link	0.7279	0.2346	F	random forest
Botometer	link	0.4987	0.4257	F T G
BotRGCN	link	0.7966	0.5750	F T G	BotRGCN
Cresci et al.	link	-	-	T	DNA
Dehghan et al.	link	-	-	F T G	Graph
Efthimion et al.	link	0.7408	0.2758	F T	efthimion
EvolveBot	link	0.7109	0.1409	F T G	random forest
FriendBot	link	-	-	F T G	random forest
Kipf et al.	link	0.7839	0.5496	F T G	Graph Neural Network
Velickovic et al.	link	0.7948	0.5586	F T G	Graph Neural Network
GraphHist	link	-	-	F T G	random forest
Hayawi et al.	link	0.7650	0.2474	F	lstm
HGT	link	0.7491	0.3960	F T G	Graph Neural Networks
SimpleHGN	link	0.7672	0.4544	F T G	Graph Neural Networks
Kantepe et al.	link	0.7640	0.5870	F T	random forest
Knauth et al.	link	0.7125	0.3709	F T G	random forest
Kouvela et al.	link	0.7644	0.3003	F T	random forest
Kudugunta et al.	link	0.6587	0.5167	F	SMOTENN, random forest
Lee et al.	link	0.7628	0.3041	F T	random forest
LOBO	link	0.7570	0.3857	F T	random forest
Miller et al.	link	0.3037	0.4529	F T	k means
Moghaddam et al.	link	0.7378	0.3207	F G	random forest
NameBot	link	0.7061	0.0050	F	Logistic Regression
RGT	link	0.7647	0.4294	F T G	Graph Neural Networks
RoBERTa	link	0.7207	0.2053	F T	RoBERTa
Rodriguez-Ruiz	link	0.4936	0.5657	F T G	SVM
Santos et al.	link	-	-	F T	decision tree
SATAR	link	-	-	F T G
SGBot	link	0.7508	0.3659	F T	random forest
T5	link	0.7205	0.2027	T	T5
Varol et al.	link	0.7392	0.2754	F T	random forest
Wei et al.	link	0.7020	0.5360	T

where - represents the baseline could not scale to TwiBot-22 dataset

Precision

Precision	Botometer-feedback-2019	Cresci-2015	Cresci-2017	Cresci-rtbust-2019	Cresci-stock-2018	Gilani-2017	Midterm-2018	Twibot-20	Twibot-22
Abreu et al.	63.63 $_{3.60}$	99.05 $_{0.21}$	98.34 $_{0.13}$	78.57 $_{1.44}$	75.45 $_{0.45}$	76.82 $_{1.20}$	97.28 $_{0.07}$	72.20 $_{0.52}$	50.92 $_{0.10}$
Alhosseini et al.	-	87.69 $_{1.23}$	-	-	-	-	-	57.81 $_{0.43}$	29.99 $_{3.08}$
BGSRD	27.50 $_{28.2}$	86.52 $_{0.64}$	75.85 $_{0.00}$	58.13 $_{11.1}$	52.78 $_{0.75}$	25.43 $_{23.2}$	84.40 $_{0.93}$	67.64 $_{2.26}$	22.55 $_{30.9}$
BotHunter	-	98.55 $_{0.56}$	98.65 $_{0.05}$	81.92 $_{2.04}$	84.29 $_{0.10}$	78.99 $_{0.96}$	99.44 $_{0.15}$	72.77 $_{0.25}$	68.09 $_{0.36}$
Botometer	21.05 -	50.54 -	93.35 -	65.22 -	68.50 -	62.99 -	31.18 -	55.67 -	30.81 -
BotRGCN	-	95.51 $_{1.02}$	-	-	-	-	-	84.52 $_{0.54}$	74.81 $_{2.22}$
Cresci	-	0.59 -	12.96 -	-	-	-	-	7.66 -	-
Dehghan et al.	-	96.15 $_{0.00}$	-	-	-	-	-	94.72 $_{0.00}$	-
Efthimion et al.	0.00 $_{0.00}$	93.82 $_{0.00}$	94.58 $_{0.00}$	68.29 $_{0.00}$	82.75 $_{0.00}$	37.50 $_{0.00}$	98.01 $_{0.00}$	64.20 $_{0.00}$	77.78 $_{0.00}$
EvolveBot	-	85.03 $_{3.77}$	-	-	-	-	-	66.93 $_{0.60}$	56.38 $_{0.40}$
FriendBot	-	95.29 $_{1.62}$	77.55 $_{0.81}$	-	-	-	-	72.64 $_{0.52}$	-
GCN	-	95.59 $_{0.69}$	-	-	-	-	-	75.23 $_{3.08}$	71.19 $_{1.28}$
GAT	-	96.10 $_{0.71}$	-	-	-	-	-	81.39 $_{1.18}$	76.23 $_{1.39}$
GraphHist	-	73.12 $_{0.10}$	-	-	-	-	-	51.27 $_{0.20}$	-
Hayawi et al.	25.00 $_{0.06}$	92.96 $_{0.03}$	95.47 $_{0.01}$	48.82 $_{0.01}$	50.73 $_{0.03}$	51.44 $_{0.05}$	85.30 $_{0.00}$	71.61 $_{0.01}$	80.00 $_{0.27}$
HGT	-	94.80 $_{0.49}$	-	-	-	-	-	85.55 $_{0.31}$	68.22 $_{2.71}$
SimpleHGN	-	95.68 $_{0.90}$	-	-	-	-	-	84.76 $_{0.46}$	72.57 $_{2.79}$
Kantepe et al.	-	81.30 $_{1.40}$	83.00 $_{0.90}$	-	-	-	-	63.40 $_{2.10}$	78.60 $_{1.80}$
Knauth et al.	57.41 $_{0.00}$	85.70 $_{0.00}$	91.56 $_{0.00}$	57.41 $_{0.00}$	99.89 $_{0.00}$	35.17 $_{0.00}$	99.91 $_{0.00}$	96.56 $_{0.00}$	-
Kouvela et al.	48.00 $_{4.47}$	99.54 $_{0.18}$	99.24 $_{0.13}$	82.27 $_{2.00}$	82.17 $_{0.46}$	79.69 $_{1.09}$	97.56 $_{0.04}$	79.33 $_{0.44}$	69.30 $_{0.14}$
Kudugunta et al.	56.67 $_{10.8}$	100.0 $_{0.00}$	98.53 $_{0.19}$	66.09 $_{2.35}$	54.87 $_{0.47}$	85.44 $_{2.42}$	99.06 $_{0.16}$	80.40 $_{0.60}$	44.31 $_{0.00}$
Lee et al.	58.97 $_{3.29}$	98.65 $_{0.14}$	99.56 $_{0.08}$	79.37 $_{2.97}$	84.75 $_{0.42}$	77.58 $_{1.31}$	97.36 $_{0.07}$	76.60 $_{0.37}$	67.23 $_{0.29}$
LOBO	-	98.47 $_{0.63}$	99.30 $_{0.08}$	-	-	-	-	74.83 $_{0.08}$	75.43 $_{0.15}$
Miller et al.	0.00 $_{0.00}$	72.07 $_{0.00}$	77.21 $_{0.18}$	52.17 $_{0.00}$	54.78 $_{0.00}$	48.89 $_{0.00}$	83.85 $_{0.00}$	60.71 $_{0.20}$	29.46 $_{0.00}$
Moghaddam et al.	-	98.33 $_{0.26}$	-	-	-	-	-	72.29 $_{0.67}$	67.61 $_{0.10}$
NameBot	45.45 $_{0.00}$	76.81 $_{0.00}$	80.39 $_{0.03}$	65.00 $_{0.00}$	58.34 $_{0.00}$	58.21 $_{0.00}$	86.93 $_{0.00}$	58.72 $_{0.00}$	67.73 $_{0.00}$
RGT	-	96.38 $_{0.59}$	-	-	-	-	-	85.15 $_{0.28}$	75.03 $_{0.85}$
RoBERTa	-	97.58 $_{0.27}$	92.43 $_{0.99}$	-	-	-	-	73.88 $_{1.06}$	63.28 $_{0.90}$
Rodriguez-Ruiz et al.	-	78.64 $_{0.00}$	79.47 $_{0.00}$	-	-	-	-	61.60 $_{0.00}$	33.23 $_{0.00}$
Santos et al.	50.00 $_{0.00}$	72.86 $_{0.00}$	81.71 $_{0.00}$	75.68 $_{0.00}$	65.39 $_{0.00}$	32.26 $_{0.00}$	88.05 $_{0.00}$	62.73 $_{0.00}$	-
SATAR	-	90.66 $_{0.67}$	-	-	-	-	-	81.50 $_{1.45}$	-
SGBot	59.70 $_{3.91}$	99.45 $_{0.20}$	98.26 $_{0.17}$	83.08 $_{2.60}$	83.90 $_{0.29}$	82.68 $_{1.88}$	99.35 $_{0.22}$	76.40 $_{0.40}$	73.11 $_{0.18}$
T5	-	91.04 $_{0.29}$	94.48 $_{0.65}$	-	-	-	-	72.19 $_{0.84}$	63.27 $_{0.71}$
Varol et al.	-	92.22 $_{0.66}$	-	-	-	-	-	78.04 $_{0.61}$	75.74 $_{0.31}$
Wei et al.	-	91.70 $_{1.70}$	85.90 $_{1.90}$	-	-	-	-	61.00 $_{2.10}$	62.70 $_{1.80}$

Recall

Recall	Botometer-feedback-2019	Cresci-2015	Cresci-2017	Cresci-rtbust-2019	Cresci-stock-2018	Gilani-2017	Midterm-2018	Twibot-20	Twibot-22
Abreu et al.	46.66 $_{3.00}$	62.13 $_{0.97}$	91.97 $_{0.69}$	89.18 $_{1.40}$	75.67 $_{0.73}$	58.87 $_{2.75}$	98.63 $_{0.08}$	82.81 $_{0.51}$	11.73 $_{0.06}$
Alhosseini et al.	-	97.16 $_{0.81}$	-	-	-	-	-	95.69 $_{1.93}$	56.75 $_{17.69}$
BGSRD	8.57 $_{8.52}$	95.56 $_{2.02}$	100.0 $_{0.00}$	35.14 $_{20.6}$	70.40 $_{26.1}$	60.00 $_{54.8}$	97.66 $_{3.66}$	73.19 $_{7.49}$	19.90 $_{27.2}$
BotHunter	-	91.48 $_{4.16}$	85.40 $_{0.19}$	83.02 $_{2.95}$	79.92 $_{0.54}$	62.29 $_{3.47}$	99.66 $_{0.06}$	86.75 $_{0.46}$	14.07 $_{0.12}$
Botometer	57.14 -	98.95 -	99.69 -	100.0 -	94.96 -	89.91 -	87.88 -	50.82 -	69.80 -
BotRGCN	-	99.17 $_{0.25}$	-	-	-	-	-	90.19 $_{1.72}$	46.80 $_{2.76}$
Cresci	-	66.67 -	95.30 -	-	-	-	-	67.47 -	-
Dehghan et al.	-	83.88 $_{0.00}$	-	-	-	-	-	82.19 $_{0.00}$	-
Efthimion et al.	0.00 $_{0.00}$	94.38 $_{0.00}$	89.23 $_{0.00}$	75.68 $_{0.00}$	58.02 $_{0.00}$	2.80 $_{0.00}$	94.04 $_{0.00}$	70.63 $_{0.00}$	16.76 $_{0.00}$
EvolveBot	-	95.83 $_{0.66}$	-	-	-	-	-	72.81 $_{0.41}$	8.04 $_{0.05}$
FriendBot	-	100.0 $_{0.00}$	100.0 $_{0.00}$	-	-	-	-	88.94 $_{0.59}$	-
GCN	-	98.81 $_{0.20}$	-	-	-	-	-	87.62 $_{3.31}$	44.80 $_{1.71}$
GAT	-	99.11 $_{0.51}$	-	-	-	-	-	89.53 $_{0.87}$	44.12 $_{1.65}$
GraphHist	-	100.0 $_{0.00}$	-	-	-	-	-	99.05 $_{0.20}$	-
Hayawi et al.	17.78 $_{0.06}$	79.31 $_{0.02}$	92.19 $_{0.03}$	81.25 $_{0.09}$	71.16 $_{0.07}$	28.00 $_{0.13}$	98.64 $_{0.00}$	83.50 $_{0.04}$	14.99 $_{0.05}$
HGT	-	99.11 $_{0.12}$	-	-	-	-	-	91.00 $_{0.57}$	28.03 $_{2.60}$
SimpleHGN	-	99.29 $_{0.40}$	-	-	-	-	-	92.06 $_{0.51}$	32.90 $_{1.64}$
Kantepe et al.	-	75.30 $_{1.20}$	76.10 $_{1.10}$	-	-	-	-	61.00 $_{1.90}$	46.80 $_{1.30}$
Knauth et al.	59.09 $_{0.00}$	97.40 $_{0.00}$	95.35 $_{0.00}$	51.24 $_{0.00}$	88.83 $_{0.00}$	44.00 $_{0.00}$	83.99 $_{0.00}$	76.30 $_{0.00}$	-
Kouvela et al.	20.00 $_{4.71}$	96.79 $_{0.75}$	98.98 $_{0.18}$	80.00 $_{1.48}$	78.78 $_{0.18}$	57.20 $_{2.42}$	98.92 $_{0.06}$	95.17 $_{0.14}$	19.17 $_{0.04}$
Kudugunta et al.	45.33 $_{8.69}$	60.95 $_{0.21}$	85.88 $_{0.37}$	50.67 $_{1.21}$	47.54 $_{0.60}$	35.14 $_{1.70}$	90.24 $_{0.66}$	33.47 $_{1.30}$	61.98 $_{0.00}$
Lee et al.	44.00 $_{3.65}$	98.46 $_{0.14}$	99.13 $_{0.00}$	86.45 $_{1.44}$	80.30 $_{0.63}$	60.19 $_{2.15}$	98.37 $_{0.10}$	83.66 $_{0.69}$	19.65 $_{0.15}$
LOBO	-	99.05 $_{0.13}$	96.13 $_{0.39}$	-	-	-	-	87.81 $_{0.37}$	25.91 $_{0.20}$
Miller et al.	0.00 $_{0.00}$	100.0 $_{0.00}$	99.11 $_{0.11}$	37.50 $_{0.00}$	58.89 $_{0.00}$	77.19 $_{0.00}$	99.81 $_{0.00}$	97.44 $_{0.47}$	97.89 $_{0.01}$
Moghaddam et al.	-	59.23 $_{0.32}$	-	-	-	-	-	84.38 $_{1.03}$	21.02 $_{0.07}$
NameBot	33.33 $_{0.00}$	91.12 $_{0.00}$	91.79 $_{0.00}$	70.27 $_{0.00}$	64.13 $_{0.00}$	36.45 $_{0.00}$	96.82 $_{0.00}$	70.47 $_{0.00}$	0.03 $_{0.00}$
RGT	-	99.23 $_{0.15}$	-	-	-	-	-	91.06 $_{0.80}$	30.10 $_{0.17}$
RoBERTa	-	94.11 $_{0.58}$	96.27 $_{1.05}$	-	-	-	-	72.38 $_{2.05}$	12.27 $_{1.22}$
Rodriguez-Ruiz et al.	-	99.11 $_{0.00}$	92.88 $_{0.00}$	-	-	-	-	98.75 $_{0.00}$	81.32 $_{0.00}$
Santos et al.	13.33 $_{0.00}$	85.80 $_{0.00}$	84.40 $_{0.00}$	75.68 $_{0.00}$	64.95 $_{0.00}$	9.35 $_{0.04}$	97.24 $_{0.00}$	58.13 $_{0.00}$	-
SATAR	-	99.88 $_{0.16}$	-	-	-	-	-	91.22 $_{1.82}$	-
SGBot	45.33 $_{2.98}$	63.67 $_{1.31}$	90.86 $_{0.39}$	81.62 $_{2.26}$	81.03 $_{0.90}$	63.62 $_{2.17}$	99.66 $_{0.20}$	94.91 $_{0.69}$	24.32 $_{0.09}$
T5	-	87.71 $_{0.66}$	90.26 $_{0.54}$	-	-	-	-	69.05 $_{1.46}$	12.09 $_{1.43}$
Varol et al.	-	97.40 $_{0.90}$	-	-	-	-	-	84.37 $_{0.67}$	16.83 $_{0.21}$
Wei et al.	-	75.30 $_{1.50}$	72.10 $_{1.50}$	-	-	-	-	54.00 $_{2.70}$	46.80 $_{1.40}$

F1

F1	Botometer-feedback-2019	Cresci-2015	Cresci-2017	Cresci-rtbust-2019	Cresci-stock-2018	Gilani-2017	Midterm-2018	Twibot-20	Twibot-22
Abreu et al.	53.84 $_{3.03}$	76.36 $_{0.72}$	95.04 $_{0.30}$	83.54 $_{1.04}$	76.93 $_{0.58}$	66.66 $_{0.10}$	97.95 $_{0.03}$	77.14 $_{0.46}$	53.44 $_{0.09}$
Alhosseini et al.	-	92.17 $_{0.36}$	-	-	-	-	-	72.07 $_{0.48}$	38.10 $_{5.93}$
BGSRD	13.03 $_{13.0}$	90.80 $_{0.60}$	86.27 $_{0.00}$	41.08 $_{13.0}$	58.18 $_{12.1}$	35.72 $_{32.6}$	90.50 $_{1.09}$	70.05 $_{2.60}$	21.14 $_{29.0}$
BotHunter	49.57 $_{3.12}$	97.22 $_{0.96}$	91.60 $_{3.12}$	82.90 $_{1.88}$	82.17 $_{0.20}$	69.18 $_{1.04}$	99.59 $_{0.02}$	79.09 $_{0.36}$	23.46 $_{0.09}$
Botometer	30.77 -	66.90 -	96.12 -	78.95 -	79.59 -	77.39 -	46.03 -	53.13 -	42.75 -
BotRGCN	-	97.30 $_{0.53}$	-	-	-	-	-	87.25 $_{0.73}$	57.50 $_{1.42}$
Cresci	-	1.17 -	22.81 -	-	-	-	-	13.69 -	-
Dehgan	-	88.34 $_{0.00}$	-	-	-	-	-	76.20 $_{0.00}$	-
Efthimion et al.	0.00 $_{0.00}$	94.10 $_{0.00}$	91.83 $_{0.00}$	71.79 $_{0.00}$	68.21 $_{0.00}$	05.22 $_{0.00}$	95.98 $_{0.00}$	67.26 $_{0.00}$	27.58 $_{0.00}$
EvolveBot	-	90.07 $_{1.98}$	-	-	-	-	-	69.75 $_{0.50}$	14.09 $_{0.08}$
FriendBot	-	97.58 $_{0.84}$	87.35 $_{0.52}$	-	-	-	-	79.97 $_{0.34}$	-
GCN	-	97.17 $_{0.43}$	-	-	-	-	-	80.86 $_{0.68}$	54.96 $_{0.91}$
GAT	-	97.58 $_{0.15}$	-	-	-	-	-	85.25 $_{0.38}$	55.86 $_{1.01}$
GraphHist	-	84.47 $_{8.23}$	-	-	-	-	-	67.56 $_{0.30}$	-
Hayawi et al.	20.49 $_{0.06}$	85.56 $_{0.01}$	93.78 $_{0.01}$	60.87 $_{0.03}$	60.75 $_{0.06}$	34.67 $_{0.11}$	91.48 $_{0.00}$	77.05 $_{0.02}$	24.74 $_{0.08}$
HGT	-	96.93 $_{0.24}$	-	-	-	-	-	88.19 $_{0.19}$	39.60 $_{2.11}$
SimpleHGN	-	97.28 $_{0.39}$	-	-	-	-	-	88.25 $_{0.18}$	45.44 $_{1.65}$
Kantepe et al.	-	78.17 $_{1.42}$	79.41 $_{1.27}$	-	-	-	-	62.23 $_{2.06}$	58.71 $_{1.61}$
Knauth et al.	41.27 $_{0.00}$	91.18 $_{0.00}$	93.42 $_{0.00}$	54.15 $_{0.00}$	94.03 $_{0.00}$	39.10 $_{0.00}$	91.26 $_{0.00}$	85.24 $_{0.00}$	37.09 $_{0.00}$
Kouvela et al.	28.10 $_{5.27}$	98.15 $_{0.38}$	99.11 $_{0.06}$	81.10 $_{1.03}$	80.44 $_{0.23}$	66.57 $_{1.72}$	98.23 $_{0.05}$	86.53 $_{0.26}$	30.03 $_{0.04}$
Kudugunta et al.	49.61 $_{8.20}$	75.74 $_{0.16}$	91.74 $_{0.17}$	49.22 $_{1.28}$	50.94 $_{0.38}$	49.75 $_{2.10}$	94.45 $_{0.32}$	47.26 $_{1.35}$	51.67 $_{0.00}$
Lee et al.	50.34 $_{3.16}$	98.56 $_{0.11}$	99.35 $_{0.04}$	82.74 $_{1.79}$	82.46 $_{0.36}$	67.78 $_{1.81}$	97.87 $_{0.07}$	79.98 $_{0.50}$	30.41 $_{0.20}$
LOBO	-	98.76 $_{0.26}$	97.69 $_{0.18}$	-	-	-	-	80.80 $_{0.20}$	38.57 $_{0.23}$
Miller et al.	0.00 $_{0.00}$	83.77 $_{0.00}$	86.80 $_{0.07}$	43.64 $_{0.00}$	56.76 $_{0.00}$	59.86 $_{0.00}$	91.14 $_{0.00}$	74.81 $_{0.26}$	45.29 $_{0.00}$
Moghaddam et al.	-	73.93 $_{0.21}$	-	-	-	-	-	77.87 $_{0.71}$	32.07 $_{0.03}$
NameBot	38.46 $_{0.00}$	83.36 $_{0.00}$	85.71 $_{0.02}$	67.53 $_{0.00}$	61.10 $_{0.00}$	44.83 $_{0.00}$	91.61 $_{0.00}$	65.06 $_{0.00}$	0.50 $_{0.00}$
RGT	-	97.78 $_{0.24}$	-	-	-	-	-	88.01 $_{0.41}$	42.94 $_{1.85}$
RoBERTa	-	95.86 $_{0.19}$	94.30 $_{0.18}$	-	-	-	-	73.09 $_{0.59}$	20.53 $_{1.71}$
Rodriguez-Ruiz et al.	-	87.70 $_{0.00}$	85.65 $_{0.00}$	-	-	-	-	63.10 $_{0.00}$	56.57 $_{0.00}$
Santos et al.	21.05 $_{0.00}$	78.80 $_{0.00}$	83.03 $_{0.00}$	75.68 $_{0.00}$	65.17 $_{0.00}$	14.49 $_{0.00}$	92.42 $_{0.00}$	60.34 $_{0.00}$	-
SATAR	-	95.05 $_{0.34}$	-	-	-	-	-	86.07 $_{0.70}$	-
SGBot	49.60 $_{3.43}$	77.91 $_{0.13}$	94.61 $_{0.19}$	82.26 $_{1.73}$	82.34 $_{0.11}$	72.10 $_{0.19}$	99.52 $_{0.02}$	84.90 $_{0.42}$	36.59 $_{0.18}$
T5	-	89.35 $_{0.26}$	92.32 $_{0.11}$	-	-	-	-	70.57 $_{0.39}$	20.27 $_{2.03}$
Varol et al.	-	94.73 $_{0.42}$	-	-	-	-	-	81.08 $_{0.48}$	27.54 $_{0.26}$
Wei et al.	-	82.65 $_{2.21}$	78.43 $_{1.66}$	-	-	-	-	57.33 $_{3.19}$	53.61 $_{1.36}$

Test1

model	Acc	F1	precision	recall
Moghaddam et al.	89.41 $_{0.30}$	24.98 $_{2.72}$	16.57 $_{1.97}$	50.79 $_{4.25}$
SGBot	91.87 $_{0.11}$	47.43 $_{1.21}$	76.16 $_{2.31}$	34.48 $_{1.56}$
BotHunter	91.44 $_{0.12}$	40.39 $_{0.32}$	78.28 $_{3.11}$	27.24 $_{0.52}$
GAT	91.14 $_{0.45}$	47.00 $_{2.92}$	64.83 $_{4.31}$	36.95 $_{3.04}$
BotRGCN	88.74 $_{0.29}$	65.89 $_{1.62}$	79.82 $_{2.53}$	56.23 $_{3.24}$
RGT	92.8 $_{0.45}$	23.39 $_{4.61}$	58.33 $_{11.78}$	14.66 $_{2.98}$

Test2

model	Acc	F1	precision	recall
Moghaddam et al.	83.93 $_{0.28}$	18.49 $_{0.95}$	11.58 $_{0.59}$	45.94 $_{3.35}$
SGBot	84.72 $_{0.31}$	26.00 $_{2.80}$	54.55 $_{2.80}$	17.11 $_{2.28}$
BotHunter	85.63 $_{0.31}$	23.38 $_{1.55}$	73.67 $_{9.81}$	13.95 $_{1.18}$
GAT	84.93 $_{0.23}$	30.47 $_{2.64}$	55.64 $_{2.02}$	21.05 $_{2.46}$
BotRGCN	85.59 $_{0.68}$	55.45 $_{2.77}$	67.45 $_{2.74}$	47.17 $_{3.65}$
RGT	87.1 $_{1.19}$	38.02 $_{7.21}$	58.50 $_{10.18}$	28.57 $_{6.68}$

Test3

model	Acc	F1	precision	recall
Moghaddam et al.	87.61 $_{0.20}$	22.34 $_{1.78}$	14.48 $_{1.26}$	49.00 $_{2.74}$
SGBot	89.52 $_{0.13}$	38.96 $_{1.77}$	68.97 $_{1.57}$	27.18 $_{1.85}$
BotHunter	89.53 $_{0.12}$	33.77 $_{0.45}$	76.62 $_{2.45}$	21.66 $_{0.25}$
GAT	89.09 $_{0.38}$	40.58 $_{2.68}$	61.84 $_{3.50}$	30.28 $_{2.75}$
BotRGCN	87.92 $_{0.51}$	59.46 $_{2.36}$	76.88 $_{3.71}$	48.66 $_{3.76}$
RGT	89.6 $_{0.72}$	26.89 $_{4.71}$	56.49 $_{11.34}$	18.05 $_{3.63}$

Citation

Please cite TwiBot-22 if you use the TwiBot-22 dataset or this repository

@article{feng2022twibot,
  title={TwiBot-22: Towards Graph-Based Twitter Bot Detection},
  author={Feng, Shangbin and Tan, Zhaoxuan and Wan, Herun and Wang, Ningnan and Chen, Zilong and Zhang, Binchi and Zheng, Qinghua and Zhang, Wenqian and Lei, Zhenyu and Yang, Shujie and Feng, Xinshun and Zhang, Qingyue and Wang, Hongrui and Liu, Yuhan and Bai, Yuyang and Wang, Heng and Cai, Zijian and Wang, Yanbo and Zheng, Lijing and Ma, Zihan and Li, Jundong and Luo, Minnan},
  journal={arXiv preprint arXiv:2206.04564},
  year={2022}
}

How to contribute

1. New dataset: convert the original data to the TwiBot-22 defined schema.
2. New baseline: load well-formatted dataset from the dataset directory and define your model.

Welcome PR!

Questions?

Feel free to open issues in this repository! Instead of emails, Github issues are much better at facilitating a conversation between you and our team to address your needs. You can also contact Zhaoxuan Tan through tanzhaoxuan at stu.xjtu.edu.cn.