This is the repository for our paper "Tasty Burgers, Soggy Fries: Probing Aspect Robustness in Aspect-Based Sentiment Analysis" (EMNLP 2020).
Authors: Xiaoyu Xing*, Zhijing Jin*, Di Jin, Bingning Wang, Qi Zhang, and Xuanjing Huang.
We provide a Aspect Robustness Probing test set for SemEval 2014 Aspect-Based Sentiment Analysis (ABSA).
- Our new enriched test sets are at data/arts_testset
- Our
AspectSetmentioned in the paper Section 2.3 (Table 4) is provided in data/aspectset
We generate our new probing test set by three automatic strategies:
- RevTgt (sentence with a red background): Reverse the sentiment of the Target aspect.
- RevNon (sentence with a green background): Reverse the sentiment of the Non-target aspect.
- AddDiff (sentence with a blue background): Add new aspects with Different sentiment.
We probed nine ABSA models (as mentioned in our paper).
- Their outputs on SemEval 2014 are in the output folder.
If you have a new ABSA dataset, you can run our code to generate you own aspect robustness probing test set.
python code/main.py -dataset_name laptop
- Version of allennlp package: You can install allennlp-2.5.0 with the Predictor https://s3-us-west-2.amazonaws.com/allennlp/models/elmo-constituency-parser-2020.02.10.tar.gz
If needed, see a dump of all the trained models and output files here.
If you have more questions, please feel free to submit a GitHub issue.