Evaluation of Unsupervised Entity and Event Salience Estimation

Authors

  • Jiaying Lu Emory University
  • Jinho D Choi Emory University

DOI:

https://doi.org/10.32473/flairs.v34i1.128482

Keywords:

salience estimation, entity extraction, event extraction

Abstract

Salience Estimation aims to predict term importance in documents.
Due to few existing human-annotated datasets and the subjective notion of salience, previous studies typically generate pseudo-ground truth for evaluation. However, our investigation reveals that the evaluation protocol proposed by prior work is difficult to replicate, thus leading to few follow-up studies existing. Moreover, the evaluation process is problematic: the entity linking tool used for entity matching is very noisy, while the ignorance of event argument for event evaluation leads to boosted performance. In this work, we propose a light yet practical entity and event salience estimation evaluation protocol, which incorporates the more reliable syntactic dependency parser. Furthermore, we conduct a comprehensive analysis among popular entity and event definition standards, and present our own definition for the Salience Estimation task to reduce noise during the pseudo-ground truth generation process. Furthermore, we construct dependency-based heterogeneous graphs to capture the interactions of entities and events. The empirical results show that both baseline methods and the novel GNN method utilizing the heterogeneous graph consistently outperform the previous SOTA model in all proposed metrics.

Downloads

Published

2021-04-18

How to Cite

Lu, J., & Choi, J. D. (2021). Evaluation of Unsupervised Entity and Event Salience Estimation. The International FLAIRS Conference Proceedings, 34. https://doi.org/10.32473/flairs.v34i1.128482

Issue

Section

Main Track Proceedings