A performance study on fine-tuned large language models in the Legal Case Entailment task
Published in The 6th Competition on Legal Information Extraction/Entailment (COLIEE-2019), 2019
Abstract
Deep learning based approaches achieved significant advances in various Natural Language Processing (NLP) tasks. However, such approaches have not yet been evaluated in the legal domain compared to other domains such as news articles and colloquial texts. Since creating annotated data in the legal domain is expensive, applying deep learning models to the domain has been challenging. A fine-tuning approach can alleviate the situation; it allows a model trained with a large out-domain data set to be retrained on a smaller in-domain data set. A fine-tunable language model “BERT” was proposed and achieved state-of-the-art in various NLP tasks. In this paper, we explored the fine-tuning based approach in legal textual entailment task using COLIEE task 2 data set. The experimental results show that fine-tuning approach improves the performance, achieving F 1 = 0.50 with COLIEE task 2 dry run data.
Recommended citation:
Hiroaki Yamada and Takenobu Tokunaga. 2019. A performance study on fine-tuned large language models in the Legal Case Entailment task. In Proceedings of the 6th Competition on Legal Information Extraction/Entailment (COLIEE-2019).