Lot 2 : ANNOTATION AUTOMATIQUE DES DECISIONS
NER sur décisions judiciaires françaises : CamemBERT
Judiciaire ou méthode ensembliste ?
Conférence EGC 2022
lien papier
Sid Ali Mahmoudi∗, Charles Condevaux∗, Bruno Mathis∗∗, Guillaume Zambrano∗,
Stéphane Mussard∗
- Détection automatique des entités dans une décision : les personnes physiques, leurs fonctions, la date du jugement, la ville, la juridiction, le numéro RG de la décision, et les normes juridiques.
- Un transformer « CamemBERT Judiciaire » a été créé
- Une méthode d’ensemble basée sur des Bi-LSTM-CRF a été créée
- Explication du code + data annotées : lien
Résumé :
Nous étudions dans cet article les apports respectifs de différentes représentations de mots, de la méthode ensembliste et d’un transformer spécialisé que nous appelons CamemBERT judiciaire, sur la tâche de recherche d’entités nommées dans les décisions de justice françaises. Nous comparons les performances des modèles BiLSTM-CRFs entre eux, individuellement ou constitués en ensembles, et avec le modèle de Tagny (2019) pris comme référence à battre. Les résultats obtenus montrent une amélioration.
* * *
*
ART 700 identification in Judicial Decisions: Comparing ML
techniques and CamemBERT
Submitted to STATS spceial issue on predictive Justice
lien
Sid Ali Mahmoudi∗, Charles Condevaux∗, Guillaume Zambrano∗,
Stéphane Mussard∗
Résumé :
In jurimetry, accurately predicting the outcome of a trial on the basis of court decisions is a challenging task, inherent in the structure of legal documents. Prior to any predictions, the models 2
must be trained on documents relating to specific legal cases. This paper focuses on a classification task based on the detection of Article 700 in judgments, which is a rule indicating whether the 4
plaintiff or defendant is entitled to reimbursement of their legal costs. Our experiments show that conventional machine learning models trained on word and document frequencies can be competitive. However, the use of transformer models specialized on legal language, such as Judicial CamemBERT, achieves higher accuracies.
* * *
*
Linking Appellate Judgments to Tribunal Judgments – Benchmarking Different ML Techniques
Conférence JURIX 2022
lien papier
Charles Condevaux∗, Bruno Mathis∗∗, Sid Ali Mahmoudi∗, Guillaume Zambrano∗,
Stéphane Mussard∗
Résumé :
The typical judicial pathway is made of a judgment by a tribunal followed by a decision of an appellate court. However, the link between both documents is sometimes difficult to establish because of missing, incorrect or badly formatted references, pseudonymization, or poor drafting specific to each jurisdiction. This paper first shows that it is possible to link court decisions related to the same case although they are from different jurisdictions using manual rules. The use of deep learning afterwards significantly reduces the error rate in this task. The experiments are conducted between the Commercial Court of Paris and Appellate Courts.
* * *
*
Exploring SBERT and Mixup Data Augmentation in Rhetorical Role Labeling of Indian Legal Sentences
Conférence CICLE 2022
Alexandre G. de Lima, Mohand Boughanem, Eduardo Henrique da S. Aranha, Taoufiq Dkaki, José G. Moreno
Résumé :
The rise of the Transformer architecture allowed the creation of huge pre-trained language models that led to new state-of-the-art achievements in general-purpose natural language applications. Such models also have the potential to boost domain-specific applications and so this motivates us to evaluate the performance of SBERT, a Transformer architecture-based model, in a case study of rhetorical role labeling of sentences in legal documents. We perform experiments using classification models and compare their performances through lexical features and semantic features generated by SBERT. We also employ the mixup data augmentation method with the semantic features. From the results, we conclude that exploiting the mixup method is beneficial and that the semantic features have a limited enhancing effect on the classification models of our case study.
* * *
*
Effect of Hierarchical Domain-specific Language Models and Attention in the Classification of Decisions for Legal Cases
CIRCLE 2022
Nishchal Prasad, Mohand Boughanem, Taoufiq Dkaki
Résumé :
In order to automate a judicial process, a model to accurately predict the best probable decision of alegal case from the facts is desired. We try to explore this task of decision prediction on unannotatedand unstructured large legal documents with only the results of the decision. For this task, we explored many available deep learning architectures including transformer-based language models (BERT, XLNet), domain-specific language model (LEGAL-BERT), attention mechanism, and sequence models (LSTM, GRU). With the different combinations of these architectures and methods, we ran extensive experiments upon an English legal dataset called ILDC and developed many hierarchical domain-specific language models all of which improves the performance by at least 2 metric points, with the best amongst them giving an improvement of approximately 3 metric points on the previous baseline models on this dataset,
showing that the domain-specific models; when fine-tuned; adapts well to a domain of the same nature but with a different syntax, lexicon and grammar setting, and improves the performance significantly
* * *
*
Evaluating Recent Legal Rhetorical Role Labeling Approaches Supported by Transformer Encoder
Conférence BRACIS 2023
Intelligent Systems
Alexandre G. Lima, Jose G Moreno, Mohand Boughanem, Taoufiq Dkaki and Eduardo Aranha
Résumé :
Pre-trained Transformer models have been used to improve the results of several NLP tasks, which includes the Legal Rhetorical Role Labeling (Legal RRL) one. This task assigns semantic functions, such as fact and argument, to sentences from judgment documents. Several Legal RRL works exploit pre-trained Transformers to encode sentences but only a few employ approaches other than fine-tuning to improve the performance of models. In this work, we implement three of such approaches and evaluate them over the same datasets to achieve a better perception of their impacts. In our experiments, approaches based on data augmentation and positional encoders do not provide performance gains to our models. Conversely, the models based on the DFCSC approach overcome the appropriate baselines, and they do remarkably well as the lowest and highest improvements respectively are 5.9% and 10.4%.
* * *
*
Legal Rhetorical Role Labeling Supported by Dynamic-Filled Contextualized Sentence Chunks
Conférence SEMEVAL 2023
Association for Computational Linguistics
Alexandre G. Lima, Jose G Moreno and Eduardo Aranha
Résumé :
This work presents and evaluates an approach to efficiently leverage the context exploitation ability of pre-trained Transformer models as a way of boosting the performance of models tackling the Legal Rhetorical Role Labeling task. The core idea is to feed the model with sentence chunks that are assembled in a way that avoids the insertion of padding tokens and the truncation of sentences and, hence, obtain better sentence embeddings. The achieved results show that our proposal is efficient, despite its simplicity, since models based on it overcome strong baselines by 3.76% in the worst case and by 8.71% in the best case.
* * *
*
