JAIT 2024 Vol.15(6): 723-734
doi: 10.12720/jait.15.6.723-734

Bridging the Gap: A Hybrid Approach to Medical Relation Extraction Using Pretrained Language Models and Traditional Machine Learning

Nesma A. Hassan1 , Rania A. Abul Seoud2 , Dina A. Salem1,*

  • 1Department of Computer and Software Engineering, Faculty of Engineering, Misr University for Science and Technology, Giza, Egypt
  • 2Department of Electrical Engineering, Faculty of Engineering, Fayoum University, Fayoum, Egypt

* Corresponding author

  • Manuscript receivedNovember 19, 2023
  • revisedFebruary 19, 2024
  • acceptedMarch 14, 2024
  • publishedJune 13, 2024

Abstract

Feature engineering can be time-consuming and challenging, requiring expertise in Natural Language Processing (NLP) techniques and methods. The objective of this study was to explore the use of contextual word embeddings, specifically Bidirectional Encoder Representations from Transformers (BERT)-generated word embeddings, for biomedical relation extraction. The study utilized machine learning models, including Support Vector Machines, Random Forests, and K-nearest neighbor algorithms, to classify relationships between medical entities based on these embeddings. The attention mechanism of a pre-trained BERT model was also used to capture information related to the relationship between medical entities, leading to more advanced biomedical relation extraction. The performance of the machine learning classifiers was evaluated as classification models. The proposed approach outperformed the most recent state-of-the-art model on two publicly available biomedical relation extraction datasets Chemical-Protein Interactions (ChemProt) and Drug-Drug Interactions (DDI), indicating that traditional machine-learning techniques can compete with recent advancements. Experiments on the ChemProt dataset show that the performance of the proposed model’s F1-Score is 0.778 and on the DDI dataset, F1-Score is 0.815. This study has demonstrated the potential for using contextual word embeddings and machine learning models for biomedical relation extraction, without the need for extensive manual feature engineering.

Keywords

  • Bidirectional Encoder Representations from Transformers (BERT)
  • hemical-Protein Interactions (ChemProt)
  • Drug-Drug Interactions (DDI)
  • K-Nearest Neighbor (KNN)
  • Natural Language Processing (NLP)
  • random forest
  • relation extraction
  • Support Vector Machine
JAIT-V15N6-723

How to Cite

Copied

Nesma A. Hassan, Rania A. Abul Seoud, and Dina A. Salem, "Bridging the Gap: A Hybrid Approach to Medical Relation Extraction Using Pretrained Language Models and Traditional Machine Learning," Journal of Advances in Information Technology, vol. 15, no. 6, pp. 723-734, 2024. https://doi.org/10.12720/jait.15.6.723-734

Copyright & License

Copyright © 2024 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

Article Metrics in Dimensions