2024
doi: 10.12720/jait.15.3.446-450
Similarity Matching for Patent Documents Using Ensemble BERT-Related Model and Novel Text Processing Method
Liqiang Yu1
,
Bo Liu2,* ,
Qunwei Lin3 ,
Xinyu Zhao3 ,
Chang Che4
- 1Division of the Social Sciences, The University of Chicago, Irvine, USA
- 2School of Software Technology, Zhejiang University, Shanghai, China
- 3College of Graduate and Professional Studies, Trine University, Phoenix, USA
- 4School of Engineering and Applied Science, The George Washington University, Atlanta, USA
- Manuscript receivedSeptember 21, 2023
- revisedNovember 3, 2023
- acceptedNovember 18, 2023
- publishedMarch 28, 2024
Abstract
In the domain of analyzing patent documents, evaluating the semantic similarity between phrases poses a considerable challenge, particularly accentuating the inherent complexities associated with Cooperative Patent Classification (CPC) research. Firstly, this study addresses these challenges, recognizing early CPC work while acknowledging past struggles with language barriers and document intricacy. Secondly, it underscores the persisting difficulties of CPC research. To overcome these challenges and bolster the CPC system, this paper presents two key innovations. Firstly, it introduces an ensemble approach that incorporates four Bidirectional Encoder Representations from Transformers (BERT)-related models, enhancing semantic similarity accuracy through weighted averaging. Secondly, a novel text preprocessing method tailored for patent documents is introduced, featuring a distinctive input structure with token scoring those aids in capturing semantic relationships during CPC context training, utilizing Binary Cross-Entropy Loss (BCELoss). Our experimental findings conclusively establish the effectiveness of both our Ensemble Model and novel text processing strategies when deployed on the U.S. Patent Phrase to Phrase Matching dataset.
Keywords
- Cooperative Patent Classification (CPC)
- data processing method
- Decoding-enhanced Bidirectional Encoder Representations from Transformers (DeBERTa)
How to Cite
Liqiang Yu, Bo Liu, Qunwei Lin, Xinyu Zhao, and Chang Che, "Similarity Matching for Patent Documents Using Ensemble BERT-Related Model and Novel Text Processing Method," Journal of Advances in Information Technology, vol. 15, no. 3, pp. 446-450, 2024. https://doi.org/10.12720/jait.15.3.446-450
Copyright & License
Copyright © 2024 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
