Home > Published Issues > 2024 > Volume 15, No. 3, 2024 >
JAIT 2024 Vol.15(3): 446-450
doi: 10.12720/jait.15.3.446-450

Similarity Matching for Patent Documents Using Ensemble BERT-Related Model and Novel Text Processing Method

Liqiang Yu 1, Bo Liu 2,*, Qunwei Lin 3, Xinyu Zhao 3, and Chang Che 4
1. Division of the Social Sciences, The University of Chicago, Irvine, USA
2. School of Software Technology, Zhejiang University, Shanghai, China
3. College of Graduate and Professional Studies, Trine University, Phoenix, USA
4. School of Engineering and Applied Science, The George Washington University, Atlanta, USA
Email: rexyu@outlook.com (L.Y.); lubyliu45@gmail.com (B.L.); linqunwei1030@outlook.com (Q.L.); lution798@gmail.com (X.Z.); liamche1123@outlook.com (C.C.)
*Corresponding author

Manuscript received September 21, 2023; revised November 3, 2023; accepted November 18, 2023; published March 28, 2024.

Abstract—In the domain of analyzing patent documents, evaluating the semantic similarity between phrases poses a considerable challenge, particularly accentuating the inherent complexities associated with Cooperative Patent Classification (CPC) research. Firstly, this study addresses these challenges, recognizing early CPC work while acknowledging past struggles with language barriers and document intricacy. Secondly, it underscores the persisting difficulties of CPC research. To overcome these challenges and bolster the CPC system, this paper presents two key innovations. Firstly, it introduces an ensemble approach that incorporates four Bidirectional Encoder Representations from Transformers (BERT)-related models, enhancing semantic similarity accuracy through weighted averaging. Secondly, a novel text preprocessing method tailored for patent documents is introduced, featuring a distinctive input structure with token scoring those aids in capturing semantic relationships during CPC context training, utilizing Binary Cross-Entropy Loss (BCELoss). Our experimental findings conclusively establish the effectiveness of both our Ensemble Model and novel text processing strategies when deployed on the U.S. Patent Phrase to Phrase Matching dataset.
Keywords—Cooperative Patent Classification (CPC), data processing method, Decoding-enhanced Bidirectional Encoder Representations from Transformers (DeBERTa)

Cite: Liqiang Yu, Bo Liu, Qunwei Lin, Xinyu Zhao, and Chang Che, "Similarity Matching for Patent Documents Using Ensemble BERT-Related Model and Novel Text Processing Method," Journal of Advances in Information Technology, Vol. 15, No. 3, pp. 446-450, 2024.

Copyright © 2024 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.