Abstract—In the domain of analyzing patent documents, evaluating the semantic similarity between phrases poses a considerable challenge, particularly accentuating the inherent complexities associated with Cooperative Patent Classification (CPC) research. Firstly, this study addresses these challenges, recognizing early CPC work while acknowledging past struggles with language barriers and document intricacy. Secondly, it underscores the persisting difficulties of CPC research. To overcome these challenges and bolster the CPC system, this paper presents two key innovations. Firstly, it introduces an ensemble approach that incorporates four Bidirectional Encoder Representations from Transformers (BERT)-related models, enhancing semantic similarity accuracy through weighted averaging. Secondly, a novel text preprocessing method tailored for patent documents is introduced, featuring a distinctive input structure with token scoring those aids in capturing semantic relationships during CPC context training, utilizing Binary Cross-Entropy Loss (BCELoss). Our experimental findings conclusively establish the effectiveness of both our Ensemble Model and novel text processing strategies when deployed on the U.S. Patent Phrase to Phrase Matching dataset.
 
Keywords—Cooperative Patent Classification (CPC), data processing method, Decoding-enhanced Bidirectional Encoder Representations from Transformers (DeBERTa)


Copyright © 2024 by the authors. This is an open access article distributed under the Creative Commons Attribution License (CC BY-NC-ND 4.0), which permits use, distribution and reproduction in any medium, provided that the article is properly cited, the use is non-commercial and no modifications or adaptations are made.