KR102315214B1 - Method, apparatus and system for determining similarity of patent documents using similarity scores and dissimilarity scores - Google Patents

Abstract

A method for determining the degree of similarity of patent documents is provided. The method for determining the degree of similarity of the patent document includes, by a server, inputting a target patent document into a first artificial intelligence model to obtain a similar patent document; obtaining, by the server, a plurality of sentences included in the target patent document, and acquiring a plurality of sentences included in the similar patent document; obtaining, by the server, a first sentence among a plurality of sentences included in the target patent document, and acquiring a second sentence among a plurality of sentences included in the similar patent document; inputting, by the server, the first sentence and the second sentence into a second artificial intelligence model, and obtaining evaluation results for the first sentence and the second sentence; includes

Description

Method, apparatus and system for determining similarity of patent documents using similarity score and dissimilarity score

The present invention relates to a method, apparatus and system for determining the similarity of a patent document using a similarity score and a dissimilarity score.

With the development of the 4th industry, the value of intellectual property rights is increasing. Accordingly, many people are trying to protect their technology and acquire the right to the technology, and interest in patent application for technology is increasing.

Meanwhile, in order to apply for a patent, prior art research is performed to determine whether one's own technology can be patented, and prior art research can be performed by searching various patent documents published in the past.

The most important thing in conducting a prior art search is to determine whether a similar patent document that can deny the inventive step of the target patent document exists.

However, the amount of patent documents is huge and it is virtually impossible to analyze all patent documents published in the past due to time constraints. It is a reality to obtain similar patent documents.

However, obtaining a similar patent document through a search-type input, etc., is often dependent on the ability of a person who conducts prior art research, and thus, in many cases, stable results of prior art research are not guaranteed.

Therefore, the need for a prior art research method that can guarantee stable results with a small investment of time is emerging.

Laid-open Patent Publication No. 10-2018-0110713, 2018.10.11

An object of the present invention is to provide a method, apparatus and system for determining the degree of similarity of a patent document using a similarity score and a dissimilarity score.

The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

A method for determining a degree of similarity of a patent document according to an aspect of the present invention for solving the above-described problem includes, by a server, inputting a target patent document into a first artificial intelligence model to obtain a similar patent document; obtaining, by the server, a plurality of sentences included in the target patent document, and acquiring a plurality of sentences included in the similar patent document; obtaining, by the server, a first sentence among a plurality of sentences included in the target patent document, and acquiring a second sentence among a plurality of sentences included in the similar patent document; inputting, by the server, the first sentence and the second sentence into a second artificial intelligence model, and obtaining evaluation results for the first sentence and the second sentence; includes

At this time, the first artificial intelligence model is a CNN-based artificial intelligence model learned by inputting a plurality of patent documents as learning data, and the second artificial intelligence model is a patent document, a prior research report matching the patent document, and the It may be an artificial intelligence model based on a Bi-LSTM model learned based on a similar patent document corresponding to a prior research report.

In this case, the acquiring of the similar patent document may include: acquiring a plurality of word sets included in the target patent document; obtaining a plurality of target patent clusters by clustering the obtained plurality of word sets; A plurality of midpoints for each of the plurality of target patent clusters are obtained, and the location of the target patent document is determined based on the plurality of midpoints for each of the plurality of target patent clusters and the number of word sets included in the target patent cluster. judging; obtaining a plurality of word sets included in the patent document, and clustering the obtained plurality of word sets to obtain a plurality of patent clusters; obtaining a plurality of midpoints for each of the plurality of patent clusters, and determining the position of the patent document based on the plurality of midpoints for each of the plurality of patent clusters and the number of word sets included in the patent cluster; and when the position of the target patent document and the position of the patent document are within a preset distance, determining the patent document as the similar patent document; may include.

In this case, the acquiring of the plurality of sentences may include: acquiring a plurality of word sets for each of the plurality of sentences included in the target patent document and the similar patent document; obtaining a plurality of importance scores for each of the obtained plurality of word sets; determining an importance score of a sentence included in the target patent document based on the plurality of importance scores; and arranging the plurality of sentences in order of importance based on the importance score of each of the plurality of sentences. may include.

In this case, the steps of obtaining a first sentence among a plurality of sentences included in the target patent document and obtaining a second sentence among a plurality of sentences included in the similar patent document include the plurality of sentences included in the target patent document. classifying into n preset groups based on the importance score; classifying the plurality of sentences included in the similar patent document into n preset groups based on the importance score; acquiring the second sentence as all sentences included in the similar patent document when the first sentence is included in a first group among n groups of the target patent document; When the first sentence is included in the kth group among the n groups of the target patent document, the second sentence is included in the first group to the (n-k+1)th group among the n groups of the similar patent document. Including; obtaining as a sentence included in , wherein n may be a natural number greater than or equal to 1, and k may be a natural number less than or equal to n.

In this case, the obtaining of the evaluation results for the first sentence and the second sentence may include: obtaining a similarity score and a dissimilarity score of the first sentence and the second sentence, respectively; determining that the first sentence and the second sentence are unrelated sentences when the dissimilarity score is equal to or greater than a preset score; obtaining at least one word set not included in the second sentence among the word sets included in the first sentence when the similarity score is equal to or greater than a preset score; obtaining at least an importance score for each of at least one word set not included in the second sentence, and determining whether a word set equal to or greater than a preset importance score exists among the at least one obtained importance score; determining the first sentence and the second sentence as matching sentences when there is no word set equal to or greater than a preset importance score; and determining that the first sentence and the second sentence are mismatched sentences when there is a set of words equal to or greater than the preset importance score. may include.

In this case, the obtaining of the word set may include: obtaining a plurality of word sets by analyzing morphemes of the target patent document; determining an erroneous word set among the obtained plurality of word sets; correcting the error word set; and obtaining at least one complex noun phrase set based on the degree of association between the plurality of word sets, wherein the determining of the erroneous word set includes semantic information for each of the obtained word sets. matching; obtaining an accuracy score for each of the plurality of word sets based on the matched semantic information and the patent document; and determining a word set whose accuracy score is less than or equal to a preset value as an erroneous word set, wherein the correcting of the erroneous word set includes: obtaining a plurality of semantic information for the erroneous word; obtaining a plurality of weights for each of the obtained pieces of semantic information; obtaining a weight having the highest degree of relevance to the patent document from among the plurality of weights; and matching semantic information corresponding to a weight having the highest degree of relevance with the patent document to the erroneous word set. The obtaining of the at least one compound noun phrase set includes: obtaining a plurality of word sets for sentences included in the patent document; obtaining a plurality of compound noun phrase set candidates obtained by combining the plurality of word sets; obtaining a frequency in which a compound noun phrase set identical to the obtained plurality of compound noun phrase set candidates is included in a patent document; and determining a compound noun phrase set candidate whose frequency is equal to or greater than a preset frequency as a compound noun phrase set. and, the compound noun phrase set candidate may include order information and spacing information of word sets included in the compound noun phrase set candidate.

In this case, the step of obtaining the importance score may include: obtaining, by the server, a word to be calculated for the importance score from the target patent document; The server includes the first detailed importance of the word in the entire patent document, the second detailed importance of the word in the patent classification information corresponding to the technical field information of the target patent document, and the word in the entire patent document calculating at least one detailed importance among third detailed importance levels of the searched patent document; and obtaining, by the server, the importance score of the word based on at least one of the first detailed importance, the second detailed importance, and the third detailed importance; may include.

Other specific details of the invention are included in the detailed description and drawings.

According to the above-described embodiment of the present invention, a prior art research method that can be guaranteed a stable result with a small time investment using an artificial intelligence model can be provided.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is an exemplary diagram for explaining a system according to an embodiment of the present invention.
2 is a flowchart illustrating a method of obtaining evaluation results for sentences included in a target patent document and a similar patent document according to an embodiment of the present invention.
3 is a flowchart illustrating a method of obtaining a similar patent document according to an embodiment of the present invention.
4 is a flowchart for explaining a method of obtaining a plurality of sentences from a target patent document and a similar patent document and arranging them in order of importance according to an embodiment of the present invention.
5 is a flowchart illustrating a method of obtaining a first sentence and a second sentence according to an embodiment of the present invention.
6 is a flowchart illustrating a method of obtaining evaluation results for a first sentence and a second sentence according to an embodiment of the present invention.
7 is an exemplary diagram for explaining a process of deriving an evaluation result using a first artificial intelligence model and a second artificial intelligence model according to an embodiment of the present invention.
8A to 8D are flowcharts illustrating a method of acquiring a word set according to an embodiment of the present invention.
9 is a flowchart illustrating a method of obtaining an importance score according to an embodiment of the present invention.
10 is a block diagram of an apparatus according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and those of ordinary skill in the art to which the present invention pertains. It is provided to fully understand the scope of the present invention to those skilled in the art, and the present invention is only defined by the scope of the claims.

The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, “comprises” and/or “comprising” does not exclude the presence or addition of one or more other components in addition to the stated components. Like reference numerals refer to like elements throughout, and "and/or" includes each and every combination of one or more of the recited elements. Although "first", "second", etc. are used to describe various elements, these elements are not limited by these terms, of course. These terms are only used to distinguish one component from another. Accordingly, it goes without saying that the first component mentioned below may be the second component within the spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used herein will have the meaning commonly understood by those of ordinary skill in the art to which this invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless specifically defined explicitly.

As used herein, the term “unit” or “module” refers to a hardware component such as software, FPGA, or ASIC, and “unit” or “module” performs certain roles. However, “part” or “module” is not meant to be limited to software or hardware. A “unit” or “module” may be configured to reside on an addressable storage medium or to reproduce one or more processors. Thus, by way of example, “part” or “module” refers to components such as software components, object-oriented software components, class components and task components, processes, functions, properties, Includes procedures, subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Components and functionality provided within “parts” or “modules” may be combined into a smaller number of components and “parts” or “modules” or as additional components and “parts” or “modules”. can be further separated.

Spatially relative terms "below", "beneath", "lower", "above", "upper", etc. It can be used to easily describe the correlation between a component and other components. A spatially relative term should be understood as a term that includes different directions of components during use or operation in addition to the directions shown in the drawings. For example, when a component shown in the drawing is turned over, a component described as “beneath” or “beneath” of another component may be placed “above” of the other component. can Accordingly, the exemplary term “below” may include both directions below and above. Components may also be oriented in other orientations, and thus spatially relative terms may be interpreted according to orientation.

In this specification, a computer means all types of hardware devices including at least one processor, and may be understood as encompassing software configurations operating in the corresponding hardware device according to embodiments. For example, a computer may be understood to include, but is not limited to, smart phones, tablet PCs, desktops, notebooks, and user clients and applications running on each device.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Each step described in this specification is described as being performed by a computer, but the subject of each step is not limited thereto, and at least a portion of each step may be performed in different devices according to embodiments.

1 is an exemplary diagram for explaining a system according to an embodiment of the present invention.

The system for determining the similarity of patent documents according to the present invention includes a server 10 and an electronic device 20 .

The server 10 is configured to obtain a target patent document, obtain a similar patent document from the acquired target patent document, and determine the similarity between a sentence of the target patent document and a sentence of the similar patent document.

Specifically, the server 10 may receive a target patent document from the electronic device 20 or may acquire a target patent document from an external server.

In the present specification, a patent document is a concept including a target patent document and a similar patent document, and may be a document about technical contents submitted by an applicant to obtain a patent registration at each country's patent office. However, the present invention is not limited thereto, and the patent document may be understood as a concept including various documents including technical contents, such as a job invention for a patent application, and a thesis. According to an embodiment, the target patent document may be at least one of a job invention book and a thesis for a patent application, and the similar patent document may be at least one of a job invention book, a thesis, and a patent application for a patent application.

The electronic device 20 is configured to provide a patent document to the server 10 . The electronic device 200 according to the present invention may be implemented as a smart phone, but this is only an embodiment, and a smart phone, a tablet personal computer (PC), a mobile phone, a video phone, e-book reader, desktop PC, laptop PC, netbook computer, workstation, server, PDA (personal digital assistant), PMP (portable multimedia) player) or a wearable device.

2 is a flowchart illustrating a method of obtaining evaluation results for sentences included in a target patent document and a similar patent document according to an embodiment of the present invention.

In step S110, the server 10 may obtain a similar patent document by inputting the target patent document into the first artificial intelligence model.

Specifically, the server 10 may obtain a similar patent document similar to the target patent document from among a plurality of patent documents stored in the first database by using the first artificial intelligence model.

In one embodiment, the server 10 analyzes each of the plurality of patent documents based on the morpheme analysis of each of the plurality of patent documents stored in the first database, and based on the morpheme analysis of the target patent document, the target patent After analyzing the document, it is possible to obtain a similar patent document with a high degree of relevance to the target patent document. A specific method of obtaining a similar patent document will be described later with reference to FIG. 3 .

In step S120 , the server 10 may acquire a plurality of sentences included in the target patent document, and acquire a plurality of sentences included in the similar patent document.

In step S130 , the server 10 may acquire a first sentence among a plurality of sentences included in the target patent document, and acquire a second sentence among a plurality of sentences included in the similar patent document.

In step S140 , the server 10 may input the first sentence and the second sentence into a second artificial intelligence model to obtain an evaluation result for the first sentence and the second sentence.

On the other hand, the first artificial intelligence model is an artificial intelligence model based on a convolutional deep neural network (CNN) that is learned by inputting a plurality of patent documents as learning data, and the second artificial intelligence model is a patent document, the patent document It may be an artificial intelligence model based on a Bi-LSTM model learned based on a prior research report matched to , and a similar patent document corresponding to the prior research report. However, the present invention is not limited thereto, and it goes without saying that various artificial intelligence models may be applied to the present invention. For example, a model such as a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), or a Bidirectional Recurrent Deep Neural Network (BRDNN) may be used as the AI model, but is not limited thereto.

In this case, convolutional deep neural networks (CNNs) are a type of multilayer perceptrons designed to use minimal preprocessing. A convolutional neural network consists of one or several convolutional layers and general artificial neural network layers on top of it, and additionally utilizes weights and pooling layers. Thanks to this structure, the convolutional neural network can fully utilize the input data of the two-dimensional structure. In addition, convolutional neural networks can be trained via standard backpropagation. Convolutional neural networks are easier to train than other feedforward neural network techniques and have the advantage of using fewer parameters. A convolutional neural network extracts features from an input image by alternately performing convolution and subsampling on the input image.

A convolutional neural network includes several convolution layers, several subsampling layers (Subsampling layer, Lacal pooling layer, Max-Pooling layer), and a fully connected layer. The convolution layer is a layer that performs convolution on an input image. In addition, the subsampling layer is a layer that locally extracts the maximum value from the input image and maps it to a 2D image, and subsampling is performed by enlarging the local area.

In the convolution layer, information such as the kernel size, the number of kernels to be used (ie, the number of maps to be generated), and a weight table to be applied during the convolution operation are required. For example, suppose that the size of the input image is 32x32, the size of the kernel is 5x5, and the number of kernels to be used is 20. In this case, if a 5×5 kernel is applied to a 32×32 input image, it is impossible to apply the kernel to two pixels above, below, left, and right of the input image. If the kernel is placed on the input image and then convolution is performed, the result value of '-8' is determined as the value of the pixel corresponding to the center element of the kernel among the pixels of the input image included in the kernel. am. Therefore, when convolution is performed by applying a 5×5 kernel to an input image having a size of 32×32, a map of size 28×28 is generated. Previously, since it was assumed that the total number of kernels to be used is 20, in the first convolutional layer, a total of 20 maps with a size of 28×28 are generated.

In the subsampling layer, information on the size of a kernel to be subsampled and information on whether to select a maximum value or a minimum value among values in the kernel region are required.

In addition, deep neural networks (DNNs) are artificial neural networks (ANNs) composed of a plurality of hidden layers between an input layer and an output layer.

In this case, the structure of the deep neural network may be composed of a perceptron. A perceptron consists of several inputs, one processor, and one output value. The processor multiplies each input value by a weight, and then sums the input values multiplied by the weight. Then, the processor outputs one output value by substituting the summed value into the activation function. If a specific value is desired as the output value of the activation function, the weight multiplied by each input value may be corrected, and the output value may be recalculated using the modified weight. In this case, each perceptron may use a different activation function. In addition, each perceptron receives the outputs from the previous layer as input, and then uses the activation function to obtain the output. The obtained output is transferred to the input of the next layer. Through the process as described above, some output values can be finally obtained.

In addition, returning to the description of the deep learning technique, a Reccurent Neural Network (RNN) refers to a neural network in which connections between units constituting an artificial neural network constitute a directed cycle. Unlike forward neural networks, recurrent neural networks can utilize the memory inside the neural network to process arbitrary inputs.

Deep Belief Networks (DBN) is a generative graphical model used in machine learning, and in deep learning, it means a deep neural network composed of multiple layers of latent variables. There is a connection between layers, but there is no connection between units within a layer.

The deep trust neural network can be used for pre-learning due to the nature of the generative model, and after learning the initial weights through pre-learning, the weights can be fine-tuned through backpropagation or other discrimination algorithms. This feature is very useful when the training data is small, because the smaller the training data, the stronger the effect of the initial value of the weight on the resulting model. The pre-learned initial weight value is closer to the optimal weight compared to the arbitrarily set initial weight value, which enables performance and speed improvement in the fine-tuning stage.

The above-described artificial intelligence and its learning method have been described for illustrative purposes, and the artificial intelligence and its learning method used in the embodiments described below are not limited. For example, all kinds of artificial intelligence technology and a learning method thereof that a person skilled in the art can apply to solve the same problem may be utilized to implement the system according to the disclosed embodiment.

3 is a flowchart illustrating a method of obtaining a similar patent document according to an embodiment of the present invention.

In step S210, the server 10 may acquire a plurality of word sets included in the target patent document.

In step S220, the server 10 may obtain a plurality of target patent clusters by clustering the plurality of obtained word sets.

In this case, various clustering techniques may be applied to clustering according to the present invention. In an embodiment, the server 10 may acquire a plurality of target patent clusters using an Elasticsearch clustering clustering technique. In another embodiment, the server 10 uses at least one clustering technique of K-means clustering, DBSCAN (Density-based spatial clustering of applications with noise) clustering, hierarchical clustering, and mixed Gaussian clustering technique to a plurality of targets A patent cluster can be obtained.

In step S230, the server 10 obtains a plurality of midpoints for each of the plurality of target patent clusters, and the plurality of midpoints for each of the plurality of target patent clusters and the number of word sets included in the target patent clusters It is possible to determine the location of the target patent document based on the .

In one embodiment, the center of the target patent cluster may be obtained based on the positions of a plurality of word sets included in the cluster, and the location of the target patent document is the center of gravity with respect to the midpoints of the plurality of target patent clusters. can

In step S240, the server 10 may obtain a plurality of word sets included in the patent document, and cluster the obtained plurality of word sets to obtain a plurality of patent clusters.

In step S250, the server 10 obtains a plurality of midpoints for each of the plurality of patent clusters, based on the plurality of midpoints for each of the plurality of patent clusters and the number of word sets included in the patent clusters The position of the patent document may be determined.

As an embodiment, the midpoint of the patent cluster may be obtained based on the positions of a plurality of word sets included in the corresponding cluster, and the position of the patent document may be the center of gravity with respect to the midpoints of the plurality of patent clusters.

In step S260, when the position of the target patent document and the position of the patent document are within a preset distance, the server 10 may determine the patent document as the similar patent document.

In one embodiment, the server 10 may determine a plurality of patent documents existing within a preset distance from the location of the target patent document as similar patent documents, and the plurality of similar patent documents having a close distance to the target patent document are determined in the order of the plurality of similar patent documents. You can sort similar patent documents of

4 is a flowchart for explaining a method of obtaining a plurality of sentences from a target patent document and a similar patent document and arranging them in order of importance according to an embodiment of the present invention.

In step S310, the server 10 may obtain a plurality of word sets for each of a plurality of sentences included in the target patent document and the similar patent document.

In step S320 , the server 10 may obtain a plurality of importance scores for each of the obtained plurality of word sets. A method of obtaining the importance score will be described later.

In step S330, the server 10 may determine the importance score of the sentence included in the target patent document based on the plurality of importance scores.

In one embodiment, the server 10 obtains the degree of association between the plurality of word sets and the target patent document, and determines the sum of the importance scores of the word sets having the obtained association degree equal to or greater than a preset value as the importance score of the sentence. can In this case, the degree of association may be obtained through various methods. For example, the degree of association between the word set and the target patent document may be obtained based on a plurality of patent documents corresponding to the IPC field corresponding to the target patent document. Specifically, the degree of association between the word set and the target patent document may be obtained based on the number of times the corresponding word set appears in a plurality of patent documents corresponding to the IPC field. As an embodiment, the higher the number of times the corresponding word set appears in a plurality of patent documents corresponding to the IPC field, the higher the correlation between the word set and the target patent document.

In step S340 , the server 10 may sort the plurality of sentences in order of importance based on the importance score of each of the plurality of sentences.

Furthermore, the server 10 may group a plurality of sentences into n groups. In this case, a method of grouping a plurality of sentences may be various. In an embodiment, the n groups may be grouped to include the same number of sentences. As another embodiment, the n groups may be grouped according to a preset importance score. For convenience of description, it is hereinafter assumed that the importance score of the sentences included in the kth group is greater than the importance score of the sentences included in the k+1th group.

5 is a flowchart illustrating a method of obtaining a first sentence and a second sentence according to an embodiment of the present invention.

In step S410, the server 10 may classify the plurality of sentences included in the target patent document into n preset groups based on the importance score.

In step S420, the server 10 may classify the plurality of sentences included in the similar patent document into n preset groups based on the importance score.

In step S430, when the first sentence is included in the first group among n groups of the target patent document, the server 10 may obtain the second sentence as the entire sentence included in the similar patent document. have.

In step S440, when the first sentence is included in the kth group among n groups of the target patent document, the server 10 converts the second sentence to a first group among the n groups of the similar patent document. It can be obtained from sentences included in the (n-k+1)th group. In this case, n may be a natural number greater than or equal to 1, and k may be a natural number less than or equal to n.

For example, it may be assumed that the target patent document and the similar patent document are classified into three groups. At this time, when the first sentence is included in the first group, the server 10 compares the first sentence with all the sentences included in the first group to the third group (that is, the entire similar patent document) of the similar patent document. evaluation results can be obtained. Also, when the first sentence is included in the second group, the server 10 may obtain an evaluation result by comparing the first sentence with all sentences included in the first group and the second group of the similar patent document. Also, when the first sentence is included in the third group, the server 10 may obtain an evaluation result by comparing the first sentence with all sentences included in the first group of similar patent documents.

That is, the server 10 obtains evaluation results only for sentences determined to be important sentences according to the importance of the sentences, thereby preventing an evaluation result acquisition process for unnecessary or less important sentences, thereby reducing the amount of calculation. Specifically, since the first group of the target patent document is a sentence with high importance, it is compared with the entire similar patent document, and since the third group of the target patent document is a sentence with low importance, it is compared with the sentences of the first group with high importance among similar patent documents By comparing, the server 10 can omit the comparison between the third group of (less important) target patent documents and the third group of (less important) similar patent documents.

Meanwhile, according to various embodiments of the present invention, the server 10 may determine whether to group the target patent document and the similar patent document into several groups. Specifically, the server 10 may determine the value of n based on the current server 10's GPU occupancy and required accuracy. When the GPU occupancy is high or the required accuracy is low, the server 10 may increase the value of n, and when the GPU occupancy is low or the required accuracy is high, the server 10 may decrease the value of n. For example, it may be assumed that the target patent document and the similar patent document are composed of 6 sentences, and n is 3 and n is 6. When n is 6, the server 10 compares 6+5+4+3+2+1=21 sentences, and when n is 3, the server 10 compares 6+6+4+4+2+ 2 = 24 sentences will be compared. That is, as the value of n increases, the number of sentences to be compared in the target patent document and the similar patent document decreases, and as the value of n decreases, the number of sentences to be compared in the target patent document and the similar patent document increases. ) If the amount of computation (GPU occupancy) that can be currently processed is small or the required accuracy is low, the server 10 may increase the value of n.

Meanwhile, in the above-described embodiment, the case in which the target patent document and the similar patent document are grouped in the same number of n has been described, but the present invention is not limited thereto. For example, it goes without saying that the target patent document and the similar patent document may be grouped in different numbers.

6 is a flowchart illustrating a method of obtaining evaluation results for a first sentence and a second sentence according to an embodiment of the present invention.

In step S510, the server 10 may obtain a similarity score and a dissimilarity score of the first sentence and the second sentence, respectively.

Specifically, the similarity score is an index for determining whether the first sentence and the second sentence are related sentences, and the dissimilarity score is an indicator for determining whether the first sentence and the second sentence are unrelated sentences.

In an embodiment, the similarity score and the dissimilarity score may be obtained by acquiring semantic information of a word set included in the first sentence and the second sentence, and determining whether the acquired semantic information is semantic information about the same subject. can

In another embodiment, the server 10 may determine the sentence structure of the first sentence and the second sentence and use it to calculate the similarity score and the dissimilarity score. For example, the server 10 may increase the similarity score when the sentence structures of the first sentence and the second sentence are similar, and increase the dissimilarity score when the sentence structures of the first sentence and the second sentence are dissimilar. .

In step S520 , when the dissimilarity score is equal to or greater than a preset score, the server 10 may determine that the first sentence and the second sentence are unrelated sentences.

In step S530 , when the similarity score is equal to or greater than a preset score, the server 10 may acquire at least one word set not included in the second sentence among the word sets included in the first sentence.

That is, even if the first sentence and the second sentence are related sentences, the first sentence and the second sentence are not sentences having the same meaning. Accordingly, the server 10 may determine whether the first sentence and the second sentence are the same sentence or whether a sentence on the same subject or a sentence with a difference exists.

In an embodiment, the server 10 may determine at least one word set not included in the second sentence among the word sets included in the first sentence in order to determine the difference between the first sentence and the second sentence.

In another embodiment, the server 10 may determine the sentence structure of the first sentence and the second sentence.

In step S540, the server 10 obtains at least an importance score for each of the at least one word set not included in the second sentence, and a word set equal to or greater than a preset importance score among the obtained at least one importance score is selected. It can be determined whether it exists or not.

In step S550 , the server 10 may determine the first sentence and the second sentence as matching sentences when there is no word set equal to or greater than the preset importance score.

In step S560 , the server 10 may determine the first sentence and the second sentence as mismatched sentences when there is a word set equal to or greater than the preset importance score.

That is, even when at least one word set of the first sentence is not included in the second sentence, there may exist a case where a word set having a low importance score is not related to the similarity between the first sentence and the second sentence. . Accordingly, the server 10 acquires a word set when the importance score of at least one word set not included in the second sentence is equal to or greater than a preset importance score, and based on the acquired word set, the first sentence and the second sentence It may be determined whether the sentence is a matching sentence or a non-matching sentence.

7 is an exemplary diagram for explaining a process of deriving an evaluation result using a first artificial intelligence model and a second artificial intelligence model according to an embodiment of the present invention.

As shown in FIG. 7 , the first artificial intelligence model may receive a target patent document as an input value and obtain a similar patent document for the target patent document. In this case, there may be a plurality of similar patent documents for the target patent document, and if there are a plurality of similar patent documents, the server 10 may sort the plurality of similar patent documents based on the association with the target patent document. of course there is

The first database is configured to store the entire patent document. Specifically, the first database obtains a word set of each of the entire patent documents through morphological analysis of each of the entire patent documents, obtains a thesaurus for the obtained word set, and an importance score for each of the obtained word sets may be obtained, and the obtained importance score may be matched to a word set and stored.

The first artificial intelligence model may acquire a similar patent document corresponding to the target patent document among all patent documents stored in the first database.

Meanwhile, when the target patent document and the similar patent document are obtained through the first artificial intelligence model, the server 10 may extract a first sentence of the target patent document and extract a second sentence of the similar patent document. The first sentence and the second sentence may be obtained through the method of FIGS. 4 and 5 described above.

Furthermore, the second artificial intelligence model may obtain an evaluation result for the first sentence and the second sentence by inputting the first sentence and the second sentence as input values. In this case, the evaluation result may be a result of whether the first sentence and the second sentence are a matching sentence, a discordant sentence, or an unrelated sentence, and may be obtained according to the above-described embodiment of FIG. 6 .

The second database may store a patent document and a prior search report matched to the patent document. In this case, the prior search report may include information on a similar patent document determined to be similar to a patent document and a specific sentence of a similar patent document corresponding to a specific sentence of the patent document.

8A to 8D are flowcharts illustrating a method of acquiring a word set according to an embodiment of the present invention.

The server 10 may acquire a target patent document. In an embodiment, the server 10 may obtain a job invention from the electronic device 20 as a target patent document.

In step S605, the server 10 may obtain a word set by analyzing the morpheme of the target patent document.

In an embodiment, the server 10 may perform morpheme analysis of a patent document using a Mecab morpheme analyzer. However, the present invention is not limited thereto, and in some cases, various morpheme analyzers such as Okt, Komoran, Hannanum, and Kkma morpheme analyzers may be used. Furthermore, it goes without saying that the server 10 may use various morpheme analyzers according to the language of the patent document to be analyzed.

In step S610 , the server 10 may determine an erroneous word set among the plurality of obtained word sets.

In an embodiment, when the plurality of acquired word sets do not satisfy a preset condition, the server 10 may determine the corresponding word as an erroneous word. In step S615, the server 10 may correct the erroneous word set.

In operation S620 , the server 10 may acquire at least one set of compound noun phrases based on the degree of association between the plurality of word sets.

In one embodiment, the server 10 obtains a sentence included in the target patent document, classifies the text included in the target patent document into sentence units, performs morphological analysis on each classified sentence, and converts the sentence into syllables. It is possible to divide into units, match the part-of-speech tag to the divided syllables, and obtain morphemes included in the sentence based on the divided syllables.

In operation S625 , the server 10 may match semantic information for each of the obtained plurality of word sets.

In this case, the semantic information may mean an intent for a word set. The server 10 may perform natural language processing using an artificial intelligence model to obtain semantic information about a word set.

Specifically, the artificial intelligence model includes a natural language understanding unit, and the natural language understanding unit can grasp an entity and an intent of a word set included in a sentence based on the sentence analysis result. It is possible to interpret a sentence through structure and main component analysis, and to perform sentence analysis using statistics/analysis, etc.

In an embodiment, the server 10 may obtain semantic information about the apple by analyzing a sentence including 'apple'. For example, it may be assumed that the word set obtained through sentence passage is “apple.” In this case, the semantic information about the apple may be semantic information indicating one kind of fruit as a noun, but other people as a verb. The server 10 may match semantic information determined to be suitable for a sentence among a plurality of semantic information on 'apple' with 'apple'.

In step S630 , the server 10 may obtain an accuracy score for each of the plurality of word sets based on the matched semantic information and the patent document.

In this case, although semantic information about the word set acquired through one sentence is acquired in step S625, the semantic information acquired in step S625 may be inaccurate. Accordingly, the server 10 may acquire the accuracy score of the semantic information matched to the word set based on the entire patent document including the sentence.

Accordingly, the server 10 may obtain an accuracy score for the semantic information of the word set obtained through sentence analysis based on the semantic information about the same word set found in the entire patent document.

For example, the semantic information about 'apple' obtained by analyzing a sentence containing 'apple' is information about an apple representing a fruit, but the semantic information of 'apple' searched in the entire patent document including the sentence is different. In the case of information about telling a person wrong, the server 10 may set the pre-matched semantic information low.

In operation S635, the server 10 may determine a word set having an accuracy score equal to or less than a preset value as an error word set.

That is, when the accuracy score is less than or equal to a preset value, the server 10 may determine that the semantic information matched with the corresponding word set is erroneously matched, and obtain the word set as the erroneous word set.

In step S640, the server 10 may obtain a plurality of semantic information for the erroneous word. Specifically, the server 10 may acquire a plurality of semantic information of the erroneous word from a plurality of patent documents.

In step S645, the server 10 may obtain a plurality of weights for each of the plurality of acquired semantic information.

In an embodiment, the server 10 may obtain a weight based on a patent document including an error word. Specifically, the server 10 may acquire at least one sentence including an erroneous word from the patent document, and acquire semantic information on the same word as the erroneous word included in the at least one acquired sentence. The server 10 may acquire weights for a plurality of semantic information based on semantic information on at least one word identical to the erroneous word.

In step S650, the server 10 may obtain a weight having the highest degree of association with the patent document among the plurality of weights.

In step S655 , the server 10 may match semantic information corresponding to a weight having the highest degree of relevance with the patent document to the erroneous word set. That is, the server 10 may determine semantic information having the greatest weight among a plurality of semantic information on the erroneous word as semantic information on the erroneous word and correct the erroneous word.

Meanwhile, according to various embodiments of the present disclosure, it goes without saying that the erroneous word set may be a case in which incorrect semantic information is matched to the word set, as well as a word set that is erroneously parsed due to an error in the morpheme analysis process. In an embodiment, when the sentence for obtaining the word set is "Natural language processing using machine learning is performed", the server 10 may obtain machine, learning, use, natural language, and processing as a word set.

In this case, due to an error in morpheme analysis, a case of acquiring a word set such as machine, learning, using eul, natural language, and processing may occur. In this case, the server 10 may determine 'use E' as an error word set and correct it.

Specifically, after acquiring semantic information for each of the acquired word sets, the server 10 may determine whether the word is an erroneous word based on the degree of association between the semantic information and the patent document. According to an exemplary embodiment, when the acquired word set is found in a patent document including a word set having a frequency greater than or equal to a preset frequency, the server 10 may determine that the corresponding word set is an error-free word set. In another embodiment, when the word set is found in the entire patent document at a frequency greater than or equal to a preset frequency, the server 10 may determine that the word set is not an erroneous word set by determining that the word set is a commonly used word set. In another embodiment, when the server 10 does not find semantic information about the word set, the server 10 may determine the corresponding word set as an erroneous word set. In another embodiment, when semantic information matched to a word set is different from a patent document including a word set, the server 10 may determine the corresponding word set as an erroneous word set.

Specifically, the server 10 may obtain a set of words included in the entire patent document, match and store semantic information for each of the obtained word sets. The server 10 may obtain a cluster of a plurality of word sets having relevant semantic information by clustering the matched word set and semantic information. The server 10 may acquire a word set cluster in which the number of word sets included in the cluster is less than or equal to a preset number from among the plurality of word set clusters. In this case, the preset number does not mean the number of a plurality of identical word sets used in the patent document, but may mean the number of a plurality of word sets having different shapes. For example, even if the word "using E" is searched for multiple times in the patent document, since "using E" is the same set of words, it can be determined as one word set. The server 10 may determine the word set included in the cluster obtained in the above step as the erroneous word set.

However, the present invention is not limited thereto, and it goes without saying that the server 10 may determine, among the plurality of word set clusters, a word set included in a cluster having a distance from another cluster equal to or greater than a preset distance as the erroneous word set. Specifically, the server 10 may obtain the center points of a plurality of clusters, and determine the distance between the clusters based on the obtained center points. For example, the obtained word set cluster may be a first word set cluster to a fourth word set cluster. In this case, the server 10 determines the distance between the first word set cluster and the second word set cluster, the distance between the first word set cluster and the third word set cluster, the first word set cluster and the fourth word set cluster, and If distances of are obtained, and all three obtained distances are equal to or greater than a preset distance, the first word set cluster may be determined as an erroneous word set.

In this case, the compound noun phrase set candidate may include order information and spacing information of word sets included in the compound noun phrase set candidate.

In an embodiment, the compound noun phrase set is determined as a combination of two or more adjacent word sets, but in the present invention, a combination of a word set in which words are spaced apart may also be obtained as a compound noun phrase set. For example, when the server 10 obtains a set of compound noun phrases from a sentence including "a first device among a plurality of devices, a second device among a plurality of devices, and a third device among a plurality of devices", a plurality of The first device of the device, the second device of the plurality of devices, and the third device of the plurality of devices may be obtained as independent compound noun phrase sets, but "(separate word 1) device of multiple devices" can be obtained as one compound noun phrase set. Of course, it can also be obtained with In this case, the compound noun phrase set may include spacing information. In the above-described embodiment, the separation information may be information that one word is included between the word “middle” and the word “device”. Of course, according to various embodiments, a plurality of words may be included between words.

개이고, 단어 세트간의 순서가 변경되는 경우 획득되는 복합 명사구 세트의 수는

개이다. 따라서, 문장이 길어지면 길어질수록, 서버(10)는 복합 명사구 세트 검색을 위해 과도한 리소스를 투입하여야 한다. 따라서, 서버(10)는 복합 명사구 세트를 구성하는 단어 세트의 최대 개수를 결정하고, 결정된 개수 내의 복합 명사구 세트를 획득할 수 있다. 이때, 서버(10)는, 단어 세트 획득을 수행하는 특허문서와 동일한 기술 분야인 복수개의 특허문서를 획득하고, 획득된 특허문서에 포함된 복합 명사구 세트에 포함된 단어 세트의 최대 개수를 복합 명사구 세트를 구성하는 단어 세트의 최대 개수로 결정할 수 있다. In another embodiment, the server 10 may determine the maximum number of word sets constituting the compound noun phrase set, and obtain the compound noun phrase set within the determined number. Specifically, when the number of words included in the sentence for obtaining the compound noun phrase set is n and the order between the word sets is not changed, the number of compound noun phrase sets obtained is

, and the number of sets of compound noun phrases obtained when the order between sets of words is changed is

it's a dog Therefore, as the sentence becomes longer, the server 10 must invest excessive resources for searching the compound noun phrase set. Accordingly, the server 10 may determine the maximum number of word sets constituting the compound noun phrase set, and obtain the compound noun phrase set within the determined number. At this time, the server 10 acquires a plurality of patent documents that are the same technical field as the patent document for obtaining the word set, and sets the maximum number of word sets included in the compound noun phrase set included in the acquired patent document to the compound noun phrase. It can be determined as the maximum number of word sets constituting the set.

9 is a flowchart illustrating a method of obtaining an importance score according to an embodiment of the present invention.

In step S710 , the server 10 may obtain a word set that is a target for calculating the importance score from the target patent document. Needless to say, the importance score of the word set included in the target patent document as well as the similar patent document may also be generated by the method described below.

In step S720, the server 10 controls the first detailed importance of the word set in the entire patent document, the second detailed importance of the word set in the patent classification information corresponding to the technical field information of the target patent document, and the entire patent document. At least one detailed importance among the third detailed importance of the search patent document including the word set may be calculated.

In operation S730 , the server 10 may obtain an importance score of the word set based on one or more of the first detailed importance, the second detailed importance, and the third detailed importance.

In an embodiment, the server 10 may calculate the first detailed importance based on the first appearance rate and the second appearance rate. In this case, the first appearance ratio means the number of appearances of the word set for obtaining the importance score in the entire patent document compared to the total number of word sets in the entire patent document, and the second appearance ratio is the number of appearances of the word set among the sentences of the entire patent document. The number of appearing sentences may be counted, and it may mean the total number of sentences in the entire patent document compared to the number of appearing sentences in which a word set appears among the sentences of the entire patent document.

Specifically, the server 10 may calculate the first detailed importance level using Equation 1 below.

Here, W1 is the first detailed importance, wpw is the number of occurrences of the word set in the entire patent document, WPW is the total number of word sets in the entire patent document, and wps is the occurrence of the word set in the entire patent document. is the number of sentences, WPS is the total number of sentences in the entire patent document, and a1 is the control constant of the second appearance ratio.

In another embodiment, the server 10 may calculate the first detailed importance level based on the first appearance rate and the second appearance rate. In this case, the third appearance ratio means the number of appearances of the word set for obtaining the importance score in the patent classification information compared to the total number of word sets in the patent classification information, and the fourth appearance ratio is the number of occurrences of the word set among the sentences of the entire patent document. It may mean the total number of sentences of the entire patent document compared to the number of appeared sentences.

Here, the patent classification information is a code capable of classifying patents according to technical fields, and may be any one of International Patent Classification (IPC), Cooperative Patent Classification (CPC), and F-Term.

Specifically, the server 10 may calculate the second detailed importance level by using Equation 2 below.

Here, W ₂ is the second specific importance, ipcw is the number of occurrences of the word set in the patent classification information, IPCW is the total number of word sets in the patent classification information, and ipcs is the number of occurrences of the word set in the entire patent document sentence. The number of occurrences is the number of sentences, IPCS is the total number of sentences in the entire patent document, and a2 is the control constant of the fourth occurrence ratio.

In another embodiment, the server 10 searches for a search patent document including a word set to be calculated of an importance score from among all patent documents, and searches for each of the searched patent documents based on the reference information of each of the searched patent documents. An influence value may be calculated, and a third detailed importance level may be obtained based on the calculated influence value.

Specifically, the server 10 calculates an influence value based on the number of items that are the same as at least one of the applicant, inventor, and right holder of the reference information of the search patent document, the number of items that are the same as other patent documents, the number of citations and the number of citations that are reference information. can

That is, the server 10 may calculate the degree to which the searched patent document is related to another patent document as an influence value. For example, when the searched patent document is cited from several patent documents, the server 10 may determine that the searched patent document has an influence on the corresponding patent document and calculates it as an influence value of the searched patent document.

Furthermore, the server 10 may calculate the average of the calculated influence values for each of the search patent documents, and calculate the calculated average as the third detailed importance level.

10 is a block diagram of an apparatus according to an embodiment.

The processor 102 may include one or more cores (not shown) and a graphic processing unit (not shown) and/or a connection path (eg, a bus, etc.) for transmitting and receiving signals to and from other components. .

The processor 102 according to one embodiment performs the method described with reference to FIGS. 1 to 9 by executing one or more instructions stored in the memory 104 .

For example, the processor 102 obtains new training data by executing one or more instructions stored in the memory, performs a test on the acquired new training data by using the learned model, and performs the test result, labeling Extracting the first learning data in which the obtained information is obtained with an accuracy greater than or equal to a predetermined first reference value, deleting the extracted first learning data from the new learning data, and removing the new learning data from which the extracted learning data is deleted It is possible to retrain the learned model using

On the other hand, the processor 102 is a RAM (Random Access Memory, not shown) and ROM (Read-Only Memory: ROM) for temporarily and / or permanently storing a signal (or data) processed inside the processor 102. , not shown) may be further included. In addition, the processor 102 may be implemented in the form of a system on chip (SoC) including at least one of a graphic processing unit, a RAM, and a ROM.

The memory 104 may store programs (one or more instructions) for processing and controlling the processor 102 . Programs stored in the memory 104 may be divided into a plurality of modules according to functions.

The steps of a method or algorithm described in relation to an embodiment of the present invention may be implemented directly in hardware, as a software module executed by hardware, or by a combination thereof. A software module may contain random access memory (RAM), read only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, hard disk, removable disk, CD-ROM, or It may reside in any type of computer-readable recording medium well known in the art to which the present invention pertains.

The components of the present invention may be implemented as a program (or application) to be executed in combination with a computer, which is hardware, and stored in a medium. The components of the present invention may be implemented as software programming or software components, and similarly, embodiments may include various algorithms implemented as data structures, processes, routines, or combinations of other programming constructs, including C, C++ , Java, assembler, etc. may be implemented in a programming or scripting language. Functional aspects may be implemented in an algorithm running on one or more processors.

As mentioned above, although embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art to which the present invention pertains can realize that the present invention can be embodied in other specific forms without changing its technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

10 : Server
20: electronic device

Claims (1)

In the method for determining the degree of similarity of patent documents,
obtaining, by the server, a similar patent document by inputting the target patent document into the first artificial intelligence model;
obtaining, by the server, a plurality of sentences included in the target patent document, and acquiring a plurality of sentences included in the similar patent document;
obtaining, by the server, a first sentence among a plurality of sentences included in the target patent document, and acquiring a second sentence among a plurality of sentences included in the similar patent document;
inputting, by the server, the first sentence and the second sentence into a second artificial intelligence model, and obtaining evaluation results for the first sentence and the second sentence; including,
Acquiring the evaluation results for the first sentence and the second sentence comprises:
obtaining a similarity score and a dissimilarity score of the first sentence and the second sentence, respectively;
determining that the first sentence and the second sentence are unrelated sentences when the dissimilarity score is equal to or greater than a preset score;
obtaining at least one word set not included in the second sentence among the word sets included in the first sentence when the similarity score is equal to or greater than a preset score;
obtaining at least an importance score for each of at least one word set not included in the second sentence, and determining whether a word set equal to or greater than a preset importance score exists among the at least one obtained importance score;
determining the first sentence and the second sentence as matching sentences when there is no word set equal to or greater than a preset importance score; and
determining the first sentence and the second sentence as mismatched sentences when there is a set of words equal to or greater than the preset importance score; A method for determining the degree of similarity of a patent document comprising a.

Images