JP7348746B2 - Research support method, research support computer program, and research support system - Google Patents

Description

The present invention relates to a research support method, a research support computer program, and a research support system.

During patent examinations and patent invalidation searches, prior art documents are searched to determine the presence or absence of patentability, such as novelty and inventive step. When searching for prior art documents, a patent document database is created by combining keywords corresponding to the claims to be searched for patent examination and reasons for patent invalidation, and patent classifications such as IPC (International Patent Classification) and FI (File Index). , and also searches non-patent literature databases. When searching, the searcher narrows down the number of documents by selecting and combining patent classifications and keywords from the viewpoint of the researcher in a large amount of patent documents, and then confirms the content of the documents once a predetermined number is reached. conduct. Then, based on the prior art documents discovered by checking the content, the cited invention is identified, and the claimed invention and one cited invention are compared to identify points of agreement and differences. Next, if a prior art search is conducted again regarding the difference and other cited documents corresponding to the structure related to the difference are found, it can be said that it is easy to apply the other cited invention to the first cited invention. Consider whether you can reason with it. This review is repeated until the patent examination and patent invalidation investigation are completed.

On the other hand, neural language understanding models have been rapidly developing in recent years.
There are two types of traditional language understanding models: a rule-based language understanding model and a statistical language understanding model.In the rule-based language understanding model, a person with deep knowledge of each domain can determine synonyms. They not only had to think about the types of subtasks necessary for language understanding, such as syntactic analysis and syntactic analysis, and design the necessary features for each subtask, but also had to come up with rules and algorithms for language understanding themselves. Therefore, it has been difficult to construct a large-scale practical system. In addition, in statistical language understanding models, humans consider the types of subtasks and design the features, but rules for language understanding are automatically constructed from training data using statistical machine learning techniques. can. However, since training data specialized for each subtask must be prepared separately, it is extremely difficult to create a model with high accuracy in practice.

In particular, when dealing with explanatory texts related to technology, things pointing to the same technical concept may be described from different aspects, or concepts that can be expressed with one word may be difficult to understand when a new technology has just been developed. When considering the semantic comparison between two sentences regarding technology-related explanatory sentences, we use traditional language understanding models to identify each subtask necessary for the comparison. Enumerating all of them was difficult in itself, and designing features and collecting data based on them was also extremely difficult.

On the other hand, a neural language understanding model uses a neural network to perform end-to-end learning from input to output, enumerating each subtask, designing features, and preparing data specialized for each subtask. Since it is possible to construct a highly accurate language understanding model without the need for this, it is advantageous when comparing explanatory texts related to technology. Neural language models generally use distributed representations that correspond to input tokens such as words. Furthermore, since the neural language understanding model deals with language, it is necessary to satisfy the following requirements.
a) Can handle sentences of arbitrary length. In other words, it can handle variable length data.
b) A mechanism can be used that takes into account the order of appearance (chronological order) of tokens (words, etc.).
c) It can handle long-distance dependencies between distant tokens.
Existing mechanisms with such characteristics include CNNs, RNNs, and models using positional embeddings, but any model can be used as a neural language understanding model. For example, models using CNN and RNN can successfully handle time-series data that cannot be handled by simple neural networks. When CNN and RNN are used as encoders, variable-length token strings can be treated as time series. A model that combines these encoders and decoders is called a sequence-to-sequence model, and is mainly used for machine translation and the like.

Furthermore, there is a transformer as an example of an encoder/decoder model using positional embedding. The encoder part of the transformer is used because the transformer can successfully supplement the knowledge of long-range dependencies between tokens by using self-attention, and has the advantage of having a small amount of calculation compared to the sequence-to-sequence model. Research is also progressing.

Here, Non-Patent Document 1 proposes a technology called BERT (Bidirectional Encoder Representations from Transformers) that uses the encoder part of a transformer, which is a model equipped with a powerful encoder that uses an attention mechanism. This was achieved by pre-training the language model using large-scale data, and then performing relatively small-scale task-specific fine-tuning. It is a model that has achieved the highest performance (state of the art) and has been attracting attention in recent years.

Jacob Devlin and 3 others, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.” [online], October 11, 2018, [searched on April 5, 2019], Internet <URL: https: //arxiv.org/pdf/1810.04805.pdf〉

In patent examinations and patent invalidation searches, it is necessary to conduct prior art searches of not only domestic but also foreign documents, so we conduct a comprehensive search of prior art documents from all over the world, which accumulate in large numbers year after year. Considering that it has to be done, it is becoming more and more time-consuming. According to the Patent Administration Annual Report 2018, the number of patent applications worldwide increased from 1.874 million in 2007 to 4.128 million in 2016, increasing 1.7 times in the past 10 years. ing.

In conventional prior art document searches, after searching by combining keywords and patent classifications, cited inventions are identified based on the prior art documents discovered from the search results, and the claimed invention is identified. In addition to the trouble of searching to find a single prior art document, the process involves comparing the cited invention with the cited invention and identifying points of agreement and differences. Points and differences must be recognized, and in order to obtain highly accurate results, patent examinations and patent invalidity searches must be conducted by experts in the subject technology over a long period of time.

Furthermore, in patent examinations and patent invalidation searches, it is preferable to present (support) not only the search results for cited inventions, but also the reasons why cited documents can be cited so that applicants and clients can clearly understand them.

Note that Non-Patent Document 1 does not specifically disclose patent examination, patent invalidity search, etc., nor does it disclose a specific method for searching for documents that have matching points with the search target. Furthermore, Non-Patent Document 1 describes a task of comparing two sentences and extracting a specific part of the sentence, but does not describe a task of comparing two sentences and determining a label for each token. do not have.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a technology that supports searching and investigating documents that have matching points with a research target.

In order to solve the above-mentioned problems, one of the typical research support methods of the present invention is a method for a computer system to support literature research, in which information about matching points between a target sentence and a reference sentence is collected. The method includes an input step of acquiring, and a display control step of generating a display screen with screen processing that enables matching points to be determined, and making the display screen displayable.

INDUSTRIAL APPLICATION This invention becomes possible to support the search investigation of the document which has a point of coincidence with a research object.
Problems, configurations, and effects other than those described above will be made clear by the following description of the embodiments.

FIG. 1 is a diagram illustrating the overall configuration of the investigation support system. FIG. 2 is a diagram illustrating the basic configuration of BERT. FIG. 3 is a diagram illustrating machine learning of a language understanding model. FIG. 4 is a diagram showing an example of converting an input target token string into an output target token string. FIG. 5 is a diagram showing an example of converting an input target token string into an output target token string. FIG. 6 is a diagram illustrating a screen interface. FIG. 7 is a diagram showing a document search system to which the research support system is applied. FIG. 8 is a diagram illustrating a screen interface of the document search system. FIG. 9 is a flowchart (1/2) showing the operation of the investigation support system. FIG. 10 is a flowchart (2/2) showing the operation of the investigation support system.

Embodiments of the present invention will be described below with reference to the drawings.

<1. Overall configuration of investigation support system 100>
FIG. 1 is a diagram illustrating the overall configuration of a research support system 100.
In the figure, an investigation support system 100 includes a determination section 101 and a calculation section 102.
The determination unit 101 includes a claim holding unit 11 , a document holding unit 12 , a preprocessing unit 13 , a conversion unit 14 , and a language understanding model 15 .
The calculation section 102 includes an input section 103, a ranking section 104, a difference extraction section 105, and a display control section 106. Regarding the calculation unit 102, in more detail, it is possible to use the structure of a document search system applying the investigation support system shown in FIG. 7, which will be described later.

The claim holding unit 11 decomposes the input natural language claim sentence CLMn (target sentence) into tokens using a morphological analyzer such as Mecab or a tokenizer such as SentencePiece and holds the tokens. The token here is a term in the technical field of natural language processing, and represents a basic unit that constitutes a natural language sentence. Preferably, a token is a unit that is shorter than a phrase and longer than a character.

The document holding unit 12 disassembles and holds a set of patent documents PDm (reference sentences) into token strings.

The preprocessing unit 13 connects the claim statement CLMn and the patent document PDm one-on-one, one after the other, to form one input target token string INi. If the length of the token string of at least one of the claim statement CLMn and the patent document PDm is too large and causes problems in subsequent operations, change the window width of a specific size from the claim statement CLMn at regular intervals. By shifting, CLMnj is generated as part of the token string, PDmk is similarly generated as part of the token string with patent document PDm, CLMnj and PDmk are connected back and forth, and j and k are all combined. This may be used as the input target token string INi.

The conversion unit 14 uses the language understanding model 15 to convert the input target token string INi into the output target token string OUTi.

The input unit 103 acquires the output target token string OUTi from the determination unit 101 as information regarding the matching point. The output target token string OUTi is, for example, a binary label, and for tokens that have the same index as the tokens in the corresponding input target token string INi, ``YES'' is displayed for tokens that are a match, and ``YES'' is displayed for tokens that are not a match. Information regarding the matching point is represented as "NO". (Details are described in Figure 4-5 below.)

The ranking unit 104 evaluates the suitability of reference sentences as comparison targets based on information regarding matching points. Depending on the progress of the process, the comparison target here is the first reference sentence to be compared (hereinafter referred to as the "main reference example") before the difference extraction described below, and the second and subsequent reference sentences after the difference extraction. This corresponds to each of the reference sentences (hereinafter referred to as "sub-citations") that are compared.
For example, the ranking unit 104 increases the eligibility evaluation as the number of matching points (for example, the number of matching tokens) in the target sentence increases.
For example, the ranking unit 104 increases the evaluation of eligibility as the proportion of matching points in the target sentence (for example, the value obtained by dividing the number of matching tokens by the number of previous tokens) increases. In addition to the ratio of the number of characters and the number of tokens, this ratio may take into account the degree of sparseness in which a plurality of matching points are concentrated or dispersed in the target sentence.
Also, for example, the ranking unit 104 increases the evaluation of eligibility as the evaluation value of the matching points becomes higher. For example, the evaluation value of the matching point may be set higher as the token indicated by the matching point is a more specific and characteristic term, such as a term in a specialized field. Furthermore, for example, the evaluation value of the matching point may be weighted higher as the location where the matching point appears in the target sentence is an important or basic location determined from the format or the like.
Furthermore, the suitability of the reference sentence as a comparison target may be comprehensively evaluated by combining these items using an evaluation function or by adding other evaluations.
Furthermore, the ranking unit 104 ranks a plurality of reference sentences based on the determined suitability.
Note that the functions of the ranking unit 104 such as eligibility evaluation and ranking overlap at least in part with the functions of the result holding unit 45 and the comprehensive output unit 46 (see FIG. 7), which will be described later. Duplicate explanations of functions similar to those of the section 45 and the general output section 46 will be omitted. Further, at least a part of the functions of the result holding section 45 and the comprehensive output section 46 may be employed as the functions of the ranking section 104.

The difference extracting unit 105 finds differences by removing points that match the main citation from the target sentence. Further, the difference extraction unit 105 adjusts the difference extraction range so that the target sentence becomes a meaningful word unit (hereinafter referred to as a linguistic unit) such as a sentence, a clause, a word, or a morpheme. This prevents the extraction of differences that have no linguistic meaning, such as in cases where only part of a morpheme is recognized as a difference when using Sentence Piece or the like. Further, the difference extracting unit 105 inputs the thus adjusted differences to the claim holding unit 11 again, and provides the determination unit 101 with searching for sub-citations.
Note that the function of processing differences of the difference extraction unit 105 overlaps at least in part with the function of the automatic query construction unit 44 (see FIG. 7), which will be described later. Duplicate explanations of functions similar to those will be omitted. Further, at least a part of the functions of the automatic query construction section 44 may be employed as the functions of the difference extraction section 105.

The display control unit 106 generates screen interfaces such as a display screen with screen processing added to enable identification of matching points, a ranking screen for primary citation candidates, and a comparison screen for sub-citations and differences, and displays the screen on the user terminal. Send to.

The investigation support system 100 having such a configuration is configured by, for example, a computer system including a CPU (Central Processing Unit), a memory, a database, and the like as hardware.

When this hardware executes the investigation support computer program, the investigation support method shown in FIGS. 9 and 10, which will be described later, is carried out.

Part or all of this hardware may be replaced by equivalent means such as a DSP (Digital Signal Processor), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit). For example, the conversion unit 14 (language understanding model 15) in which a large number of vector operations occur is realized by a calculation means such as a GPU equipped with a large number of calculation cores, and the calculation unit 102 in which processing such as decision branching occurs is implemented by a CPU, etc. It may be realized by the calculation means.

Further, part or all of the hardware may be centralized or distributed on servers on a network and placed in the cloud so that multiple people can use it jointly via the network.

<2. Description of language understanding model 15>
Next, the language understanding model 15 will be explained.

As mentioned above, the language understanding model 15 is a model whose purpose is to make a computer understand natural language like a human being does. In this specification, BERT, which is a type of neural language understanding model, will be mainly explained, but other neural language understanding models or traditional language understanding models may be used.

FIG. 2 is a diagram illustrating the basic configuration of BERT. BERT is configured by connecting transformer encoders Tm in a network. E1 to En are input embeddings, which may be generated by adding token embeddings, segment embeddings, and position embeddings. Each transformer encoder Tm is configured with a multihead attention 21, a residual connection/normalization block 22, a feedforward network 23, a residual connection/normalization block 24, and the like. T1 to Tn are contextual representations corresponding to each input token.

<3. Machine learning of language understanding model 15>
Next, machine learning of the language understanding model 15 will be explained.
FIG. 3 is a diagram illustrating machine learning of the language understanding model 15.

In the figure, a learning sentence that is known to have matching points in advance is used as input data. For example, in patent trial decision documents, there are points that are semantically (technically) consistent, such as "the claimed invention" and "matters stated in the main citation," even though they are expressed differently. Part of the text data is stored. Which parts of the "claimed invention" are the points of agreement with the main cited example are specifically described in ``Identification of points of agreement''. Note that similar matters can be extracted from patent examination documents as well as patent trial decision documents.

After each training text is divided into tokens, it is divided into tokens and has some technical similarities, such as ``the claimed invention subject to trial decision'' and ``matters stated in the main citation.'' One input data is created by connecting sentences before and after each other with appropriate tokens in between. At this time, a symbol representing a sentence break ([SEP]) may be added as an appropriate token, and a symbol representing a sentence break ([CLS]) or a sentence end symbol (similar to the symbol representing a sentence break) may be added. SEP]) may be added. On the other hand, by using longest partial matching between the descriptions of "the invention claimed in the claim that is the subject of the trial decision" and "identification of matching points," the "invention claimed in the claim that is the subject of the trial decision" is divided into "main" in token units. The results determined to be consistent or inconsistent with the matters stated in the references are created as output data.

Alternatively, between the "invention claimed in the claim subject to trial decision" and "matters stated in the main citation", appropriate tokens may be inserted between sentences that are determined to have extremely low relevance using a specific index. , and connect them before and after each other to create output data in which one input data is created and all tokens are determined to be inconsistent.

Further, the learning data may be increased using a known data expansion method. A learning set is obtained by collecting such input/output data as learning data. The language understanding model 15 performs machine learning using the learning set.

In addition, in a model such as BERT that uses attention inside the encoder, when outputting the context expression Ti corresponding to each input token, refer to the input embedding Ei corresponding to itself in the "claimed invention subject to trial decision". In addition to referring to the input embedding Ej in "matters stated in the main citation," as a result of learning, the "invention related to the claim subject to the trial decision" and "matters stated in the main citation The internal parameters of the language understanding model 15 are updated so that tokens that technically and semantically match the item ``matter'' have a high degree of relevance.
Through such machine learning, among the language understanding models 15, models such as BERT that use attention generate high values at the corresponding attention positions for combinations of tokens that are estimated to be matching points. become.

<4. Estimated operation of language understanding model 15>
The estimation operation of the language understanding model 15 that has undergone machine learning will be explained.
FIG. 4 is a diagram showing an example in which the conversion unit 14 uses the language understanding model 15 to convert the input target token string INi to the output target token string OUTi.

As the output target token string OUTi, among the tokens of the input natural language claim statement CLMn (CLMnj), the tokens corresponding to the tokens whose content matches the description in the patent document PDm (PDmk) are set as "YES". , and the others are output as "NO" (pattern 1).

In this way, the cited invention is recognized by outputting points of agreement (“YES”) and points of difference (“NO”) with the patent document PDm on a token-by-token basis for the input natural language claim statement CLMn. By omitting the step of comparing the claimed invention and one cited invention, or by correcting the mechanically determined result, it is possible to save human labor.

As another configuration, the conversion unit 14 outputs the number of “YES” tokens COUNTyes (OUTi) in addition to outputting “YES” and “NO” as they are based on the estimation processing of the language understanding model 15. It's okay. A configuration may be adopted in which the higher the value of this COUNTyes(OUTi), the higher the eligibility evaluation. Furthermore, based on these numbers, the display order (ranking) of the main citation example candidates, which will be described later, may be rearranged.

This makes it possible to display a large amount of documents in some way distinguishable according to the score, allowing more effective prior art documents to be quickly discovered, and making it possible to devise queries using combinations of keywords and patent classifications. This will reduce the amount of time and effort required, making it possible to conduct patent examinations and patent invalidation searches more efficiently.

Among the descriptions in the patent document PDm (PDmk), the tokens corresponding to the tokens that match the natural language claim statement CLMn (CLMnj) may be output as "YES", and the others as "NO" ( Pattern 2), both of the outputs corresponding to Pattern 1 and Pattern 2 may be output. As a result, not only the matching portions of the claims but also the matching portions of the patent documents can be found, so that when a human being corrects the output result, it is possible to quickly refer to the comparison portion.

FIG. 5 shows a configuration in which only pattern 1 is used and pattern 2 is not used. In such a case, any token corresponding to pattern 2 may be input, and either all points of agreement ("YES") or points of difference ("NO") may be input.

Patent document PDm includes not only text information such as claims and specifications, but also captions that are automatically generated from drawings to explain the content described in the drawings, character strings in codes and flow diagrams, etc. May contain the character strings listed in . This makes it possible to identify points of agreement and differences using not only text information but also drawings.

The set of patent documents PDm in the document holding unit 12 may be the entire text or part of the patent documents. Abstracts and summaries of patent documents may also be included.

Furthermore, the set of patent documents PDm in the document holding unit 12 may be selected by a prior search formula, or some or all of the documents in at least one database may be selected. With this, it is possible to perform document conversion only on documents that have a higher probability of becoming cited documents, without having to perform time-consuming document conversion on all documents.

The input target token string INi may include a sentence initial symbol ([CLS]), a sentence break symbol ([SEP]), and a sentence end symbol ([SEP]), and the output target token string OUTi does not include these symbols. You can output the symbol as is.

Of the claim statement CLMn and the patent document PDm, words and morphemes specified by the user or automatically specified by the algorithm are placed in some kind of placeholder before being made into the input target token string INi. It may be replaced, and in that case, the output of the corresponding token may also be output as a placeholder. This reduces the possibility that if a new word that is not in the original learning data appears in a claim or cited document, it will be determined as a mismatch. Then, for the word replaced by the placeholder, after the output target token string OUTi, the claim statement CLMn and patent The documents PDm may be compared to determine whether the token corresponding to the word replaced by the placeholder matches or does not match. With such a configuration, it is possible to accurately determine whether new words that are not in the learning data match or do not match.

The language understanding model 15 used by the conversion unit 14 may be a model using a transformer, such as BERT (Bidirectional Encoder Representations from Transformers) or a model that is an extension of BERT such as MT-DNN. , an RNN sequence-to-sequence model with attention may be used, or some other model related to language understanding may be applied.

Among the input target token string INi, output target token strings corresponding to symbols such as tokens and sentence initials corresponding to patent document PDm (PDmk) may be output as "NO" for convenience. This makes it possible to reduce the total number of labels assigned to each token. Further, another type of token may be provided to represent these tokens. This makes it possible to perform COUNTno, which will be described later, with fewer processing steps. Furthermore, a "?" token may be added to indicate irrelevance, such as pending judgment. As a result, if the training data is good, it is not necessary to forcefully classify items that cannot be compared well into ``YES'' and ``NO'', which is expected to further improve accuracy.

CLMnj and CLMnj-1, which are obtained by dividing CLMn, and PDmk and PDmk-1, which are obtained by dividing PDm, each hold a token that corresponds to the same part of the original CLMn and PDm, and each of them has a different judgment with YES/NO. If so, "YES" may be output with priority. As a result, even if there is no statement in the divided part that allows direct comparison between the claim and the cited document, if they match somewhere in the divided part, the corresponding token in OUTi will also be "YES", which will be the correct result. Therefore, even when CLMn and PDm are divided and compared, it is possible to output correct results.

In addition, in the divided part, the number of tokens on the left and right of the token is scored, the reliability of the output token is calculated based on the score, and OUTi's YES/NO is determined based on the reliability. may be determined. As a result, the target token happens to be at the beginning of a sentence in the divided part, and the result of an erroneous determination can be avoided without being reflected in OUTi without being affected by the token that should originally exist before it. Further, a "?" indicating that the determination is pending may be output, or YES/NO may be determined by rule-based machine learning.

COUNTyes (OUTi) may separately output only the premise description part of the "YES" token (such as the "in" part in Jepson format claims) as COUNTyes (OUTi (preamble)). As a result, during ranking, which will be described later, it is possible to target only documents whose premises completely match, and it is possible to exclude documents whose premises are completely different. In addition, only the remaining part (≒ characteristic part of the invention) after excluding the premise part may be output separately, and only the important words, verb parts, and chemical formula parts listed in advance can be output using COUNTyes (OUTi (important words)). It may be output separately as COUNTyes (OUTi (verb)) or COUNTyes (OUTi (chemical formula)). Furthermore, instead of the number of "YES" tokens COUNTyes, the number of "NO" tokens COUNTno and the number of "?" tokens COUNT? may be output.

Regarding COUNTyes (OUTi), for example, a token created based on SentencePiece may be re-created based on a morphological analyzer such as mecab in order to weight a specific morpheme. In that case, if the tokens based on SentencePiece are smaller than the morphemes, and each token is divided into "YES" and "NO", the one with the largest number may be designated as "YES", or it may be determined by majority vote. Often, decisions can be made based on rules or machine learning. Furthermore, if the token based on SentencePiece is rougher than the morpheme, it may be determined that each morpheme is the same "YES" and "NO" as SentencePiece. If the SentencePiece and the morpheme break are misaligned, the SentencePiece may be first decomposed at the same break as the morpheme break, and then the method described above when the SentencePiece-based token is smaller than the morpheme may be applied.

<5. Screen interface of investigation support system 100>
Next, the screen interface will be explained.

FIG. 6 is a diagram illustrating a screen interface of the investigation support system 100.
In the figure, a claim to be the subject of patent examination or patent invalidation investigation is input into an input area 31 (display area for target sentences). Furthermore, at least a portion of the currently selected cited document is displayed in the cited document display area 32 (reference text display area). A literature list is displayed in the literature list display area 33 (a list display area of a plurality of reference sentences). Furthermore, in order to create a set of patent documents PDm to be converted, a text box for inputting keywords, IPC (International Patent Classification), FI, etc., and a search button may be provided.

In the input area 31, a claim can be entered, the claim is displayed, and a portion of the statement of the claim that corresponds in content to the statement of the document in the cited document display area 32 is displayed. The color of has changed (highlighted). By highlighting, it becomes possible to quickly check the mechanically recognized matching parts of claims on the screen.

At least a portion of the cited document is displayed in the cited document display area 32, and the color of the portion of the document description that corresponds to the content of the document description in the input area 31 is changed ( Highlighting). By highlighting, it is possible to quickly check the matching parts of cited documents that have been mechanically recognized on the screen. Note that the matched portion may be displayed in units of paragraphs or in units of predetermined characters in the cited document display area.

In the input area 31 or the cited document display area 32, a person can carefully examine the contents of the cited document display area 32 in the portion where the color has been changed. After careful inspection, if the content matches (does not exist), it is determined that the claim and the cited document description correspond (do not correspond) in the input area 31 or the cited document display area 32. Select the part to be colored (return to original color) with the mouse. A strip menu may be displayed by right-clicking on the selected portion, and a content match (mismatch) may be selected from the menu, or it may be colored manually (returning to the original color).

With the above configuration, a document searcher can easily refer to the parts mechanically determined as matching points, identify cited inventions from scratch, and identify the claimed invention and one cited invention. Labor can be saved by omitting the step of comparing the results or by modifying the mechanically recognized results.

The document list display area 33 describes a list of target patent documents, and by selecting a document in the document list display area 33, the cited document display area 32 is changed to the corresponding document. In the literature list display area 33, multiple scores based on COUNTyes (OUTi) are displayed as "matching index ○", and by clicking on the "matching index ○" column or the column in which the index name is written, Documents can be sorted based on the score.

As a result, a large number of documents can be displayed in some way to distinguish them according to their scores, and more effective prior art documents can be quickly discovered, and documents sorted by score can be viewed from above. By doing so, the time and effort required to devise queries based on combinations of keywords and patent classifications is reduced, making it possible to perform patent examinations and patent invalidity searches more efficiently.

Note that a configuration may be adopted in which only the highlighted portion of the input area 31 is displayed as the matching point display area 34 separately from the input area 31. This allows the points of agreement to be viewed as a single sentence, making it easier for literature researchers to closely examine whether there are any important structures that do not match.

As a "match index ○", in addition to sorting based on the number of "YES" tokens COUNTyes (OUTi) mentioned above, only documents whose preambles are all "YES" (COUNTyes (OUTi (preamble)) ==COUNT (OUTi ( Even if you sort using multiple COUNT conditions, such as sorting by the number whose remaining part is "YES" (COUNTyes(OUTi(not preamble))) Alternatively, you can sort based on the overall COUNT score, such as adding weights to multiple COUNT conditions.In addition, you can simply use the number of "YES" tokens COUNTyes (OUTi) as is. Instead, COUNTyes (OUTi) normalized in some way may be used.

Highlighting may be expressed by color type and density, or by symbols etc. instead of color. You may also change the font of the text, change the thickness of the text, add some kind of text decoration such as adding an underline, or apply some commonly used font changes. Good too. You can also add animation to the characters.

When using the attention mechanism, the color of the token on the cited document display area 32 side that corresponds to the token on the input area 31 side that is moused over may be particularly emphasized, or the description on the cited document display area 32 side may be highlighted. The color of the corresponding part on the input area 31 side may be particularly emphasized by hovering over the mouse. This makes it easier for people to examine which parts of each token match. Regarding color enhancement, a plurality of colors may be used corresponding to each of the outputs of a plurality of attention heads, or a single color associated with the average of the attention heads may be used.

Based on the result of manually coloring/returning to the original color, the arrangement order of the document list display area 33, which will be described later, may be changed again. Note that for the parts that have been manually colored/restored to the original color, the colored parts are changed to YES and the parts restored to the original color are changed to NO for the output target token string OUTi. Then, using the output target token string OUTi' that reflects the result of manually coloring/returning to the original color, the arrangement order of the document list display area 33, which will be described later, is changed again from the value of COUNTyes (OUTi'). do. This enables even more accurate sorting that reflects the results of manual scrutiny. Furthermore, by saving the results of manually coloring/returning to the original color and using it as learning data, it is possible to further improve the accuracy of comparison. For example, the result of coloring/returning to the original color by a person is stored in a cloud server (not shown), and the claim statement CLMn, patent document PDm, and an output reflecting the result of coloring/returning to the original color by the person. The target token sequence OUTi'' is included in at least a part of the new learning data, and the BERT model is periodically fine-tuned to improve accuracy.

<6. Application to document search system>
FIG. 7 is a diagram showing a document search system to which the research support system 100 is applied.

In the figure, a patent application holding unit 41 holds claims, specifications, and drawings of patent applications to be searched. The document conversion unit 42 corresponds to the determination unit 101 in the investigation support system 100 in FIG.

The plurality of databases 43 are databases of patent documents and non-patent documents, and specific document data is extracted from the data held in the database 43 based on a search using a search query or other document selection methods. The converter 42 is provided with at least some documents.

The automatic query construction unit 44 generates an edited claim (hereinafter referred to as “edited claim”) that is newly input to the document conversion unit 42 from the claim input to the document conversion unit 42 and the output result of the conversion unit 14 shown in FIG. Generate a completed claims query The result holding unit 45 holds the output result of the converting unit 14 in FIG. The comprehensive output unit 46 creates an output result from the result of the result holding unit 45.

The present application holding unit 41 edits the claims of the patent application to be searched or inputs them to the document converting unit 42 without editing them. As an editing method, when searching for claim A, assuming that claim B cites claim A, for example, (1) connect A and B with a conjunction or the like to form one sentence. (2) If B contains the statement "characterized by using XX in place of XX," create a claim in which XX in A is replaced with XX. (3) If there is a description of “○○, △△, or ××”, write three sentences for A that include each of the three structures. (4) Extract issues, etc. from the specification and add them to A. (5) Various methods can be considered, such as adding a caption automatically generated from the drawing to the invention specifying matter in A.

The document conversion unit 42 outputs an output result based on the input claim or edited claim query and the contents of the document holding unit 12. The plurality of databases 43 select patent documents in the database by combining at least one or more results of existing search methods such as keyword search, classification search such as IPC or FI, or concept search, or Based on another document selection method, at least some documents or all documents are provided to the document holding unit 12 according to FIG.

The automatic query construction unit 44 passes the claims in the main application holding unit 41 to the document conversion unit 42 as they are, or converts them into one In order to present a new document to fill in the discrepancies when the discrepancy cannot be covered by the second document, a new claim is constructed to be passed to the document conversion unit 42 by referring to the data in the result holding unit 45. do. For example, (1) to (4) below.

(1) The "YES" part in the output target token string OUTi has already been searched and is a match between the claim and the target document. Next, in order to search for the remaining differences, the document conversion unit 42 The edited claim query entered in OUTi should contain only tokens that correspond to "NO" in OUTi. At that time, in order to prevent the document from becoming unnatural, even if only a specific token in one word is "YES", in order to maintain the correct word form, the token in OUTi that corresponds to the specific token is It may be considered as "NO" and included in the claim query, or particles etc. may be included in the claim query based on rule base/machine learning so as not to make the grammatical structure strange.

(2) Furthermore, in addition to (1), words may be made into higher-level concepts as appropriate in order to make the search target more appropriate. For example, "mobile phone" in a claim query may be converted into "wireless communication device" using a synonym dictionary created using machine learning or the like, and then converted into a higher-level concept.

(3) In addition to (1), in order to prevent the dependency relationship from becoming strange, if the dependency source is a token that corresponds to "NO", it is assumed that the modification destination is a token that corresponds to "YES". The token in OUTi corresponding to the specific token may also be considered as "NO" and included in the claim query.

(4) In addition to (1), regarding the patent document PDm, the tokens corresponding to the tokens that match the claim query in content are set as "YES", and the others are set as "NO", and the output target token string OUTi is set. If included, the problem, structure, and field of the invention of the token corresponding to "YES" may be included in the claim query. Note that the query created by the automatic query construction unit 44 may be modified manually.

The result holding unit 45 holds the output target token string OUTi and the number of "YES" tokens COUNTyes (OUTi) dating back to the past N searches. The comprehensive output unit 46 creates an output result from the result of the result holding unit 45. For example, for document PD1 at the N-1st search, document PD2 at the N-time search using an edited claim query to search for remaining differences is determined by the claim token set corresponding to document PD1. , if one of the tokens with the same index is "YES" for the claim token set corresponding to document PD2, then the tokens with the same index in the composite claim token set of documents PD1-2 are "YES". Combinations of multiple documents are ranked and output in descending order of the number of "YES" tokens COUNTyes (OUTi) in the deemed and combined claim token set.

In addition, the comprehensive output unit 46 separately determines which parts of the edited claim query for searching for remaining differences are common general technical knowledge for the document PD1 at the time of the N-1st search, and The Nth search may be performed by editing the claim query to search for differences. Then, among the composite claim tokens, a portion determined to be common technical knowledge or well-known technology is output as "YES".

As a result, in order to directly identify differences with the invention and suggest other cited inventions that should be applied to the differences, the steps of re-searching and identifying and comparing cited inventions can be omitted. Alternatively, by making corrections to each result, it is possible to save labor and to perform patent examination and patent invalidation search more efficiently.

When re-editing the claim query, the automatic query construction unit 44 may use the output target token string OUTi and the number of "YES" tokens COUNTyes (OUTi) edited manually or using an algorithm.

For a composite claim token, how many tokens exist on the left and right of the token in the set of PD1k obtained by dividing the document at the N-1st search and the set of PD2k obtained by dividing the document at the N-th search? and calculate the confidence level based on the score; based on the confidence level, score the number of “YES” and “NO”; and based on the score, calculate whether the synthetic claim token is YES/NO. It may be determined whether there is a token, or a new token called “YES (judgment pending)” may be output for tokens with different judgments between PD1 and PD2, and YES/NO is determined by rule-based machine learning. You may.

To separately determine common general knowledge in the general output unit 46, (1) input a claim divided into tokens, and check whether a certain part of the token is determined to be common knowledge or well-known art; This may be done by applying the common general knowledge determination system 1 to be output, or (2) among the claims divided into tokens, a subset of tokens for which you want to determine whether or not they are common general knowledge or well-known technology. It is also possible to use a common general knowledge determination system 2 that inputs the following and determines whether the subset is common general knowledge or a well-known technique using a binary value. These common technical knowledge determination systems may be configured using existing language understanding models.

Among the composite claim tokens, a new token "YES (determination pending)" may be output, or YES/NO may be determined by rule-based machine learning.

FIG. 8 is a diagram illustrating a screen interface of the document search system. The parts already described in the explanation of FIG. 6 will be omitted.

In the input area 51, a claim to be the subject of patent examination or patent invalidity search is input, in the cited document display area 52 at least a part of the currently selected cited document is displayed, and in the document list display area 53. A list of literature is displayed. In addition, a text box for inputting keywords, IPC, FI, etc., and a search button may be provided in order to create a set of patent documents PDm prior to the document conversion process.

A claim is displayed in the input area 51, and the color of the part corresponding to the part of the description of the claim that matches the description of the document in the cited document display areas 52a to 52b is changed. (emphasis processing). Note that among the claims, the contents corresponding to the cited document display areas 52a to 52b may be changed to different colors. The determined portion may be changed to a different color.

At least a portion of the selected cited document is displayed in the cited document display area 52a, and the color of the portion of the document description that corresponds to the content of the document description in the input area 51 is changed. ing. In addition, at least a portion of another selected cited document is displayed in the cited document display area 52b, and a portion of the document description that corresponds to the portion that matches the document description in the input area 51 is displayed. The color has been changed.

In the document list display area 53, a list of target patent documents is displayed as a combination, for example, the main citation Y1 and the sub citation Y2 are displayed in this order. By selecting a document in the document list display area 53, the cited document display areas 52a to 52b are changed to the documents corresponding to Y1 and Y2. Here, the number of documents displayed in one line may be one or a combination of a plurality of documents.

The list of patent documents in the document list display area 53 is sorted in descending order of the number of "YES" tokens COUNTyes (OUTi) in the composite claim token set described above, so that it is more likely to be described in the claimed invention and in multiple cited documents. It is possible to carefully examine the combinations of cited documents in order from the one that most closely matches the cited invention.

As a result, in order to directly identify differences with the invention and suggest other cited inventions that should be applied to the differences, the steps of re-searching and identifying and comparing cited inventions can be omitted. Alternatively, by making corrections to each result, it is possible to save labor and to perform patent examination and patent invalidation search more efficiently.

The number of cited documents display areas 52 may be increased depending on the number of cited documents to be displayed simultaneously, or may be made selectively displayable. This makes it possible to display any number of cited documents that are easy for the user to view.

As the "match index ○", in addition to sorting based on the number of "YES" tokens COUNTyes (OUTi) mentioned above, not only using the number of "YES" tokens COUNTyes (OUTi) as is, but also normalizing COUNTyes in some way (OUTi) may also be used. Also, by simultaneously displaying multiple indicators of COUNTyes (OUTi) as "matching index ○" in the document list display area 33, and clicking the "matching index ○" column, the documents can be displayed again based on the relevant index. May be rearranged.

It may be configured such that only the highlighted portion of the input area 51 is displayed as the matching point display area 54 separately from the input area 51.

As the learning data, trial decision bulletins, notices of reasons for refusal, and search reports of search companies may be used.

<7. Operation of investigation support system>
Next, the specific operation of the investigation support system 100 shown in FIG. 1 will be explained.
9 and 10 are flowcharts showing the operation of the investigation support system 100.
The steps will be explained below in accordance with the step numbers shown in the figure.

Step S01: The research support system 100 accesses a database of patent documents (not shown) and narrows down the group of patent documents (hereinafter referred to as "reference sentences") to an appropriate number of samples by pre-search. The group of reference sentences that have been narrowed down is input to the literature holding unit 12.

Step S02: The document holding unit 12 decomposes the input reference sentences into tokens one by one and outputs them to the preprocessing unit 13. The claim holding unit 11 takes in a claim (hereinafter referred to as a “target sentence”) that is the subject of a literature search, breaks it down into token units, and outputs it to the preprocessing unit 13 . The preprocessing unit 13 generates an input target token string INi by concatenating the token group of the target sentence and the token group of the reference sentence of one document back and forth, and outputs it to the conversion unit 14. Note that how many of the target sentence token group and the reference sentence token group are included in the input target token string INi at one time may be changed as appropriate.

Step S03: The conversion unit 14 provides the input target token string INi to the input of the language understanding model 15. The language understanding model 15 compares the target sentence and the reference sentence, determines the portion of the target sentence's tokens that is written in the reference sentence as a match, and outputs the result as an output target token string OUTi. .

Step S04: The input unit 103 acquires this output target token string OUTi as information regarding matching points in token units. Based on this information, the display control unit 106 generates a screen interface that allows matching points to be determined, and displays it in the input area 31 (see FIG. 6).

Step S05: The ranking unit 104 evaluates the suitability of the reference sentence as a main citation example based on the weighted evaluation value according to the number, proportion, or position of matching points in the target sentence.

Step S06: If the ranking unit 104 has completed evaluating the eligibility of all reference sentences held by the document holding unit 12, the ranking unit 104 moves the operation to step S07. In other cases, the ranking unit 104 returns the operation to step S02 in order to process the remaining reference sentences.

Step S07: The ranking unit 104 creates a ranking of the reference sentences with higher qualifications.

Step S08: The display control unit 106 generates a screen interface that makes it possible to determine the ranking order of eligibility based on the ranking created by the ranking unit 104, and displays it in the document list display area 33 (see FIG. 6). .

Step S09: The display control unit 106 selects the reference sentence ranked first in the ranking or the rank selected by the user as a candidate for the main reference example.

Step S10: The display control unit 106 generates a screen interface that allows matching points to be determined for the main reference example candidates, and displays the screen interface again in the input area 31 (see FIG. 6).

Step S11: If an operation is performed to change the main reference example on the user terminal (see FIG. 1), the display control unit 106 returns the operation to step S09 to accept the change of the main reference example candidate. On the other hand, if the user terminal (see FIG. 1) performs an operation to confirm the primary reference example (or does not perform an operation to change it), the display control unit 106 shifts the operation to step S21.

Step S21: The difference extraction unit 105 removes points that match the determined main reference example from the target sentence, and extracts differences in units of tokens.

Step S22: The difference extracting unit 105 connects the front and rear tokens with respect to the differences in token units to generate differences in language units such as clauses or sentences. Through this processing, the differences become connected in meaning, like clauses or sentences, and the language becomes understandable. Therefore, it becomes possible to process differences including sentence meaning in the language understanding model 15.

Step S23: The difference extraction unit 105 adds at least one logical element of the problem, technical field, and function and action of the main cited example to the difference. By this processing, in searching for sub-citations to be described later, it becomes possible to search for sub-citations that have at least one of the same problem, technical field, function, and action as the main citation. Such sub-citations can be logically combined with the main citation, making them more suitable as sub-citations.

Step S24: The difference extraction unit 105 outputs the difference to the claim holding unit 11. The preprocessing unit 13 generates an input target token sequence INi by concatenating the token group of differences and the token group of one reference sentence back and forth, and outputs it to the conversion unit 14. The conversion unit 14 provides the input target token string INi to the input of the language understanding model 15. The language understanding model 15 outputs an output target token string OUTi that indicates points of difference and points of agreement with the reference sentence on a token-by-token basis. The ranking unit 104 evaluates the suitability of a reference sentence as a sub-citation based on a weighted evaluation value according to the number, proportion, or position of matching points among different points.

Step S25: When the ranking unit 104 completes the evaluation of the suitability of all reference sentences held by the document holding unit 12, the ranking unit 104 moves the operation to step S26. In other cases, the ranking unit 104 returns the operation to step S24 in order to process the remaining reference sentences.

Step S26: The ranking unit 104 creates a ranking of reference sentences with higher suitability as sub-citations. The display control unit 106 generates a screen interface that makes it possible to determine the ranking order of eligibility based on the ranking created by the ranking unit 104, and displays it on a screen similar to that shown in FIG.

Step S27: The display control unit 106 selects the reference sentence ranked first in the ranking or the rank selected by the user as a candidate for the sub-citation.

Step S28: The display control unit 106 generates a screen interface that allows matching points to be determined for the main citation example candidates, and displays it on a screen similar to that shown in FIG.

Step S29: If an operation to change the sub-citation is performed on the user terminal (see FIG. 1), the display control unit 106 returns the operation to step S27 to accept the change of the sub-citation candidate. On the other hand, if the user terminal (see FIG. 1) performs an operation to confirm the sub-citation (or does not change the sub-citation), the display control unit 106 shifts the operation to step S30.

Step S30: The difference extracting unit 105 generates a new difference by excluding points that match the confirmed sub-citation from the differences.

Step S31: If an operation to continue searching for sub-citations is performed on the user terminal (see FIG. 1), the display control unit 106 returns the operation to step S22 to continue searching for further sub-citations. On the other hand, if the user terminal (see FIG. 1) performs an operation to complete the sub-citation search (or does not continue the search), the display control unit 106 completes the operation.
The investigation support method is executed through the series of operations described above.

<8. Effects of Examples>
Hereinafter, the effects of the above-described embodiment will be explained.

(1) In the embodiment, a display screen is generated and made displayable based on information regarding the matching points between the target sentence and the reference sentence, with screen processing added to make it possible to determine the matching points. Therefore, it becomes possible for a literature researcher to grasp the points of agreement between the target sentence and the reference sentence.

(2) In the embodiment, information regarding matching points is obtained for each token. Therefore, it is possible to determine matching points at a finer level of granularity than in units such as sentences, paragraphs, sentences, and clauses, which makes it possible to determine which constituent features are different than when determining the match between a claim and a cited example in units of clauses. It becomes possible to specifically present to the user whether the

(3) In the example, a neural language understanding model is used to determine the match between sentences, rather than simple keyword matching, so synonyms are considered using the distributed representation of words, and end-to-end analysis is performed using the neural language understanding model. It is possible to perform more precise matching points by taking into account the parts of speech, syntactic structure, word relationships, etc. of sentences acquired through learning. For example, it is possible to distinguish homographs not only based on the coincidence of tokens, but also based on the tendency of other tokens that co-occur in the front and back directions of the tokens. It becomes possible to determine matching points by distinguishing between them.

(4) In the embodiment, the suitability of a reference sentence to a target sentence as a main citation example is evaluated according to the number and proportion of matching points in the target sentence, or an evaluation value based on the matching points. Therefore, it becomes possible to use a plurality of reference sentences as main reference examples and compare which reference sentences are suitable as main reference examples. In addition, unlike when simply using search rankings, eligibility is evaluated based on the match points in the target sentence, so it is possible to evaluate the match points in detail as to why the reference sentence is highly qualified. By displaying the information in a transparent manner, it becomes possible to present the information to the user, and the user can utilize the information after evaluating the basis for its high suitability.

(5) In the embodiment, the evaluation value is determined by weighting the matching points according to their appearance in the target sentence. For example, in the case of a target sentence created according to a predetermined format such as a claim of a patent, important parts indicating characteristics can be selected according to their appearance in the target sentence. Therefore, by increasing the evaluation weight for matching points that appear in important parts of the target sentence, and weakening the evaluation weight for matching points that appear in unimportant parts, it is possible to determine whether or not they are suitable as main citation examples. This makes it possible to make a judgment taking into account the degree of occurrence.

(6) In the embodiment, it is possible to display a display screen on which it is possible to determine the ranking order of suitability as a main reference example for a plurality of reference sentences. Therefore, it becomes possible for a literature researcher to grasp which reference sentence is eligible as a main citation.

(7) In the example, it is possible to find points of difference by removing points of agreement between the target sentence and the main reference example from the target sentence according to the main reference example selected from the reference sentences based on eligibility. Become.

(8) In the embodiment, points of agreement between the target sentence and the main reference example are removed from the target sentence, and based on the remaining tokens, linguistic units are extracted from the target sentence and used as differences. Therefore, it becomes possible to extract differences from the target sentence in units having sentence meaning.

(9) In the embodiment, sub-citations that match part or all of the differences are searched. Therefore, it is possible to support literature searchers in searching for sub-citations.

(10) In the example, at least one logical element of the problem, technical field, function, and action extracted from the main cited example is added to the difference, and the sub cited example has points that match the logical elements of the primary cited example. Search for. Therefore, the sub-citations to be searched have one of the logical elements of "commonality of problem," "commonality of technical field," and "commonality of function or action" with respect to the main citation. Because of such logical elements, it is possible to establish a rationale that combines the invention described in the main citation with the invention described in the sub-citation, so it becomes possible to search for sub-citations that are useful for determining the inventive step of a patent.

(11) In the embodiment, points that match the sub-citations are reduced from the differences, and the sub-citation search is repeated for the remaining differences. Therefore, it becomes possible to recursively search for sub-citations that are useful in determining the inventive step of a patent.

(12) In the embodiment, a display screen is generated and displayed on which the correspondence between sub-citations and differences can be determined. Therefore, it becomes possible for the document searcher to grasp which reference sentences are eligible as subcitations.

(13) The embodiment includes a screen interface that highlights portions corresponding to matching points in the target sentence display area. Therefore, it becomes possible for a literature researcher to easily understand which part of the target sentence the reference sentence matches.

(14) In the embodiment, the ranking can be rearranged by changing the judgment items regarding the suitability of the main citations and sub-citations. Therefore, the literature searcher can compare which reference sentences are suitable as main citations and sub-citations while switching judgment items from various viewpoints.

<9. Supplementary information for examples>
In addition, in the embodiment, the case where support for patent examination and patent invalidation investigation was particularly explained. However, the present invention is not limited thereto. It can be widely applied to determining matching points between sentences. For example, it may be applied to searches for sentences similar to the target sentence in response to plagiarism issues in papers and reports, copyright issues in texts, etc.

Further, in the embodiment, a case in which a language understanding model is used has been particularly described. However, the present invention is not limited thereto. For example, matching points may be determined based on a rule base, thesaurus dictionary data, or a count base.

Further, in the embodiment, the case where the evaluation of suitability such as the number and proportion of matching points and the evaluation value is performed after outputting the language understanding model has been particularly described. However, the present invention is not limited thereto. The suitability may be evaluated within the language understanding model and the suitability may be output as part of the output of the language understanding model.

Note that the present invention is not limited to the above-described embodiments, and includes various modifications. For example, the embodiments described above are described in detail to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to having all the configurations described.
Furthermore, it is possible to add, delete, or replace some of the configurations of the embodiments with other configurations.

DESCRIPTION OF SYMBOLS 11... Claim holding part, 12... Literature holding part, 13... Preprocessing part, 14... Conversion part, 15... Language understanding model, 41... Main application holding part, 42... Document conversion part, 42... Document conversion part, 43... Database, 44...Query automatic construction section, 45...Result holding section, 46...Comprehensive output section, 100...Investigation support system, 101...Judgment section, 102...Calculation section, 103...Input section, 104...Ranking section, 105...Difference Point extraction section, 106...display control section

Claims (13)

A method for a computer system to support literature research, the method comprising:
an input step for obtaining information about points of agreement between the target sentence and the reference sentence;
a determination step of inputting the target sentence and the reference sentence to a language understanding model that determines matching points for each token, and obtaining information regarding the matching points;
An investigation support method comprising the steps of: generating a display screen to which screen processing has been applied to make it possible to determine the matching points, and displaying the display screen. In the investigation support method according to claim 1 ,
A research support method characterized in that the language understanding model is a neural language understanding model. In the research support method according to claim 1 or 2 ,
It is characterized by comprising a ranking step of evaluating the suitability of the reference sentence to the target sentence as a main comparison target (hereinafter referred to as "main reference example") according to an evaluation value based on the matching points in the target sentence. Research support method. In the investigation support method according to claim 3 ,
The ranking step includes:
The evaluation value is obtained by weighting the matching points according to the occurrence location in the target sentence, and the suitability of the reference sentence as the main citation example is evaluated according to the evaluation value. Research support method. In the investigation support method according to claim 3 or 4 ,
The display control step includes:
An investigation support method characterized by: generating a display screen that includes screen processing that makes it possible to determine the ranking order of the eligibility as the main citation examples for the plurality of reference sentences, and making the display screen displayable. . In the investigation support method according to claim 4 or 5 ,
a difference extraction step of determining differences by removing the matching points between the target sentence and the main reference example from the target sentence based on the main reference example selected from the reference sentences based on the eligibility; An investigation support method characterized by comprising the following. In the investigation support method according to claim 6 ,
The difference extraction step includes:
Research support characterized by extracting linguistic units from the target sentence based on remaining tokens after removing the matching points between the target sentence and the main reference example from the target sentence, and using the linguistic units as the differences. Method. In the investigation support method according to claim 6 or 7 ,
A sub-citation search step for searching for a sub-comparison target (hereinafter referred to as "sub-citation") that matches some or all of the differences.
An investigation support method characterized by: In the investigation support method according to claim 8 ,
The difference extraction step includes:
Adding at least one logical element of the problem, technical field, function, and action extracted from the main cited example to the difference,
The sub-citation search step includes:
An investigation support method, comprising: searching for the sub-citation having the matching point with the logical element of the main citation. In the research support method according to claim 8 or 9 ,
The difference extraction step includes:
Excluding points of agreement between the differences and the sub-citations from the differences,
The sub-citation search step includes:
Regarding the remaining differences, search for the next sub-citation,
An investigation support method characterized in that the step of extracting differences and the step of searching for sub-citations are repeated. In the research support method according to any one of claims 8 to 10 ,
The display control step includes:
An investigation support method comprising: generating a display screen that has been subjected to screen processing that makes it possible to determine the correspondence between the sub-citation and the difference, and making the display screen displayable. A computer program for research support, which causes a computer to execute the research support method according to any one of claims 1 to 11 . A research support system for supporting literature research, comprising:
an input unit that obtains information regarding points of agreement between the target sentence and the reference sentence;
a determination unit that inputs the target sentence and the reference sentence to a language understanding model that determines matching points for each token, and obtains information regarding the matching points;
An investigation support system comprising: a display control unit that generates a display screen that has been subjected to screen processing that allows the matching points to be determined, and that enables the display screen to be displayed.

Images