JP2022006237A - Natural language processing system and natural language processing method - Google Patents

Abstract

To efficiently improve an accuracy of natural language processing such as machine translation.SOLUTION: A natural language processing system generates a synonymous expression of a sentence of a single language corpus using a synonymous expression generation model, calculates an evaluation value of an ease of natural language processing for a sentence and the synonymous expression of the sentence, generates the synonymous expression generation model using synonymous expression information including the sentence, the synonymous expression of the sentence, and the evaluation value as learning data, learns the synonymous expression generation model such that an evaluation value of the synonymous expression to be translated more easily becomes high and an evaluation value of the synonymous expression difficult to be translated becomes low, generates an adaptive destination bilingual corpus by collecting a pair of a synonymous expression easy to be machine-translated with an adaptive source translation model and a translation of the synonymous expression and a pair of a synonymous expression difficult to be machine-translated with the adaptive source translation model and a reference translation for a sentence in an adaptive source bilingual corpus which is an origin of the synonymous expression difficult to be machine-translated with the adaptive source translation model, and learns an adaptive destination translation model using the generated adaptive destination bilingual corpus.SELECTED DRAWING: Figure 1

Description

The present invention relates to a natural language processing system and a natural language processing method.

Patent Document 1 describes an encoder / decoder-type neural machine translation technique for enhancing an encoder by using a monolingual corpus of a target language in order to improve the accuracy of the entire translator.

Further, Non-Patent Document 1 describes a technique for automatically searching a part related to the prediction of an object from the original text.

In addition, Non-Patent Document 2 describes a technique for training both systems of neural machine translation (NMT) and statistical machine translation (SMT) using only a monolingual corpus. Has been done.

Japanese Unexamined Patent Publication No. 2019-153023

Bahdanau, Dzmitry, Kyunghyun Cho, Yoshua Bengio, "Neural machine translation by jointly learning to align and translate.", ArXiv preprint arXiv: 1409.0473 (2014)., [Online], [Search June 1, 2nd year], Internet <URL: https://arxiv.org/pdf/1409.0473.pdf> Mikel, et al. An effective approach to unsupervised machine translation. ArXiv preprint arXiv: 1902.01313, 2019., [Online], [Search June 1, 2nd year of Reiwa], Internet <URL: https://arxiv.org/ pdf / 1902.01313.pdf ＞

In recent years, with the advent of NMT, translation accuracy has improved significantly compared to rule-based machine translation (RMT) and SMT, and practical accuracy is particularly high in domains with sufficient bilingual corpora. Translation is now possible. However, it is difficult to apply NMT in a situation where there is not enough bilingual corpus necessary for learning a translation model.

When sufficient learning resources cannot be secured in a certain domain (hereinafter referred to as "adaptation destination domain"), the knowledge gained in another resource-rich domain (hereinafter referred to as "adaptation source domain") is applied to the adaptation destination. It is known that high translation accuracy can be achieved in the target domain by transferring to a domain. For example, if a translation corpus of the target domain exists even in a small amount, the translation model is pre-learned by the translation corpus of the adaptation source domain, and then fine tuning is performed in the adaptation destination domain. A method has been proposed. In addition, when only a single language corpus of the adaptation destination domain can be used, a pseudo bilingual corpus can be generated by using reverse translation. In reverse translation, two translation models with opposite processing directions are prepared (for example, English-Japanese translation model and Japanese-English translation model), and the target sentence generated by one translation model is the source sentence of the other translation model. (For example, refer to Patent Document 1 and Non-Patent Document 2).
Even in a situation where only a monolingual corpus can be used, by repeatedly adding a bilingual corpus simulated by the monolingual corpus of the adaptation destination domain to the bilingual corpus of the adaptation source domain and re-learning using reverse translation. NMT learning is possible.

By the way, in order to improve the accuracy of the translation model in the adaptation destination domain, a domain having a field close to that of the adaptation destination domain is usually selected as the application source domain. However, due to the nature of the domain, the writing style of the statement of the application source domain and the statement of the application destination domain are often different. For example, in the field related to patents, the patent specification of the application source domain (patent gazette, patent publication, etc.) is used to learn a translation model for translating the notice of reasons for refusal of the application domain for which a sufficient bilingual corpus is not prepared. Consider the case of using as. In this case, for example, in the notice of reasons for refusal, expressions such as "please refer to ..." regarding the request or question from the examiner are often used, but such expressions are rarely used in the patent specification. Therefore, even if the translation model learned by using the patent specification as the application source domain is used as it is, sufficient translation accuracy cannot be obtained.

The present invention has been made based on such a background, and provides a natural language processing system and a natural language processing method capable of efficiently improving the accuracy of natural language processing such as machine translation. The purpose.

One of the present inventions for achieving the above object is a natural language processing system, which is configured by using an information processing apparatus, and has a storage unit for storing a monolingual corpus and a synonymous expression generation model, and the monolingual. Calculates the evaluation value of the ease of natural language processing for each of the synonymous expression generation unit that generates synonymous expressions of the corpus sentence using the synonymous expression generation model and the synonymous expressions generated for the sentence and the sentence. The evaluation unit, the synonymous expression generated from the sentence, and the synonymous expression information including the evaluation value are used as learning data, and the synonymous expression that is easy to process in natural language is generated from the input sentence. It includes a synonymous expression generation model learning unit that generates the synonymous expression generation model.

In addition, the problems disclosed in the present application and the solutions thereof will be clarified by the column of the form for carrying out the invention and the drawings.

According to the present invention, the accuracy of natural language processing such as machine translation can be efficiently improved.

It is a system flow diagram explaining the function of a machine translation system. This is a configuration example of an information processing device constituting an information processing system. It is a flowchart explaining the domain adaptive learning process. It is a figure explaining the preprocessing of a noise reduction process. It is a flowchart explaining a translation process. It is a figure which shows the translation example by the conventional machine translation technique. It is a figure which shows the translation example by the conventional machine translation technique. It is a figure which shows the translation example by a machine translation system. It is a figure which shows the translation example by a machine translation system. It is a figure which shows the translation example by a machine translation system. It is a figure which shows the translation example by a machine translation system. It is a system flow diagram of the search system of 2nd Embodiment. It is a flowchart explaining the synonymous expression generation model generation process. It is a flowchart explaining the search process.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and are appropriately omitted or simplified for the sake of clarification of the description. The present invention can also be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[First Embodiment]
FIG. 1 shows a system flow diagram of the machine translation system 1 described as the first embodiment. The machine translation system 1 is a bilingual corpus (hereinafter, "adaptation destination") used for learning a translation model (hereinafter, "adaptation destination translation model") in a target domain (hereinafter, "adaptation destination domain"). It is an information processing system that generates a "translation corpus". The machine translation system 1 is a translation model trained in a resource-rich domain (hereinafter referred to as "adaptation source domain") as a means for realizing generation of an adaptation destination translation corpus (hereinafter, "adaptation source translation model"). The translation result of the expression peculiar to the target domain is obtained under the restriction that only a single language corpus of the source language (hereinafter referred to as "source language") in the target domain is available. Acquire.

Generally, for expressions specific to the adaptation destination domain, there are synonymous expressions corresponding to the adaptation source domain. For example, in the field related to patents, the patent specification of the applicable source domain (patent gazette, patent publication) for learning the applicable destination translation model for translating the notice of reasons for refusal of the applicable domain for which a sufficient bilingual corpus is not prepared. Consider the case of using it as a gazette etc.). In this case, for example, with respect to the command tone expression "please refer to" in the adaptation destination domain, there is a synonymous expression "reference" in the adaptation source domain. In addition, regarding the questionable expression of "what does it mean" in the adaptation destination domain, there is a synonymous expression of "what does it mean" in the adaptation source domain. Here, since the expression in the adaptation source domain is a familiar expression for the adaptation source translation model, it can be translated accurately. Therefore, in order to accurately translate the text of the notice of reasons for refusal of the applicable domain, it is considered effective to use synonymous expressions in the patent specification of the applicable domain. From this point of view, the machine translation system 1 of the present embodiment has a function (stylistic conversion function) of associating the expression of the adaptation destination domain with the expression of the adaptation source domain.

The machine translation system 1 acquires a bilingual pair by converting a sentence into a synonymous expression according to the ease of translation. The machine translation system 1 automatically generates an adaptive translation corpus to be used for learning an adaptive translation model from two methods having different logics.

In the first method (hereinafter referred to as "easy example generation"), the machine translation system 1 uses an adaptation source translation model to express an expression that is difficult to translate (cannot be correctly translated) in the adaptation destination domain. Convert to synonymous expressions that are easy to translate. By performing translation processing using the converted synonymous expressions, there is a high possibility that correct translation results will be obtained. Further, the machine translation system 1 collects a pair of translation results obtained through an expression of an adaptation destination domain that is difficult to translate and a synonymous expression of the expression, and collects a pair of adaptation destination translation corpus (hereinafter referred to as "easy example"). .) Generated as.

In the second method (hereinafter referred to as "hard example generation"), the machine translation system 1 converts an easily translatable expression of the adaptation source domain into a difficult-to-translate expression of the adaptation destination domain. Then, a collection of pairs of difficult-to-translate expressions of the adaptation destination domain and reference translations corresponding to the sentences of the adaptation source domain is generated as an adaptation destination bilingual corpus (hereinafter referred to as "hard example"). In this way, if the adaptation destination translation model is trained based on the expressions that are difficult to translate and the reference translation obtained in the above procedure, how can the expressions of the adaptation destination domain that are difficult to translate in the adaptation destination domain be translated into the target language (? Hereinafter, it is possible to learn whether to translate into a "target language").

As shown in FIG. 1, the machine translation system 1 includes functions of a domain adaptation learning unit 10, a data expansion unit 20, an adaptation destination translation model learning unit 30, and a translation processing unit 40. Of these, the domain adaptive learning unit 10 includes a data input processing unit 11, a data storage unit 12, an adaptive source translation model learning unit 13, an adaptive source translation model storage unit 14, a domain adaptive model learning unit 15, and a domain adaptive model storage unit. Includes 16. Further, the data input processing unit 11 includes a user interface 111 and a data processing unit 112. Further, the domain adaptation model learning unit 15 includes a single language corpus input unit 151, a synonymous expression generation unit 152, a reward output unit 153, a synonymous expression information storage unit 154, and a synonymous expression generation model learning unit 155. Further, the data expansion unit 20 includes an adaptation destination translation corpus generation unit 21 and an adaptation destination translation corpus selection unit 22. Further, the adaptation destination translation model learning unit 30 includes an adaptation destination translation model learning unit 31, an adaptation destination translation model learning unit 32, and an adaptation destination translation model storage unit 33. Further, the translation processing unit 40 includes an input interface 41, a translation unit 42, and an output interface 43. Details of each of these functions will be described later.

FIG. 2 shows an example of the hardware configuration of the information processing apparatus 100 constituting the machine translation system 1. As shown in the figure, the information processing device 100 includes a processor 101, a main storage device 102, a communication device 103, an input device 104, an output device 105, and an auxiliary storage device 106.

The processor 101 is, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an AI (Artificial Intelligence) chip, an FPGA (Field Programmable Gate Array), a SoC (System on Chip), and an ASIC. (Application Specific Integrated Circuit) etc. are used.

The main storage device 102 is a device for storing programs and data, and is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile memory (NVRAM (Non Volatile RAM)), and the like.

The communication device 103 is a device that communicates with other information processing devices such as a user terminal via a communication network, a communication cable, or the like, and is a wireless or wired communication module (wireless communication module, communication network adapter, USB). Module etc.).

The input device 104 and the output device 105 form a user interface unit 3 of the machine translation system 1. The input device 104 is a user interface that accepts user input and data input from the outside, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a voice input device (for example, a microphone), and the like. The output device 105 is a user interface that outputs various information to the user, and is a display device (liquid crystal display, organic EL panel, etc.) that displays various information, and a voice output device (for example, a speaker) that outputs various information by voice. ), Printers that print on paper media, etc.

The auxiliary storage device 106 is a device for storing programs and data, for example, an SSD (Solid State Drive), a hard disk drive, and an optical storage medium (CD (Compact Disc)).
), DVD (Digital Versatile Disc), etc.), IC card, SD card, etc. The auxiliary storage device 106 stores programs and data for realizing the functions of the machine translation system 1. The auxiliary storage device 106 can write / read programs and data via the reading device of the recording medium and the communication device 103.

Programs and data stored (stored) in the auxiliary storage device 106 are read out to the main storage device 102 at any time. Each function included in the machine translation system 1 is obtained by the processor 101 reading and executing a program stored in the main storage device 102, and each storage unit (12, 14, 154, 16, 16; 31 and 33) are realized by readable storage of predetermined data in the auxiliary storage device 106.

All or part of the functions of the machine translation system 1 can be combined with other arithmetic units (for example, FPGA (Field Programable Gate Array) or ASIC (Application Specific Integrated Circuit)).
It may be realized by hardware such as.

The information processing device 100 is, for example, a personal computer (desktop type or notebook type), a smartphone, a tablet, a general-purpose machine, or the like. All or part of the information processing apparatus 100 may be realized by using virtual information processing resources such as a cloud server provided by a cloud system.

FIG. 3 is a flowchart illustrating a process performed by the domain adaptive learning unit 10 shown in FIG. 1 (hereinafter, referred to as “domain adaptive learning process S300”). Hereinafter, the domain adaptive learning process S300 will be described with reference to FIG. 1 as appropriate.

First, the user interface 111 of the data input processing unit 11 of the domain adaptation learning unit 10 is subjected to a monolingual corpus of the adaptation source bilingual corpus and the adaptation destination domain from a user such as a machine translation system administrator (hereinafter, "adaptation destination unit post-corpus corpus"). ”) Accepts the input of each location information. Then, the user interface 111 acquires the adaptation source bilingual corpus and the adaptation destination monolingual corpus from the data storage area specified by the received location information (s301), and data the acquired adaptation source bilingual corpus and the adaptation destination monolingual corpus. Output to the processing unit 112.

The location information may be information that specifies a storage area of an information processing device constituting the machine translation system 1, such as a folder name or a directory name, or may be used in a communication network such as the Internet such as a URL (Uniform Resource Locater). It may be information indicating the location of the database to be connected. The user interface 111 is not limited to the input device 104 (keyboard or the like) included in the information processing device constituting the machine translation system 1, and may be, for example, a communication device 103. If the communication device 103 is the user interface 111, the user interface 111 receives the location information input to the user's information processing device (personal computer or the like) via the communication network.

Subsequently, the data processing unit 112 reads the input adaptation source bilingual corpus and adaptation destination single language corpus, converts the sentences contained therein into data in a format that can be machine translated by the translation processing unit 40, and after conversion. Data is stored in the data storage unit 12 (s302). The data converted by the data processing unit 112 is composed of a symbol sequence such as a word string described as, for example, x ₁ , x ₂ , ..., X _S and the like. In the above symbol series, the final symbol described as "x _S " is a terminal symbol (hereinafter referred to as "EOS"), and the number of symbols included in the series is S including EOS.

Subsequently, the adaptation source translation model learning unit 13 generates and learns the adaptation source translation model (s303). The adaptation source translation model learning unit 13 executes an encoder / decoder type neural machine translation. The adaptation source translation model learning unit 13 operates in a mode for performing machine learning (hereinafter referred to as "learning mode") or a mode for performing machine translation (hereinafter referred to as "machine translation mode"). In the learning mode, the adaptation source translation model learning unit 13 generates an adaptation source translation model using the adaptation source translation corpus read from the data storage unit 12 as learning data, and among the generated adaptation source translation models, the one with the highest accuracy. Is stored in the adaptation source translation model storage unit 14. In the machine translation mode, the adaptation source translation model learning unit 13 converts the input sentence described in the source language into the target language using the adaptation source translation model. As will be described later, in the machine translation mode, the adaptation source translation model learning unit 13 is output from the domain adaptation model learning unit 15, and the adaptation destination sentence x _n (n = 1) consisting of a symbol sequence whose final symbol is EOS. , 2, ..., s) and the synonymous expression y _n ^m (m = 1,2, ..., b, n = 1,2, ..., T) of the sentence are received as input data, and the received input data x Perform machine translation on _n , y _n ^m .
Then, the adaptation source translation model learning unit 13 transfers the output data y ₁ , y ₂ , ..., Y _T consisting of T symbol sequences whose final symbol is EOS to the domain adaptation model learning unit 15 as a result of machine translation. give feedback.

Subsequently, the domain adaptation model learning unit 15 generates a learning model (hereinafter referred to as "synonymous expression generation model") that generates synonymous expressions of the adaptation destination sentences so that the adaptation destination sentences in the monolingual corpus can be appropriately translated. ) Is generated, and the adaptation source translation model is trained so that the adaptation destination sentence is appropriately translated (loop processing of s304 and s305 to s318).

Specifically, first, in preparation for the loop processes s305 to s318, the single language corpus input unit 151 of the domain adaptation model learning unit 15 reads the adaptation destination single language corpus from the data storage unit 12, and synonymous with the adaptation destination single language corpus. Output to the expression generation unit 152. Further, the synonymous expression generation unit 152 reads the input adaptation destination monolingual corpus and the synonymous expression generation model stored in the domain adaptation model storage unit 16 (s304).

In the loop processes s305 to s318, the synonymous expression generation unit 152 first executes the encoder / decoder method neural statement generation process (s305). Specifically, in the mode in which the synonymous expression generation unit 152 generates a sentence using the learned synonymous expression generation model (hereinafter referred to as "sentence generation mode"), one sentence in the read single language corpus is used. After encoding x ₁ , x ₂ … x _S , beam search decoding is executed, and b output data y ¹ , y ² ,
…, Generate y ^b . It should be noted that each of the b output data y ¹ , y ² , ..., Y ^b is a symbol series consisting of T words with EOS as the final symbol. For example, if y ¹ , y ¹ = y ₁ ¹ , y ₂ ¹ , ..., y _T ¹ . b symbol sequences generated by executing beam search decoding y ¹ , y ² ,
Each of ..., y ^b is a synonym for one sentence x ₁ , x ₂ ... x _S in a monolingual corpus.

Subsequently, the synonymous expression generation unit 152 removes noise, which is an inappropriate expression, from the ^b synonymous expressions y ¹ , y ² , ..., Y b (s306). The similarity calculation used for noise removal can be performed based on evaluation indexes such as BLEU and RIBES. The synonym expression generation unit 152 performs preprocessing for suppressing an excessive penalty due to paraphrase of synonyms, and then performs similarity calculation for removing noise. In the above preprocessing, the synonymous expression generation unit 152 is generated by the original adaptation destination sentence (hereinafter, may be referred to as “original sentence”) included in the single language corpus and beam search decoding. A common character string is extracted from each of the b synonymous expressions, and the extracted common character string is replaced with a certain variable name. When extracting a common character string, for example, the longest match is used as a reference. Then, after all the parts that satisfy the criteria are replaced with the variable names, the part between the two variables before and after is regarded as a synonym.

FIG. 4 shows an example of preprocessing. As shown in the figure, four sentences having the same meaning but different notations "Refer to the materials disclosed in the prior art documents 6 and 7 of the specification.", "Prior art documents 6 and 7 of the specification. ”,“ Refer to the materials disclosed in the prior art documents 6 and 7 of the specification. ”,“ The materials disclosed in the prior art document 6 of the specification should be referred to. If there is ".", The synonymous expression generation unit 152 extracts "disclosed in Prior Art Documents 6" and "7" and "reference materials" as common character strings from these sentences. .. Then, the variable name of "Prior Art Document 6 of the specification" is set to "ele.1", "disclosed in 7" is set to "ele.2", "reference to material" is set to "ele.3", and the kuten " When "." Is "ele.4", the synonym expression generation unit 152 considers "to", "and", and "and" between each variable as synonyms. By performing such preprocessing, the synonym expression generation unit 152 prevents the synonyms identified by the procedure shown in FIG. 4 from being counted as different character strings in the similarity calculation used for noise reduction.

The synonymous expression generation unit 152 calculates the similarity between each synonymous expression and the original sentence in order to remove noise when the preprocessing is executed, and the finally calculated similarity is a predetermined index. A synonymous expression having a predetermined value or less (for example, BLUE value ≤ 0.5) in (for example, BLUE value) is regarded as noise and removed. Further, the synonymous expression generation unit 152 sets a set of one adaptation destination sentence and b synonymous expressions acquired by beam search for the synonymous expression after noise reduction, in a reward output unit 153. Send to. It is assumed that there are N sentences to which the application is made. That is, it is assumed that there are N sets of learning data for performing stylistic conversion.

Returning to FIG. 3, the reward output unit 153 subsequently uses the original sentence x _n and the synonymous expression y _n ^m (m = 1, 2, ..., S) of the sentence x _n (n = 1, 2, ..., S). ..., b, n = 1, 2, ..., T) and the ease of machine translation are evaluated. First, the reward output unit 153 reads the learned adaptation source translation model generated by the adaptation source translation model learning unit 13 from the adaptation source translation model storage unit 14 (s307). Then, the reward output unit 153 ^learns the adaptation source translation model by reading the adaptation source sentence x _n sent from the synonymous expression generation unit 152 and the synonymous expression y _{n m} . Let Part 13 perform machine translation, and calculate the ease of machine translation (reward) for the sentence x _n to which it is applied and the synonymous expression y _n ^m (s308).

Here, the ease of processing may be evaluated in terms of the length of the sentence, the dependency structure of the sentence, etc., but in the machine translation system 1, the log-likelihood logp ( y _n ^m ) is used. The log-likelihood logp (y _n ^m ) is required when translating the target synonymous expression into the target language based on the adaptation source translation model. The log-likelihood logp (y _n ^m ) is a value indicating the reliability of the adaptation source translation model and is a value indicating the accuracy of the translation result. The log-likelihood logp (y _n ^m ) increases when the translation result is generated correctly and decreases when the translation result is generated incorrectly. The reward output unit 153 consists of the sentence x _n of the adaptation destination, its synonymous expression y _n ^m , and the evaluation value logp (y _n ^m ) of the ease of machine translation obtained by the processing of the reward output unit 153. A set of three pieces of data (hereinafter referred to as "synonymous expression information") is stored in the synonymous expression information storage unit 154 (s309).

Subsequently, the synonymous expression generation model learning unit 155 generates a synonymous expression generation model (s310). The synonymous expression generation model learning unit 155 is a function of executing the encoder / decoder type neural sentence generation processing. In the learning mode, the synonymous expression information is extracted from the synonymous expression information storage unit 154, and the extracted synonymous expression information is used. In order to increase the likelihood of synonymous expressions that are easier to translate as learning data and reduce the likelihood of synonymous expressions that are difficult to translate, the synonymous expression y _n ^m of that sentence x _n is derived from the sentence x _n to which it is applied. Learn to generate. That is, the above-mentioned methods of "easy exampl generation" and "hard example generation" are applied to the learning.

Here, the probability that the synonymous expression y _n ^m is generated follows the ease of processing logp (y ^m ) for the synonymous expression. Therefore, in the learning mode, the synonymous expression generation model learning unit 155 first uses the evaluation value logp (x _n ) of the reward calculated from the sentence x _n of the adaptation destination as a reference, as shown in the following formula 1, and uses the reference. And the difference r from the evaluation value logp (y _n ^m ) of the reward of the synonymous expression y _n ^m is obtained.
[Formula 1]

The synonymous expression generation model generated by "easy exampl generation" is a model for converting the sentence x _n of the adaptation destination into a synonymous expression that can be processed by the adaptation source translation model. Therefore, "easy
"Examppl generation" has a higher likelihood when it is easier to translate than the original sentence, and a lower likelihood when it is more difficult to translate than the original sentence. Therefore, the synonymous expression generation model learning unit 155 generates a synonymous expression generation model so as to minimize the loss function L _e shown in the following mathematical expression 2.
[Formula 2]

On the other hand, the synonymous expression generation model generated by "hard example generation" is a model for converting the sentence x _n to which it is applied so that it is difficult to translate. Therefore, the synonymous expression generation model learning unit 155 generates a synonymous expression generation model so as to minimize the loss function L _h shown in the following mathematical formula 3 when performing “hard example generation”.
[Formula 3]

The synonymous expression generation model learning unit 155 stores the synonymous expression generation model generated as described above in the domain adaptive model storage unit 16 (s311). Then, the synonymous expression generation model learning unit 155 updates the synonymous expression generation model at each execution opportunity of the learning mode.

The synonymous expression generation model may be generated by, for example, pre-learning by the synonymous expression generation model learning unit 155. In that case, the synonymous expression generation unit 152 can use the pre-learned synonymous expression generation model stored in the domain adaptive model storage unit 16 when the synonymous expression generation model is first read in the sentence generation mode. .. Pre-learning by the synonymous expression generation model learning unit 155 is performed, for example, as follows.

First, the synonymous expression generation model learning unit 155 reads the monolingual corpus of the adaptation destination domain from the monolingual corpus input unit 151, and learns an encoder / decoder method neural model so as to restore the adaptation destination sentence x _n . Since the synonymous expression generation model pre-learned by such a procedure can generate sentences in the same language as the single language corpus to which it is applied, multiple synonymous expressions should be generated by performing beam search decoding. Is possible. The synonymous expression generation model learning unit 155 stores the pre-learned synonymous expression generation model in the domain adaptive model storage unit 16.

As described above, the synonymous expression generation model is trained for each pre-learning and subsequent learning opportunities. The synonymous expression generation unit 152 reads the synonymous expression generation model stored in the domain adaptation model storage unit 16 at each learning opportunity by the synonymous expression generation model learning unit 155, and also reads the synonymous expression generation model stored in the domain adaptation model storage unit 16 and adapts the target domain from the single language corpus input unit 151. Read the monolingual corpus of (s312). Then, the synonymous expression generation unit 152 has b output data y ¹ , y ² , ..., Depending on the sentence generation mode, as in the process of generating the synonymous expression (s305) and the process of removing noise (s306) described above. While y ^b is generated as a synonymous expression (s313), the noise of the generated synonymous expression is removed after performing the above-mentioned preprocessing (s314), and the synonymous expression after noise reduction and the original sentence thereof are rewarded. It is transmitted to the output unit 153.

When the reward output unit 153 receives the synonymous expression transmitted from the synonymous expression generation unit 152 and the sentence that is the source thereof, the reward output unit 153 applies the synonymous expression to the adaptive source translation as in the case of the above-mentioned ease of translation (reward). The model learning unit 13 translates it into the target language, outputs a reward (log-likelihood) based on the translation result, and sets up a synonymous expression, the translation result, the log-likelihood, and the original sentence (hereinafter, "synonymous"). (Representation-related information) ”is transmitted to the bilingual logarithm extraction unit 156 (s315).

The bilingual pair extraction unit 156 executes a process of extracting the bilingual pair based on the synonymous expression-related information transmitted by the synonymous expression generation unit 152 (s316). In this process, the bilingual pair extraction unit 156 first sorts the synonymous expression-related information received from the synonymous expression generation unit 152 in order from the one with the largest log-likelihood, and the log-likelihood is higher r% according to the number of learnings. Extract the pairs contained in.
Note that r is reduced as the number of learnings increases. By extracting the translation pair in this way, the translation pair extraction unit 156 reduces the possibility that the translation result including the noise generated by the reward output unit 153 adversely affects the learning. That is, the bilingual pair extraction unit 156 fine-tunes the synonymous expression generation model using the relatively high-quality learning data after the synonymous expression generation unit 152 pre-learns the learning data in a noisy state. Therefore, a synonymous expression with higher translation accuracy can be generated than when a synonymous expression generation model is randomly given to the synonymous expression generation unit 152.

The parallel translation pair extraction unit 156 extracts the original sentence and the translation result from the extracted synonymous expression-related information, generates a pair as learning data, and uses this learning data in the adaptation source translation model learning unit 13. Send. The adaptation source translation model learning unit 13 learns the adaptation source translation model from the learning data transmitted from the parallel translation pair extraction unit 156 (s317), and stores the learned adaptation source translation model in the adaptation source translation model storage unit 14. (S318).

FIG. 5 is a flowchart illustrating processing performed by the data expansion unit 20, the adaptation destination translation model learning unit 30, and the translation processing unit 40 (hereinafter, referred to as “translation processing S500”). Hereinafter, the translation process S500 will be described with reference to FIG. 1 as appropriate.

The adaptation destination translation corpus generation unit 21 of the data expansion unit 20 pseudo-generates a translation translation corpus for learning so that the adaptation source translation model learning unit 13 can translate the sentence of the adaptation destination domain by the adaptation source translation model. .. The adaptation destination translation corpus generation unit 21 reads the monolingual corpus of the adaptation destination domain from the data storage unit 12, reads the learned adaptation source translation model from the adaptation source translation model storage unit 14, and learns from the domain adaptation model storage unit 16. Read the completed synonymous expression generation model (s501).

When generating a pseudo translation corpus, the adaptation destination translation corpus generation unit 21 first generates synonymous expressions for sentences contained in the monolingual corpus of the adaptation destination domain using a synonymous expression generation model (s502). ). Further, the adaptation destination translation corpus generation unit 21 translates the generated synonymous expression into the target language by the learned adaptation source translation model. Then, the adaptation destination translation corpus generation unit 21 acquires a set of three data consisting of the sentence of the original adaptation destination domain, the translation result obtained through the synonymous expression, and the likelihood output by the translation model. (S503).

The adaptation destination translation corpus selection unit 22 of the data expansion unit 20 sorts the set of the above sets acquired by the adaptation destination translation corpus generation unit 21 in the order of the likelihood output based on the translation model (s504). Then, the adaptation destination translation corpus selection unit 22 uses the data of the upper r% in the rearranged order to learn a translation model (adaptation destination translation model) for translating the sentence of the adaptation destination domain. (Adaptation destination translation corpus) is extracted, and the extracted adaptation destination translation corpus is stored in the adaptation destination translation corpus storage unit 31 of the adaptation destination translation model learning unit 30 (s505).

The adaptation destination translation model learning unit 32 of the adaptation destination translation model learning unit 30 generates a trained adaptation destination translation model. The adaptive destination translation model learning unit 32 is a function of executing an encoder / decoder type neural machine translation. The adaptation destination translation model learning unit 32 reads the adaptation destination translation corpus from the adaptation destination translation corpus storage unit 31, generates a trained adaptation destination translation model using this adaptation destination translation corpus as training data (s506), and generates an adaptation destination translation model (s506). The translation model is stored in the adaptation destination translation model storage unit 33 (s507). As the adaptation destination translation model, other translation models such as a rule-based format and a statistical machine translation format may be used.

Subsequently, the translation processing unit 40 translates the sentence of the adaptation destination domain received from the user using the adaptation destination translation model generated by the adaptation destination translation model learning unit 30, and outputs the translation result. Specifically, first, the input interface 41 accepts a sentence input from the user of the machine translation system 1 (s508). Subsequently, the translation unit 42 reads the translation model from the adaptation destination translation model storage unit 33, and translates the received sentence into the target language (s509). Subsequently, the output interface 43 outputs the translated text to an output device 105 such as a display (s510).

As described above, according to the machine translation system 1 of the present embodiment, the source in the adaptation destination domain utilizes the fact that the synonymous expression corresponding to the expression peculiar to the adaptation destination domain exists in the adaptation destination domain. Sentences in the target domain can be translated accurately under the constraint that only a monolingual corpus of languages is available. As described above, according to the machine translation system 1 of the present embodiment, the accuracy of natural language processing can be efficiently improved.

As an example of translation processing by the machine translation system 1, Japanese is used as the source language, English is used as the target language, the document of the source domain is used as the patent gazette, and the document of the destination domain is used as the notice of reasons for refusal. An example of translating a Japanese notice of reasons for refusal into English is shown. It is assumed that the bilingual corpus of the patent gazette and the single language corpus of the notice of reasons for refusal in Japanese are stored in the data storage unit 12 via the data input processing unit 11.

6A and 6B are translation examples (comparative examples) by conventional machine translation technology, respectively, in which the text of the notice of reasons for refusal in Japanese and the text of the patent gazette are input texts, and the input texts are the patent gazettes. An example of translation into English using a translation model learned in the domain of is shown. In the notice of reasons for refusal shown in FIG. 6A, the description part (underlined part in the figure) regarding the request or question from the examiner shown by the underline in the figure is the expression in the patent gazette shown in FIG. 6B (in the figure, It is different from the underlined part). As described above, simply applying the translation model learned in the domain of the patent gazette to the notice of reasons for refusal cannot appropriately translate the expression peculiar to the notice of reasons for refusal.

7A-7D show an example in which the Japanese (source domain) sentence of the notice of reasons for refusal is translated into the English (application domain) sentence by the application destination translation model of the machine translation system 1 of the present embodiment (same as above). The input sentence and the translated sentence shown in the figure) are shown. The synonymous expressions shown in each figure are examples of synonymous expressions used to generate learning data (adapted target translation corpus) used for learning the adaptive destination translation model. As described above, according to the machine translation system 1 of the present embodiment, the sentence of the notice of reasons for refusal can be accurately translated into an English sentence.

[Second Embodiment]
FIG. 8 is a system flow diagram illustrating the functions of the information processing system (hereinafter, referred to as “search system 50”) shown as the second embodiment. The hardware configuration of the search system 50 is the same as that of the information processing apparatus 100 shown in FIG.

As shown in the figure, the search system 50 includes a domain adaptive learning unit 60 and a search processing unit 70. Further, the domain adaptive learning unit 60 includes a data input processing unit 61, a search engine storage unit 62, a storage unit 63, a domain adaptive model learning unit 64, and a domain adaptive model storage unit 65.
including. Further, the domain adaptation model learning unit 64 includes a query input unit 641, a synonymous expression generation unit 642, a reward output unit 643, a synonymous expression information storage unit 644, and a synonymous expression generation model learning unit 645. Further, the search processing unit 70 includes an input interface 71, a domain adaptation determination unit 72, a query synonymous expression generation unit 73, a search engine unit 74, and an output interface 75. Further, the search engine unit 74 includes a query input unit 741, a search content storage unit 742, a similarity calculation unit 743, and a search result output unit. The similarity calculation unit 743 is, for example, a search engine generally used in an information retrieval site on the Internet.

The search processing unit 70 provides an information search service to a user, similar to an information processing system constituting an information search site open to the Internet, a stand-alone database search device, and the like. Specifically, when the search processing unit 70 receives an input of a query from a user via a communication network such as the Internet, the user receives the content stored in the search content storage unit 742 as a search result for the received query. Send to the user terminal to operate. Further, when the search processing unit 70 receives a query composed of a single sentence or a combination of a plurality of sentences from the user, the search processing unit 70 provides more relevant and appropriate content. Then, the domain adaptive learning unit 60 uses a single language corpus as a collection of queries input by the user into the search processing unit 70 (hereinafter referred to as "query corpus"), and a query received from the user based on the query corpus. Generates a synonymous expression generation model that transforms the correct content into a synonymous expression that is easy to extract.

FIG. 9 is a flowchart illustrating a process performed by the domain adaptive learning unit 60 (hereinafter, referred to as “synonymous expression generation model generation process S900”). Hereinafter, the synonymous expression generation model generation process S900 will be described with reference to FIG. 8 as appropriate.

First, when the user interface 611 of the data input processing unit 61 inputs information regarding the location of the single language corpus to which the data is applied by inputting a user's operation or the like, the query corpus read from the input location information is transmitted to the data processing unit 612. Output. The data processing unit 612 converts the input sentence in the query corpus into data in a format that can be processed by the domain adaptation model learning unit 64, and stores the converted data in the storage unit 63. Subsequently, the query input unit 641 reads the query corpus from the storage unit 63, and outputs the read query corpus to the synonymous expression generation unit 642 (s901). As for the data converted by the data processing unit 612, as in the machine translation system 1 of the first embodiment, one sentence describing the query is a symbol sequence such as a word string x ₁ , x ₂ , ..., It has been converted to x _S.

Subsequently, the domain adaptive learning unit 60 starts the loop processes s901 to s910, and first, the synonymous expression generation unit 642 executes the encoder / decoder type neural statement generation process (s902). The synonymous expression generation unit 642 operates in the same manner as the synonymous expression generation unit 152 of the machine translation system 1 of the first embodiment, and in the sentence generation mode, one sentence x ₁ , x ₂ in the query corpus read from the storage unit 63. After encoding ,…, x _s , beam search decoding is executed to generate b output data y ¹ , y ² ,… y ^b , which are synonymous expressions. It should be noted that each of the b output data y ¹ , y ² , ... y ^b is a symbol series consisting of T words with EOS as the final symbol.

Subsequently, the synonymous expression generation unit 642 generated b synonymous expressions based on the degree of similarity between the generated synonymous expressions y ¹ , y ² , ..., y ^b and the query x ₁ , x ₂ , ..., x _s . Noise reduction processing is performed to remove inappropriate expressions from y ¹ , y ² , ..., y ^b (s903). At that time, the synonymous expression generation unit 642 performs preprocessing on the generated synonymous expression as in the machine translation system 1 of the first embodiment, and between each synonymous expression and the original sentence. The similarity is obtained, and the synonymous expression in which the obtained similarity is finally equal to or less than a predetermined value (for example, BLUE value ≤ 0.5) in a predetermined index (for example, BLUE value) is regarded as noise and removed. The synonymous expression generation unit 642 transmits the synonymous expression after noise reduction and the query that is the source thereof to the reward output unit 643 (s904).

Subsequently, the reward output unit 643 reads the existing search engine from the search engine storage unit 62 (s905), and the query x _n (n = 1, 2, ..., S) is expressed as a synonym for it y _n ^m (m = 1). , 2
, ..., b, n = 1, 2, ..., T), and the ease of processing when inquiring the synonymous expression y _n ^m to the search engine is calculated (s906). The reward output unit 643 also requests the ease of processing for the query x _n . In the search system 50 of the second embodiment, as an index of ease of processing (reward), for example, a query x _n such as TF / IDF and all stored in the search content storage unit 742 as a search target. The entropy H (y _n ^m ) calculated from the degree of relevance to the content of is used. The higher the entropy value, the better the search engine will extract the content for the query x _n .

Subsequently, the reward output unit 643 is composed of a set of a query x _n , a synonymous expression expression y _n ^m , and an evaluation value H (y _n ^m ) for ease of processing for one query x _n . The synonymous expression information is stored in the synonymous expression information storage unit 644 (s907). In this example, it is assumed that there are N sets of synonymous expression information. The synonymous expression information is used as learning data for the synonymous expression generation model learning unit 645 to generate a synonymous expression generation model.

Subsequently, the synonymous expression generation model learning unit 645 reads the synonymous expression generation model (s908). The synonymous expression generation model learning unit 645 is a function of executing the neural sentence generation process of the encoder / decoder method, and reads the synonymous expression generation model from the domain adaptive model storage unit 16 in the learning mode.

Subsequently, the synonymous expression generation model learning unit 645 learns to generate the synonymous expression y _n ^m from the query x _n using the synonymous expression information as learning data, and generates a synonymous expression generation model which is a learning result. (S909). The probability that the synonymous expression y _n ^m is generated is learned according to the evaluation value H (y _n ^m ) of the ease of processing (reward) for the synonymous expression.

Specifically, the synonymous expression generation model learning unit 645 first uses the evaluation value H (x ⁿ ) of the ease of processing calculated from the query x _n by the formula 4 shown below as a reference, and uses the standard. Find the difference between the synonymous expression y _n ^m and the evaluation value H (y _n ^m ).
[Formula 4]

Then, the synonymous expression generation model learning unit 645 learns so that the loss function shown in the following equation 5 is minimized.
[Formula 5]

The synonymous expression generation model learning unit 645 stores the latest synonymous expression generation model in the domain adaptive model storage unit 65 each time the learning process is completed (s910).

The synonymous expression generation model may be generated by, for example, pre-learning by the synonymous expression generation model learning unit 645. In that case, the synonymous expression generation unit 642 can use the pre-learned synonymous expression generation model stored in the domain adaptive model storage unit 65 when the synonymous expression generation model is first read in the sentence generation mode. .. In the above pre-learning, for example, the synonymous expression generation model learning unit 645 reads the query corpus from the storage unit 63 and queries.
This is done by learning an encoder / decoder neural model to restore x _n . The synonymous expression generation model learning unit 155 stores the pre-learned synonymous expression generation model in the domain adaptive model storage unit 16. Since the model pre-trained by such a procedure can generate sentences in the same language as the query corpus, it is possible to generate multiple synonymous expressions by performing beam search decoding.

Subsequently, the processing performed by the search processing unit 70 (hereinafter, referred to as “search processing S1000”) will be described.

FIG. 10 is a flowchart illustrating the search process S1000. Hereinafter, the search process S1000 will be described with reference to the figure.

First, the input interface 51 accepts a query input from the user (s1001), and transmits the accepted query to the domain adaptation determination unit 72 (s1002).

The domain adaptation determination unit 72 determines whether or not it is necessary to convert the transmitted query into synonymous expressions (s1003). Specifically, the domain adaptation determination unit 72 reads the search engine from the search engine storage unit 62 and inquires the query to the search engine unit 74. The similarity calculation unit 743 of the search engine unit 74 assigns a ranking to each content and generates a search result by using the degree of association between the query and all the contents included in the search content storage unit 122. Then, each search result to which the ranking is given is transmitted to the domain adaptation determination unit 72.

The domain adaptation determination unit 72 determines the degree of relevance (for example, an evaluation index such as TF / IDF) between the content having the highest ranking among the transmitted contents and the query (S1003). If the degree of relevance is less than the predetermined threshold s (s1003: YES), the process proceeds to s1004. If the degree of relevance is equal to or greater than the threshold s (s1003: NO), the process proceeds to s1006.

In S1004, the query synonym expression generation unit 73 receives the query transmitted from the domain adaptation determination unit 72. Then, the query synonymous expression generation unit 73 reads the synonymous expression generation model from the domain adaptation model storage unit 65, generates the synonymous expression of the query, and transmits the generated synonymous expression as a query to the query input unit 741 of the search engine unit 74. (S1005).

In s1006, the query input unit 741 transmits the character string of the query received from the query synonym expression generation unit 73 or the domain adaptation determination unit 72 to the similarity calculation unit 743. The similarity calculation unit 743 calculates the degree of relevance between the query and all the contents included in the search content storage unit 742, assigns a ranking to each content, and generates a search result (s1007).

Subsequently, the output interface 75 outputs the above search result (s1008). The output interface 75 outputs the search results in an appropriate format, for example, a content having the highest ranking and a link to the content, or a list in which the content and links to each content are arranged in the order of ranking.

As described above, the search system 50 of the present embodiment determines the relevance of each query in the query corpus, its synonymous expression, each of the query and the synonymous expression, and all the contents possessed by the search system 50. Using the included synonymous expression information as training data, generate a synonymous expression generation model for converting the input query into a synonymous expression query for which the correct content can be easily extracted, and synonymously express the input query with the synonymous expression generation model. Convert to and enter as a new query. Therefore, even if a query written in natural language including ambiguous expressions is input, the correct content can be output.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and includes various modifications. Further, for example, the above-described embodiment is described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the described configurations. Further, it is possible to add, delete, or replace a part of the configuration of each embodiment with other configurations.

Further, each of the above configurations, functional units, processing units, processing means and the like may be realized by hardware by designing a part or all of them by, for example, an integrated circuit. Further, each of the above configurations, functions, and the like may be realized by software by the processor interpreting and executing a program that realizes each function. Information such as programs, tables, and files that realize each function can be stored in memory, hard disks, recording devices such as SSDs (Solid State Drives), and ICs.
It can be placed on a recording medium such as a card, SD card, or DVD.

Further, the arrangement form of various functional units, various processing units, and various databases of each information processing apparatus described above is only an example. The arrangement form of various function units, various processing units, and various databases can be changed to the optimum arrangement form from the viewpoint of the performance, processing efficiency, communication efficiency, and the like of the hardware and software included in these devices.

Further, the configuration of the database (schema, etc.) for storing various data described above can be flexibly changed from the viewpoints of efficient use of resources, improvement of processing efficiency, improvement of access efficiency, improvement of search efficiency, and the like.

1 Machine translation system, 10,60 domain adaptive learning unit, 11,61 data input processing unit, 12,63 storage unit, 13 adaptation source translation model learning unit, 14 adaptation source translation model storage unit, 15,64 domain adaptation model learning unit Unit, 16,65 Domain adaptation model storage unit, 20 data expansion unit, 21 adaptation destination translation corpus generation unit, 22 adaptation destination translation corpus selection unit, 30 adaptation destination translation model learning unit, 31 adaptation destination translation corpus storage unit, 32 adaptation Destination translation model learning unit, 33 adaptation destination translation model storage unit, 40 translation processing unit, 41,71 input interface, 42 translation unit, 43,75 output interface, 50 search system, 62 search engine storage unit, 70 search processing unit, 72 Domain adaptation discrimination unit, 73 Query synonymous expression generation unit, 111,611 user interface, 112,612 data processing unit, 151 single language corpus input unit, 152,642 synonymous expression generation unit, 153,643 reward output unit, 154 644 Synonymous expression information storage unit, 156 translation pair extraction unit, 641 query input unit, 741 query input unit, 742 search content storage unit, 743 similarity calculation unit, 744 search result output unit

Claims (12)

It is configured using an information processing device and
A storage unit that stores a monolingual corpus and a synonymous expression generation model,
A synonymous expression generation unit that generates synonymous expressions of sentences in the monolingual corpus using the synonymous expression generation model, and
An evaluation unit that calculates the evaluation value of the ease of natural language processing for each of the sentence and the synonymous expression generated for the sentence, and an evaluation unit.
The synonymous expression that generates the synonymous expression that is easy to process in natural language from the input sentence by using the sentence, the synonymous expression generated from the sentence, and the synonymous expression information that is information including the evaluation value as learning data. The domain adaptive model learning unit that generates the generation model,
Natural language processing system with. The natural language processing system according to claim 1.
The domain adaptive model learning unit executes a pre-learning process to generate an encoder / decoder-type neural model for restoring a sentence of the monolingual corpus as the synonymous expression generation model.
The synonymous expression generation unit uses the synonymous expression generation model generated by the pre-learning process when first generating a synonymous expression of a sentence.
Natural language processing system. The natural language processing system according to claim 1.
The synonymous expression generation unit generates a plurality of the synonymous expressions by executing beam search decoding after encoding the synonymous expressions generated by using the synonymous expression generation model.
The domain adaptation model learning unit can easily perform the natural language processing on an input sentence using the sentence, the plurality of synonymous expressions generated from the sentence, and the synonymous expression information including the evaluation value as learning data. Generate the synonymous expression Generate the synonymous expression generation model,
Natural language processing system. The natural language processing system according to claim 3.
The synonym expression generation unit performs preprocessing for recognizing synonyms included in a plurality of generated synonyms as the same character string, and
Following the preprocessing, a noise reduction process for removing noise from the plurality of synonymous expressions,
And run
In the preprocessing, the common character string is extracted based on the longest match, and the characters between the common character strings extracted by each synonym are used as synonyms.
In the noise reduction processing, the similarity between the synonymous expression and the sentence from which the synonymous expression is derived is obtained based on a predetermined index, and the synonymous expression whose similarity is equal to or less than a predetermined value is regarded as noise. To remove,
Natural language processing system. The natural language processing system according to any one of claims 1 to 4.
The storage unit stores an adaptation source bilingual corpus, which is a bilingual corpus of the adaptation source domain, and an adaptation destination monolingual corpus, which is a monolingual corpus of the adaptation destination domain.
An adaptation source translation model learning unit that generates an adaptation source translation model, which is a translation model that machine-translates a sentence of the adaptation source domain using the adaptation source bilingual corpus as learning data.
The adaptation destination translation model learning unit that generates the adaptation destination translation model that translates the sentence of the adaptation destination domain,
An adaptation destination translation corpus generator that generates an adaptation destination translation corpus, which is a translation corpus of the adaptation destination domain,
A translation unit that translates the input sentence of the adaptation destination domain using the adaptation destination translation model, and
Equipped with
The synonymous expression generation unit generates a synonymous expression of the sentence of the adaptation destination domain of the adaptation destination monolingual corpus.
The evaluation unit calculates the ease of processing when the adaptation source translation model learning unit machine-translates the sentence of the adaptation destination domain and each of the synonymous expressions of the sentence as the evaluation value.
The domain adaptation model learning unit learns the synonymous expression generation model so that the evaluation value of the synonymous expression that is easier to translate becomes high and the evaluation value of the synonymous expression that is difficult to translate becomes low.
The adaptation destination translation corpus generation unit is a pair of a synonymous expression that is easy to machine translate in the adaptation source translation model and a translation of the synonymous expression, and a synonym expression that is difficult to machine translate in the adaptation source translation model. The target translation corpus is generated by collecting each pair of the reference translation for the sentence in the adaptation source translation corpus that is the origin of the synonymous expression.
The adaptation destination translation model learning unit generates the adaptation destination translation model using the adaptation destination translation corpus as learning data.
Natural language processing system. The natural language processing system according to any one of claims 1 to 4, wherein the natural language processing system is used.
It has a search processing unit that outputs the content corresponding to the query entered by the user by the search engine.
The storage unit stores a query corpus that collects queries for search engines as the monolingual corpus, and stores the contents.
The synonymous expression generation unit generates a synonymous expression of the sentence of the query corpus.
The evaluation unit calculates the evaluation value based on the query included in the query corpus, each of its synonymous expressions, and the degree of relevance to all the contents stored in the storage unit.
The domain adaptation model learning unit minimizes the loss function of the difference between the evaluation value based on the degree of association between the query and the content and the evaluation value based on the synonymous expression of the query and the degree of association with the content. To generate the synonymous expression generation model as described above,
The search processing unit generates a synonymous expression from the user-input query by the synonymous expression generation model, and inputs the synonymous expression as a query to the search engine.
Natural language processing system. Information processing equipment
A monolingual corpus, and a step to memorize synonymous expression generation models,
A step of generating a synonymous expression of a sentence of the monolingual corpus using the synonymous expression generation model,
A step of calculating the evaluation value of the ease of natural language processing for each of the sentence and the synonymous expression generated for the sentence, and
The synonymous expression that generates the synonymous expression that is easy to process in natural language from the input sentence by using the sentence, the synonymous expression generated from the sentence, and the synonymous expression information that is information including the evaluation value as learning data. Steps to generate a generation model,
Natural language processing method to execute. The natural language processing method according to claim 7.
The information processing device
A step of executing a pre-learning process to generate an encoder / decoder neural model for restoring a sentence of the monolingual corpus as the synonymous expression generation model.
A step of using the synonymous expression generation model generated by the pre-learning process when first generating a synonymous expression of a sentence,
A natural language processing method that further executes. The natural language processing method according to claim 7.
The information processing device
A step of generating a plurality of the synonyms by executing beam search decoding after encoding the synonyms generated using the synonym generation model, and
Using the sentence, the plurality of synonymous expressions generated from the sentence, and the synonymous expression information including the evaluation value as learning data, the synonymous expression that generates the synonymous expression that is easy to process in natural language for the input sentence. Steps to generate a generative model,
A natural language processing method that further executes. The natural language processing method according to claim 9.
The information processing device
A step of executing preprocessing for recognizing synonyms contained in a plurality of generated synonyms as the same character string,
Following the preprocessing, a step of executing a noise reduction process for removing noise from the plurality of synonymous expressions.
Further run,
In the preprocessing, the common character string is extracted based on the longest match, and the characters between the common character strings extracted by each synonym are used as synonyms.
In the noise reduction processing, the similarity between the synonymous expression and the sentence from which the synonymous expression is derived is obtained based on a predetermined index, and the synonymous expression whose similarity is equal to or less than a predetermined value is regarded as noise. To remove,
Natural language processing method. The natural language processing method according to any one of claims 7 to 10.
The information processing device
A step to memorize the adaptation source bilingual corpus, which is the translation corpus of the adaptation source domain, and the adaptation destination monolingual corpus, which is the monolingual corpus of the adaptation destination domain.
A step of generating an adaptation source translation model, which is a translation model for machine-translating a sentence of the adaptation source domain using the adaptation source bilingual corpus as learning data.
A step of generating an adaptive translation model that translates a sentence in the destination domain,
A step of generating an adaptation destination bilingual corpus, which is a bilingual corpus of the adaptation destination domain,
The step of translating the input sentence of the adaptation destination domain using the adaptation destination translation model,
A step of generating a synonym for a sentence in the destination domain of the destination monolingual corpus,
A step of calculating the ease of processing when machine-translating each of the sentence of the adaptation destination domain and the synonymous expression of the sentence as the evaluation value.
A step of learning the synonymous expression generation model so that the evaluation value of the synonymous expression that is easier to translate becomes higher and the evaluation value of the synonymous expression that is difficult to translate becomes lower.
The pair of synonymous expressions that are easy to machine translate in the adaptation source translation model and the translation of the synonymous expressions, and the synonymous expressions that are difficult to machine translate in the adaptation source translation model and the adaptation that became the origin of the synonymous expressions. A step of collecting each pair of a reference translation for a sentence in the original translation corpus to generate the adaptation destination translation corpus, and
A step of generating the adaptive translation model using the adaptive translation corpus as learning data.
A natural language processing method that further executes. The natural language processing method according to any one of claims 7 to 10.
The information processing device
A step in which a search engine outputs content that corresponds to a user-entered query,
A step of storing the query corpus that collects queries to search engines as the monolingual corpus and storing the content.
Steps to generate synonyms for the query corpus statement,
A step of calculating the evaluation value based on each of the queries included in the query corpus and their synonymous expressions and the degree of relevance to all the contents to be stored.
The synonymous expression generation model is set so that the loss function of the difference between the evaluation value based on the degree of association between the query and the content and the evaluation value based on the synonymous expression of the query and the content is minimized. Steps to generate and
A step of generating a synonymous expression from a user-input query by the synonymous expression generation model and inputting the synonymous expression into a search engine as a query.
A natural language processing method that further executes.

Images