JP2022042030A - Information processing system and information processing program - Google Patents

Abstract

To provide an information processing system and an information processing program, capable of improving processing accuracy.SOLUTION: An information processing system for performing machine learning again according to a learned target model that outputs numerical information indicating characteristics of a word, which is a learned BERT model that is generated by performing the machine learning for one or more words, includes: an acquisition unit 231 that acquires an unknown word that is not a target of the machine learning in the learned BERT model from first sentence information; and a learning unit 232 that uses second sentence information to perform the machine learning for at least the unknown word acquired by the acquisition unit 231 according to the learned BERT model. The learning unit 232 determines whether or not the unknown word is used as a target of the machine learning on the basis of a predetermined determination reference, and performs the machine learning for the unknown word determined as a target of the machine learning according to the learned target model.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing system and an information processing program.

Conventionally, a technique for performing natural language processing has been known (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2011-1130997

By the way, in natural language processing, various processes related to sentence information may be performed using a model generated by performing machine learning on words.

However, when processing is performed using such a model, if the field of text information to be processed becomes a field different from the field where machine learning is performed, the processing accuracy may decrease and the processing may be performed. There was room for improvement from the perspective of improving accuracy.

The present invention has been made in view of the above, and an object of the present invention is to provide an information processing system and an information processing program capable of improving processing accuracy.

The information processing system according to claim 1 is a trained target model generated by performing machine learning on one or more words in order to solve the above-mentioned problems and achieve the object, and is a trained target model of words. The first sentence information of unknown words that are not the target of machine learning in the trained target model, which is an information processing system for performing machine learning again for the trained target model that outputs numerical information indicating the characteristics. It is provided with an acquisition means acquired from the above, and a learning means for performing machine learning on the unknown word acquired by the acquisition means at least for the learned target model using the second sentence information.

The information processing system according to claim 2 is the information processing system according to claim 1, wherein the learning means targets the unknown word acquired by the acquisition means for machine learning based on a predetermined determination criterion. It is determined whether or not to do so, and machine learning is performed on the learned target model regarding the unknown word that is determined to be the target of machine learning among the unknown words acquired by the acquisition means.

The information processing system according to claim 3 is the information processing system according to claim 2, wherein the learning means is acquired by the acquisition means based on the criteria regarding the part of the unknown word acquired by the acquisition means. It is determined whether or not the unknown word is the target of machine learning.

The information processing system according to claim 4 is the arithmetic system according to claim 2 or 3, wherein the learning means is the first sentence information or the second sentence information of the unknown word acquired by the acquisition means. Based on the criteria regarding the frequency of appearance within, it is determined whether or not the unknown word acquired by the acquisition means is targeted for machine learning.

The information processing system according to claim 5 is the information processing system according to any one of claims 1 to 4, wherein the acquisition means is an important word in the first sentence information based on a predetermined specific criterion. Alternatively, the important phrase is specified, and the unknown word is acquired from the first sentence information based on the specified important word or the important phrase.

The information processing system according to claim 6 stores word dictionary information in which words uniquely identified by the trained target model are recorded in the information processing system according to any one of claims 1 to 5. The acquisition means includes a storage means for storing, and the acquisition means acquires words other than the words recorded in the word dictionary information among the words included in the first sentence information as the unknown words.

The information processing system according to claim 7 is the information processing system according to claim 6, wherein the learning means newly records the unknown word acquired by the acquisition means in the word dictionary information, and the second. Using the text information, machine learning is performed on the learned target model at least for the unknown word newly recorded in the word dictionary information.

The information processing system according to claim 8 is the information processing system according to any one of claims 1 to 7, wherein the first sentence information is information indicating a sentence corresponding to a title of a news article. The second sentence information is information indicating a sentence corresponding to the body of a news article.

The information processing program according to claim 9 is a trained target model generated by performing machine learning on one or more words, and the trained target model that outputs numerical information indicating the characteristics of the words. An information processing program for performing machine learning again, which is an acquisition means for acquiring an unknown word that is not the target of machine learning in the trained target model from the first sentence information, and a second sentence information. Is used to make the trained target model function as at least a learning means for performing machine learning on the unknown word acquired by the acquisition means.

According to the information processing system according to claim 1 and the information processing program according to claim 9, for example, the trained target model can be updated by performing machine learning on unknown words acquired by the acquisition means. Therefore, it is possible to improve the processing accuracy of the processing performed based on the trained target model. Further, for example, since a newly trained target model is not generated, machine learning can be performed using a relatively small amount of second sentence information, so that the time for performing the machine learning can be shortened. Is possible.

According to the information processing system according to claim 2, for example, by performing machine learning on the learned target model regarding the unknown words determined to be the target of machine learning among the unknown words acquired by the acquisition means. Since unknown words can be narrowed down appropriately, it is possible to shorten the time for machine learning.

According to the information processing system according to claim 3, it is appropriate to determine whether or not an unknown word is subject to machine learning, for example, by determining whether or not the unknown word is subject to machine learning based on the criteria for part of speech. It is possible to make a judgment.

According to the information processing system according to claim 4, for example, whether or not an unknown word is targeted for machine learning is determined by determining whether or not an unknown word is targeted for machine learning based on a criterion regarding frequency of appearance. It becomes possible to make an appropriate judgment.

According to the information processing system according to claim 5, for example, a useful unknown word is acquired by acquiring an unknown word from the first sentence information based on an important word or an important phrase specified in the first sentence information. Therefore, it is possible to improve the processing accuracy of the processing performed based on the trained target model.

According to the information processing system according to claim 6, it is useful, for example, by acquiring a word other than the word recorded in the word dictionary information among the words included in the first sentence information as an unknown word. Since unknown words can be acquired, it is possible to improve the processing accuracy of the processing performed based on the trained target model.

According to the information processing system according to claim 7, at least machine learning is performed on unknown words newly recorded in the word dictionary information, for example, machine learning is performed using a relatively small amount of second sentence information. Since it can be performed, it is possible to shorten the time for performing the machine learning.

According to the information processing system according to claim 8, the first sentence information is information indicating a sentence corresponding to the title of the news article, and the second sentence information is information indicating the sentence corresponding to the text of the news article. As a result, for example, machine learning can be appropriately performed for unknown words, so that it is possible to improve the processing accuracy of the processing performed based on the trained target model.

An embodiment of the present invention is a block diagram of an information system. It is a figure exemplifying the word dictionary information for BERT. It is explanatory drawing of each model recorded in a server apparatus. It is a figure exemplifying the dictionary information for morphological analysis. It is explanatory drawing of the information for re-learning. It is a flowchart of a re-learning process. It is a figure which illustrated the dictionary information for morphological analysis in the state which important phrases are stored. It is a figure which illustrated the processing example. It is a figure which illustrated the processing example. It is a figure which illustrated the processing example. It is a figure which illustrated the processing example. It is a figure exemplifying the word dictionary information for BERT in the state where an unknown word is newly registered. It is a flowchart of the trained BERT model re-learning process. It is a flowchart of a classification process.

Hereinafter, the information processing system and the embodiment of the information processing program according to the present invention will be described in detail with reference to the drawings. However, the present invention is not limited to the embodiments. Here, after explaining the basic concept and terms, a specific embodiment will be described.

(Basic concept)
First, the basic concept will be explained. The information processing system according to the present invention is a system for performing machine learning (that is, "re-learning") again for a trained target model, for example, a dedicated system for performing machine learning again, or a general-purpose system. Install an information processing program on the system used (for example, a general-purpose computer, a server computer, or multiple computers distributed on a network (that is, a so-called cloud computer)) and perform machine learning again. It is a concept that includes a system etc. realized by implementing a function to perform.

The "learned target model" is a model generated by performing machine learning on one or more words, and is a model that outputs numerical information indicating the characteristics of the words. The "numerical information indicating the characteristics of a word" is, for example, numerical information corresponding to a distributed expression indicating the characteristics of a word, that is, a concept including numerical information indicating vector information corresponding to the distributed expression of a word. .. The "characteristic of a word" may be interpreted as a concept corresponding to the meaning of a word. The trained target model is arbitrary, but is a concept including, for example, a model of natural language processing corresponding to a known BERT (Biorectional Encoder Representations from Transfermers), or any other model according to the above definition.

The information processing system includes, for example, acquisition means and learning means, and optionally includes storage means.

=== Each means ===
The "acquisition means" is a means for acquiring an unknown word that is not the target of machine learning in the trained target model from the first sentence information. Further, the "acquisition means" is, for example, specifying an important word or an important phrase in the first sentence information based on a predetermined specific criterion, and unknown from the first sentence information based on the specified important word or the important phrase. It is a concept that includes means for acquiring words. Further, the "acquisition means" is a concept including, for example, a means for acquiring a word other than a word recorded in the word dictionary information among the words included in the first sentence information as an unknown word.

The "learning means" is a means for performing machine learning on at least the unknown words acquired by the acquisition means for the trained target model using the second sentence information. Further, the "learning means" is, for example, based on a predetermined determination criterion, it is determined whether or not the unknown word acquired by the acquisition means is targeted for machine learning, and among the unknown words acquired by the acquisition means. , A concept including means for performing machine learning on a trained target model for unknown words determined to be the target of machine learning.

Further, the "learning means" includes, for example, a means for determining whether or not the unknown word acquired by the acquisition means is subject to machine learning based on the criteria regarding the part of speech of the unknown word acquired by the acquisition means. It is a concept. Further, the "learning means" is, for example, machine learning of the unknown words acquired by the acquisition means based on the criteria regarding the frequency of appearance of the unknown words acquired by the acquisition means in the first sentence information or the second sentence information. It is a concept including means for determining whether or not to be the target of.

Further, the "learning means" is, for example, newly recording an unknown word acquired by the acquisition means in the word dictionary information, and newly recording the learned target model in at least the word dictionary information using the second sentence information. It is a concept that includes means for performing machine learning related to unknown words.

The "storage means" is a means for storing word dictionary information in which words uniquely identified in the trained target model are recorded.

=== Each term ===
The "unknown word" is a word that is not the target of machine learning in the trained target model, and specifically, is a concept that includes a word or the like that cannot be uniquely identified in the trained target model. For example, it is a concept including words and the like that are not recorded in the word dictionary information.

The "word dictionary information" is, for example, information in which words uniquely identified in the trained target model are recorded. For example, when processing is performed using the trained target model, the words recorded in the word dictionary information can be uniquely identified and numerical information indicating the characteristics of the words can be output.

The "predetermined specific standard" is a standard for specifying an important word or an important phrase in the first sentence information. The "word" may be interpreted as a concept including, for example, the smallest unit having a meaning or a function, and the "phrase" is, for example, a concept corresponding to a plurality of words. It may be interpreted as something. That is, for example, a "phrase" may be interpreted as a concept corresponding to two or more words.

The "predetermined determination criterion" is a criterion for determining whether or not an unknown word is a target of machine learning. This criterion is arbitrary, but is a concept including, for example, a criterion regarding the part of speech of an unknown word, a criterion regarding the frequency of appearance of an unknown word, a combination thereof, or other criteria.

The "first sentence information" is, for example, information indicating a sentence (that is, a set of one or more sentences) for which an unknown word is to be acquired, and one example corresponds to the title of a news article. It is a concept that includes information indicating sentences. The "second sentence information" is, for example, information for performing machine learning on at least unknown words for a trained target model, and as an example, is a concept including information indicating a sentence corresponding to the text of a news article. be. As the first sentence information and the second sentence information, arbitrary information other than the information related to the news article (for example, information related to the blog, publicly known corpus information, etc.) may be used.

The above-mentioned direct or indirect use cases of the trained target model (including the retrained trained target model) are optional and can be used for processing to execute various tasks related to natural language processing. , For example, a trained object model may be used to perform various tasks related to classification, prediction, generation, etc. That is, the trained object model may be used to read, imply, paraphrase, dialogue, summarize, etc. Alternatively, it may be configured to perform arbitrary processing such as translation.

Then, in the embodiment shown below, for example, a case where the trained target model is relearned and used for a task of dividing news articles into categories (that is, a classification task) will be illustrated and described. The "category" is a group of classifications based on the description contents of news articles, and in the present embodiment, a predetermined number (for example, 10 to 20 pieces, etc.) of categories (for example, "product introduction"". (Category of, "Comparison of products", etc.) are predetermined, and the case where each news article is divided into any category will be described as an example.

(Constitution)
First, the information system according to the present embodiment will be described. FIG. 1 is a block diagram of an information system according to an embodiment of the present invention.

The information system 100 includes, for example, a terminal device 1 and a server device 2. The number of terminal devices 1 is arbitrary, and a plurality of terminal devices 1 may be provided, but the description will be focused on those shown in FIG.

(Configuration-Terminal device)
The terminal device 1 is a device used by a user, and specifically, is a device that is communicably connected to a server device 2 via a network, and is, for example, a computer including a personal computer or the like. The specific configuration of the terminal device 1 is arbitrary, but for example, a known configuration of a personal computer can be applied.

(Configuration-Server device)
The server device 2 is an information processing system, and includes, for example, a communication unit 21, a recording unit 22, and a control unit 23.

(Configuration-Server device-Communication unit)
The communication unit 21 in FIG. 1 is a communication means for communicating with an external device (for example, a terminal device 1). The specific type and configuration of the communication unit 21 is arbitrary, but it can be configured by using, for example, a known communication circuit or the like.

(Configuration-Server device-Recording unit)
The recording unit 22 of FIG. 1 is a recording means (storage means) for recording a program and various data necessary for the operation of the server device 2, and for example, a hard disk or a flash memory (not shown) as an external recording device is used. It is composed of. However, instead of the hard disk or flash memory, or together with the hard disk or flash memory, any other recording medium including a magnetic recording medium such as a magnetic disk or an optical recording medium such as a DVD or a Blu-ray disk can be used. ..

As shown in FIG. 1, for example, the recording unit 22 stores BERT word dictionary information, learned BERT model, task model, morphological analysis word dictionary information, relearning information, and classification target information. ..

(Configuration-Server device-Recording unit-Word dictionary information for BERT)
The “word dictionary information for BERT” in FIG. 1 is the word dictionary information described above, and is information in which words uniquely identified by the trained BERT model are recorded. For example, each word is stored in each word. It is information for converting into a word ID for identifying.

FIG. 2 is a diagram illustrating the BERT word dictionary information. This BERT word dictionary information is arbitrary, but for example, it is a text file in which one word or the like uniquely identified in the trained BERT model is recorded in each line, and as an example, as shown in FIG. It will be described as a text file in which "UNK", "next year", "fifth generation communication", "to", etc. are stored in the first to fourth lines and the like. Note that "UNK" here is information having a different property from other words ("next year", "5th generation communication", "to", etc.), and is specifically stored in the BERT word dictionary information. This is information for converting a word other than the (recorded) word into a word ID.

The specific process of converting the word ID into a word ID using the word dictionary information for BERT is arbitrary, but for example, the control unit 23 is the same as the word to be converted in the word dictionary information for BERT of FIG. A word is specified and converted into a word ID corresponding to the line number (that is, the line number) in the file which is the word dictionary information for BERT in FIG. 2 in which the specified word is recorded. This will be described below.

Here, for example, "next year" is recorded in the second line in the word dictionary information for BERT in FIG. 2, so that the word ID corresponding to the second line is "ID2" (described for convenience). In the same way, "5th generation communication" and "to" are converted into "ID3" (word ID corresponding to the 3rd line) and "ID4" (word ID corresponding to the 4th line). Will be done. On the other hand, if "○○ service", "△△ plan", and "XX data" are not stored in the BERT word dictionary information of FIG. 2, all of these words are the BERT word dictionary information of FIG. It will be converted to "ID1" corresponding to the first line in which "UNK" is stored.

The specific storage method of such BERT word dictionary information is arbitrary, but for example, when a trained BERT model is generated by performing machine learning (that is, the first machine learning), the machine concerned. A predetermined number of words (for example, 32,000 including "UNK") to be learned in learning are stored as initial values of the BERT word dictionary information. Then, by performing the re-learning process described later after this, additional words are appropriately stored in the BERT word dictionary information.

Since machine learning for generating a trained BERT model (that is, initial machine learning) is known, only an outline will be described. Specifically, when an arbitrary sentence such as a news article or a dissertation article is input as a large amount of learning data, the control unit 23 converts the input information into the word dictionary information for BERT by an arbitrary method. Acquire the number of words to be stored as the initial value (for example, 31999 words excluding "UNK"), store the acquired words in the BERT word dictionary information as the initial value, and use the BERT word dictionary information. The trained BERT model is generated by converting the sequence of words included in each sentence of the above-mentioned input sentence into the sequence of word IDs and performing machine learning based on the sequence of the converted word IDs. do. By processing in this way, machine learning is performed in consideration of the word sequence (that is, the context) in each sentence in the above-mentioned input sentence indicated by the word ID sequence, and the learning is performed by the machine learning. A finished BERT model will be generated.

As described above, the trained BERT model is generated by machine learning performed using the word ID converted using the BERT word dictionary information. Therefore, the trained BERT is included in the BERT word dictionary information. Words and the like that are uniquely identified by the model will be stored.

The initial value of the word dictionary information for BERT may be stored by various other methods. For example, the information determined by the administrator may be input by an arbitrary method (for example, an input (not shown) of the server device 2. It is input to the server device 2 by a method using means (keyboard, mouse, etc.) or a recording medium such as a memory stick, and the input information is recorded as the initial value of the word dictionary information for BERT. It may be that. Further, in this case, the control unit 23 uses the word included in each sentence of the above-mentioned input sentence based on the initial value of the BERT word dictionary information (that is, the information corresponding to the information determined by the administrator). The trained BERT model may be generated after converting the sequence of words into the sequence of word IDs.

Further, the device that generates the trained BERT model is not limited to the server device 2, but the trained BERT model is generated by appropriately using the word dictionary information for BERT in other devices, and then the generated learned BERT model is used. It may be input and stored in the server device 2 together with the above-mentioned BERT word dictionary information (BERT word dictionary information used when the learned BERT model is generated).

(Configuration-Server device-Recording unit-Learned BERT model)
FIG. 3 is an explanatory diagram of each model recorded in the server device. The "learned BERT model" in FIG. 1 is, for example, as shown in the column of "model" = "learned BERT model" in FIG. 3, in which a sequence of word IDs is input and the input word. It is a model that outputs a sequence of distributed expression numerical information corresponding to each ID. The "distributed expression numerical information" is a concept indicating numerical information corresponding to the distributed expression of the word identified by the word ID.

The specific storage method of such a trained BERT model is arbitrary, but for example, it is generated and stored by performing the above-mentioned "first machine learning", and after that, a re-learning process (described later) is performed. It will be updated by performing FIG. 6), and after that, machine learning will be performed again using the information used when generating the task model (that is, the text of the title of the news article, for example) to update. Will be done. Then, in the present embodiment, for convenience of explanation, when it is necessary to separately explain the trained model at each stage, the trained BERT model generated by performing the above-mentioned "first machine learning" is "preliminarily". When a trained BERT model updated by performing a re-learning process (FIG. 6) described later is referred to as a "learned BERT model" and a task model is generated. A trained BERT model that has been updated by performing machine learning again using the information used for (that is, for example, the text of the title of a news article) is referred to as a "classified learning BERT model".

Further, in the present embodiment, the "pre-learned BERT model" may be interpreted as corresponding to the "learned target model". In addition, the interpretation is not limited to this correspondence, and other interpretations may be applied, for example, depending on the trained BERT model for which the re-learning process (FIG. 6) is executed.

It should be noted that the "unknown word learned BERT" performed using the information used when generating the task model (that is, for example, the text of the title of the news article), which is performed to update the "classified learned BERT model". Machine learning for "models" is known in the same way as "first machine learning" (that is, machine learning for generating "pre-trained BERT models"), so detailed explanations are omitted and only an outline is given. To explain. That is, in this machine learning process, the information used when generating the task model of the "unknown word learned BERT model" without adding information to the BERT word dictionary information (for example, news article). (That is, specializing in the information used when generating the task model), and updating to the "classified and learned BERT model".

(Configuration-Server device-Recording unit-Task model)
The "task model" in FIG. 1 is a model for executing a task, and specifically, a processing result performed using a trained BERT model (specifically, for example, a "classified trained BERT model"). It is a model for executing a task by using the above, and in the present embodiment, it is a model for executing a task for dividing news articles into categories. In this task model, for example, as shown in the column of "model" = "task model" in FIG. 3, a sequence of distributed expression numerical information is input, and the sequence of the input distributed expression numerical information belongs to. It is a model that outputs information indicating a category (that is, for example, information indicating a category to which a news article belongs, and information indicating which category the news article belongs to). As described here, it is preferable to execute the task model by using the result of the processing performed by using, for example, the "classified learned BERT model". For example, the "unknown language learned BERT model" is preferable. You may execute it by using the result of the processing performed by using.

The specific storage method of such a task model is arbitrary, but for example, machine learning with a teacher is performed using the text of the title of the news article and the information indicating the category to which the news article belongs. This may generate a task model and store the generated task model. The task model may be generated by the server device 2 or may be generated by another device.

(Configuration-Server device-Recording unit-Morphological analysis word dictionary information)
The "word dictionary information for morphological analysis" in FIG. 1 is information used in morphological analysis for dividing a sentence into words. FIG. 4 is a diagram illustrating morphological analysis dictionary information. In this morphological analysis word dictionary information, for example, as shown in FIG. 4, information corresponding to the item "word information", information corresponding to the item "part of speech information", and information corresponding to the item "reading information" Are interrelated.

The information corresponding to the item "word information" is word information indicating a word (in FIG. 4, "AB phone" or the like). The information corresponding to the item "part of speech information" is part of speech information indicating the part of speech of a word (in FIG. 4, "noun", not shown "verb", "auxiliary verb", etc.). The information corresponding to the item "reading information" is reading information indicating the reading of a word (in FIG. 4, "AB phone" or the like).

Then, for example, the information at the top of the word dictionary information for morphological analysis (information on the combination of "AB phone", "noun", and "AB phone") in FIG. 4 has the part of speech of the word "AB phone". It is shown to be a "noun" and the reading of the "AB phone" is "AB phone". The method for storing the word dictionary information for morphological analysis is arbitrary, but it may be stored, for example, by inputting information to the server device 2 by any method (information for re-learning described later and information for re-learning). The same applies to the classification target information).

(Configuration-Server device-Recording unit-Information for re-learning)
The “re-learning information” in FIG. 1 is information used for re-learning the learned BERT model, and is, for example, information used in the re-learning process (FIG. 6) described later. FIG. 5 is an explanatory diagram of information for re-learning. Arbitrary information can be used as the re-learning information. For example, information related to the classification target information described later may be used, or information not related to the classification target information may be used. In the form, a case where the classification target information is a news article belonging to the communication field and the news article in the communication field which is the same field as the field to which the classification target information belongs is used as the re-learning information will be described. Here, for example, as shown in FIG. 5, a news article is composed of a title and a text, and an article corresponding to "article number" = "1" (that is, the title is "AB phone XR and BC phone". "Compare!" And the text is "The specs of the AB phone XR and BC phone released the other day are ... And when comparing these, ... is superior.") , "Article number" = "2" (that is, the title is "BB that leads the 5th generation communication era has appeared.", And the text is "The service of the 5th generation communication era was last year's xx." It started in March, but when this service becomes widespread, it will become ... ”), and articles in multiple communication fields including articles corresponding to“ article number ”=“ 3 ”,“ 4 ”, etc. Will be described when is stored as information for re-learning.

In addition, it is interpreted that the title of the news article, which is "information for re-learning", corresponds to "first sentence information", and the text of the news article corresponds to "second sentence information". May be good.

(Configuration-Server device-Recording unit-Classification target information)
The “classified target information” in FIG. 1 is information to be classified into categories using a trained BERT model (specifically, for example, a “classified learned BERT model”) and a task model. Any information can be used as the classification target information, but for example, as described above, it is a news article belonging to the communication field, and is similar to the news article as the re-learning information exemplified in FIG. We will use other information in the form, i.e., information indicating other news articles composed of a title and body. The number of news articles as the classification target information is arbitrary, but for example, hundreds to thousands, or tens of thousands of news articles may be stored.

(Configuration-Server device-Control unit)
The control unit 23 in FIG. 1 is a control means for controlling the server device 2, and specifically, the CPU, various programs interpreted and executed on the CPU (basic control programs such as an OS, and startup on the OS). It is a computer configured to include an application program that realizes a specific function) and an internal memory such as a RAM for storing the program and various data. In particular, the program according to the embodiment is installed in the server device 2 via an arbitrary recording medium or network to substantially configure each part of the control unit 23.

The control unit 23 includes, for example, an acquisition unit 231 and a learning unit 232 in terms of functional concept. The acquisition unit 231 is an acquisition means for acquiring an unknown word that is not the target of machine learning in the learned BERT model from the first sentence information (a sentence corresponding to the title of a news article which is information for re-learning). The learning unit 232 uses the second sentence information (sentence corresponding to the text of the news article which is the re-learning information) to perform machine learning on the learned BERT model at least for the unknown words acquired by the acquisition unit 231. It is a means. The processing performed by each of the control units 23 will be described later.

(process)
Next, for example, a re-learning process and a classification process will be described as processes performed by the information system 100 configured in this way.

(Processing-Re-learning process)
FIG. 6 is a flowchart of the re-learning process (hereinafter, each step is referred to as “S”). The re-learning process is a process performed by the server device 2, and is generally a process of re-learning the learned BERT model generated by performing machine learning. The timing for executing this re-learning process is arbitrary, but for example, a large number of news articles in a new field are input and recorded as classification target information (FIG. 1), and the classification process described later (that is, the news article) is recorded. Predetermined timing before performing the process of dividing into categories) after a large number of other news articles in the same field as the classified target information are input and recorded as relearning information (FIG. 1). (For example, the timing at which the user performs an operation to execute the re-learning process via the terminal device 1) is started, and the description will be made from the point where the re-learning process is started.

Here, for example, in the recording unit 22 of FIG. 1 described above, the above-mentioned initial values are recorded as BERT word dictionary information by performing "first machine learning", and "pre-learning" as a learned BERT model. The case where the "finished BERT model" is recorded will be described as an example. Further, for example, since the classification target information is a news article in the communication field, the BERT model learned by using other news articles in the communication field so that the distributed expression numerical information can be appropriately output for the word related to the communication field. (Detailedly, the case of re-learning (“pre-learned BERT model”) and updating to “unknown language learned BERT model” will be described as an example. In reality, the server device 2 handles a large amount of information, but in the following description, the handling of some information will be illustrated and described for convenience of explanation.

=== SA1 ===
In SA1 of FIG. 6, the acquisition unit 231 acquires the re-learning information. Specifically, although it is arbitrary, for example, the title of the news article recorded as the re-learning information in the recording unit 22 of FIG. 1 is acquired.

Here, for example, the titles of news articles such as "article number" = "1" to "4" illustrated in FIG. 5 are acquired. That is, sentence information including a plurality of sentences such as "Comparing AB phone XR and BC phone!" And "BB leading the 5th generation communication era has appeared." Is acquired.

=== SA2 ===
In SA2 of FIG. 6, the acquisition unit 231 acquires important words and important phrases from the re-learning information acquired in SA1. Specifically, although it is arbitrary, for example, the title of the news article acquired by SA1 is divided into words, and important words and important phrases are acquired.

= Processing to divide into words =
The process of dividing a sentence into words is a process of dividing a sentence into words, for example, a process performed based on morphological analysis. Regarding this processing, for example, with reference to the word dictionary information for morphological analysis in FIG. 4, the same word as the word specified by the word information in the word dictionary information for morphological analysis is used for the title of the news article acquired by SA1. Divide as individual words.

Here, for example, although the term "AB phone XR" in the title of the news article acquired in SA1 is the name of a new product and is described as one proper noun in the news article, FIG. 4 It is assumed that the "AB phone XR" itself is not stored as the word information in the word dictionary information for morphological analysis. In addition, the same applies to the "5th generation communication era" as the "AB phone XR". FIG. 8 is a diagram illustrating a processing example. In this case, for example, "Comparing AB phone XR and BC phone!" Acquired in SA1 and "BB leading the 5th generation communication era has appeared." Are shown in the upper part of FIG. 8, for example. , "Compare AB phone, XR, and BC phone,!" And "Lead 5th generation communication, era, BB, appeared," and so on. .. Note that "," indicates the delimiter between the divided words.

= Processing to acquire important words and phrases =
The process of acquiring important words and important phrases is the process of identifying important words and important phrases based on the above-mentioned predetermined specific criteria for the words divided by the "process of dividing into words", and the specified important words and phrases. This is the process of acquiring important phrases.

Any standard is applied as the predetermined specific standard, but for example, a standard based on the concept of "Positionrank" or "TF (Term Frequency) -IDF (Inverse Document Frequency)" may be applied. Since known concepts can be applied as these criteria, only an outline will be described.

<Positionrank>
The criteria based on "Positionrank" is interpreted as, for example, a word or phrase in a sentence containing a plurality of sentences (the "phrase" here is interpreted as a plurality of consecutive words (two as an example)). ) Criteria, etc., in which words or phrases that are more closely related to each other are given a ranking of importance in each word or phrase, and those with higher rankings are regarded as important words or phrases. Is a concept that includes. The degree of relevance between each word or each phrase can be determined by any method including a known method, and for example, the co-occurrence width (that is, the number of consecutive words (3 to 5 as an example). Individuals, etc.)) may be set, and words included in the common co-occurrence width may be considered to have a higher degree of relevance, and as an example, the degree of relevance may be determined using a numerical value.

<TF-IDF>
The criteria based on "TF-IDF" are, for example, the more frequently appearing words or phrases in each sentence in a sentence containing a plurality of sentences (for example, the first sentence information), the higher the importance (degree of importance). In addition, it is a concept including a standard that a word or phrase having a predetermined importance or higher is regarded as an important word or important phrase, assuming that the smaller the number of sentences appearing, the higher the importance of the word or phrase.

In addition, important words and important phrases may be specified by using criteria other than the criteria described here. Then, here, for example, with respect to the word divided by the "process of dividing into words", as shown in the lower part of FIG. 8, "BC phone", "BB" and the like are specified and acquired as important words, and also. Specify "AB phone XR" (that is, the phrase corresponding to "AB phone, XR") and "5th generation communication era" (that is, the phrase corresponding to "5th generation communication, era") as important phrases. To get.

=== SA3 ===
In SA3 of FIG. 6, the acquisition unit 231 acquires an unknown word from the relearning information acquired in SA1. Specifically, although it is arbitrary, for example, the title of the news article acquired by SA1 is divided into words and an unknown word is acquired.

= Processing to divide into words =
The same process as the process of dividing into SA2 words is performed. Here, for example, as shown in the upper part of FIG. 8, "Compare AB phone, XR, and BC phone,!" And "Fifth generation communication, leading the era, BB," However, it appeared ,. ”, Etc.

= Processing to acquire unknown words =
The process of acquiring an unknown word is a process of identifying and acquiring an unknown word from the words divided by the "process of dividing into words". Regarding this process, for example, the BERT word dictionary information recorded in the recording unit 22 of FIG. 1 is referred to, and the word is recorded in the BERT word dictionary information among the words divided by the "process of dividing into words". The word that has not been specified is specified as an unknown word and then acquired.

Here, for example, a case where "AB phone", "XR", "BC phone", and "!" Are not recorded in the BERT word dictionary information of FIGS. 1 and 2 will be described. FIG. 9 is a diagram illustrating a processing example. In this case, as shown in FIG. 9, "AB phone", "XR", "BC phone", and "!" Are acquired as unknown words.

=== SA4 ===
In SA4 of FIG. 6, the acquisition unit 231 adds an important phrase to the morphological analysis word dictionary information of FIG. 1 and stores it. Specifically, although it is arbitrary, for example, among the important phrases acquired by SA2, an important phrase composed of only unknown words acquired by SA3 is specified, and the specified important phrase is used for morphological analysis in FIG. It is recorded as word information of word dictionary information, and also stores part-of-speech information and reading information associated with the recorded word information. The method of storing the part of speech information and the reading information is arbitrary, and for example, the part of speech and the reading may be estimated and automatically stored by any method, or the information regarding the part of speech and the reading may be stored for the user. A message prompting the user to input the information may be displayed on a display (not shown), and then stored based on the information input by the user.

FIG. 7 is a diagram illustrating morphological analysis dictionary information in a state where important phrases are stored. Here, for example, the important phrases acquired by SA2 are "AB phone XR" and "5th generation communication era", and the unknown words acquired by SA3 are "AB phone", "XR", and "BC phone". , And "!" Etc. The "AB phone XR" is composed only of the unknown words "AB phone" and "XR" acquired by SA3, while the "fifth generation communication era" is composed only of the unknown words acquired by SA3. Since this is not the case, "AB phone XR" is added to the morphological analysis word dictionary information of FIG. 7 and stored. In this way, the information related to "AB phone XR", which is the information at the bottom of FIG. 7, is newly registered in the word dictionary information for morphological analysis.

=== SA5 ===
In SA5 of FIG. 6, the acquisition unit 231 acquires an unknown word from the relearning information acquired in SA1. Specifically, although it is arbitrary, for example, the title of the news article acquired by SA1 is divided into words and an unknown word is acquired.

= Processing to divide into words =
Basically, the same process as the process of dividing into words of SA2 is performed, but in particular, the process is performed based on the word dictionary information for morphological analysis of FIG. 7 to which an important phrase is added in SA4.

FIG. 10 is a diagram illustrating a processing example. Here, for example, "Comparing AB phone XR and BC phone!" Acquired in SA1 and "BB leading the 5th generation communication era has appeared." Are shown in the upper part of FIG. 10, for example. , "Compare AB phone XR and BC phone,!" And "Lead 5th generation communication, era, BB, appeared ,." And so on. In particular, since "AB phone XR" is stored as word information in the word dictionary information for morphological analysis in FIG. 7, the "AB phone XR" is different from the case of "processing to divide into words" of SA2 or SA3. Will be divided into one word.

= Processing to acquire unknown words =
A process similar to the process of acquiring an unknown word of SA3 is performed. Here, for example, the process of acquiring an unknown word is a process of identifying and acquiring an unknown word from the words divided by the "process of dividing into words". Regarding this process, for example, the BERT word dictionary information recorded in the recording unit 22 of FIG. 1 is referred to, and the word is recorded in the BERT word dictionary information among the words divided by the "process of dividing into words". The unwritten word is identified as an unknown word and acquired. The case where "AB phone XR", "BC phone", and "!" Are not recorded in the BERT word dictionary information of FIGS. 1 and 2 will be described. In this case, as shown in the lower part of FIG. 10, "AB phone XR", "BC phone", "!", Etc. are acquired as unknown words.

=== SA6 ===
In SA6 of FIG. 6, the acquisition unit 231 acquires an unknown word as a candidate for re-learning from the unknown words acquired in SA5. That is, for example, from the unknown words acquired by SA5, the trained BERT model (specifically, for example, "pre-learned BERT model") is re-learned, and the effect of bringing it to the communication field side is relatively high. Get the presumed unknown word. Specifically, although it is arbitrary, for example, from the unknown words acquired in SA5, only the unknown words that match the important words or important phrases acquired in SA2 are acquired.

Here, for example, the unknown words acquired in SA5 are "AB phone XR", "BC phone", "!", Etc. as shown in the lower part of FIG. 10, and as shown in the lower part of FIG. The important words acquired by SA2 are "BC phone" and "BB", and the important phrases acquired by SA2 are "AB phone XR" and "5th generation communication era", while "!" Is not an important word or phrase. FIG. 11 is a diagram illustrating a processing example. Therefore, in SA6, among "AB phone XR", "BC phone", "!", Etc., "AB phone XR", "BC phone" and the like are acquired (that is, as shown in FIG. 11). , "!" Is excluded as unnecessary).

=== SA7 ===
In SA7 of FIG. 6, the learning unit 232 determines whether or not there is an unknown word to be relearned (that is, re-machine learning) among the unknown words acquired in SA6. Specifically, although it is arbitrary, for example, it is determined whether or not each unknown word is subject to machine learning based on the above-mentioned predetermined determination criteria, and the determination is made based on the determination result.

An arbitrary criterion is applied as the predetermined determination criterion, and for example, a case where the above-mentioned criterion regarding the part of speech of an unknown word and the criterion regarding the frequency of appearance of an unknown word are applied will be described.

= Criteria for part of speech of unknown words =
The "criteria for the part of speech of an unknown word" is a criterion for determining whether or not the unknown word is a predetermined part of speech, and in the present embodiment, a case where the predetermined part of speech is a noun will be described.

= Criteria for frequency of unknown words =
The "criteria for the frequency of appearance of unknown words" is a criterion for determining whether or not an unknown word appears more than a predetermined number of times, and specifically, relearning recorded in the recording unit 22 of FIG. It is a criterion for judging based on whether or not an unknown word appears more than a predetermined number of times in the title of a news article or the text of the news article, which is information for use. In this embodiment, the predetermined number of times is 5 times. A case will be described.

= Variations of each standard =
Various variations are assumed for these criteria, and these variations may be arbitrarily applied. Regarding the "criteria for part of speech of unknown words", for example, one or more arbitrary part of speech other than nouns (for example, verbs, adjectives, etc.) may be used as a predetermined part of speech.

In addition, the "criteria for the frequency of appearance of unknown words" is, for example, a criterion for determining whether or not an unknown word appears more than a predetermined number of times only in the title of a news article which is information for re-learning. It may be used as a criterion for judging based on whether or not an unknown word appears more than a predetermined number of times only in the text of a news article which is information for re-learning, or it is information for re-learning. Even if it is a criterion for judging based on whether or not an unknown word appears a predetermined number of times or more only in the title of a news article and the unknown word appears a predetermined number of times or more only in the title of the news article. Alternatively, it may be a criterion for judging based on whether or not an unknown word appears a total of a predetermined number of times or more in the title or text of a news article which is information for re-learning.

Further, the predetermined number of times in the "criteria for the frequency of appearance of unknown words" may be determined according to the predetermined part of speech in the above-mentioned "criteria for part of speech of unknown words". Further, the predetermined number of times or the predetermined part of speech of each of these criteria may be determined according to the field to which the news article indicated by the re-learning information or the classification target information in FIG. 1 belongs, or may be arbitrarily optimized. It may be determined by using a method (for example, Bayesian optimization, a method corresponding to Nelder-Mead).

= Criteria applied in this embodiment =
Then, in the present embodiment, as the "criteria for the part of speech of the unknown word", a criterion for judging based on whether or not the unknown word is a noun is used, and for the "criteria for the frequency of appearance of the unknown word". , Judgment based on whether or not the unknown word appears 5 times or more only in the title of the news article which is the information for re-learning, and the unknown word appears 5 times or more only in the text of the news article. The case where the standard is used will be described.

= Specific processing of SA7 =
Specifically, regarding the processing of SA7, first, the part of speech of each unknown word acquired in SA6 is specified by using an arbitrary method (for example, a method of specifying by referring to the word dictionary information for morphological analysis in FIG. 7). In addition, the title of the news article and the text of the news article indicated by the re-learning information recorded in the recording unit 22 of FIG. 1 are acquired, and each unknown word acquired by SA6 appears in the title. Specify the frequency and the frequency of appearance in the text. Next, for each unknown word acquired by SA6, the above-mentioned specified part of speech is a "noun", the frequency of appearance in the above-mentioned specified title is "5 times" or more, and the above-mentioned identification is performed. When the frequency of appearance in the text is "5 times" or more, it is determined that the word is an unknown word to be relearned (hereinafter, also referred to as "unknown word to be relearned"). On the other hand, for each unknown word acquired by SA6, when the above-mentioned specified part of speech is not a "noun", when the frequency of appearance in the above-mentioned specified title is less than "5 times", or when the above-mentioned identification is performed. If the frequency of appearance in the text is less than "5 times", it is determined that the word is an unknown word that is not the target of re-learning (hereinafter, also referred to as "non-target unknown word") (that is, the target of re-learning). Judge that it is not an unknown word).

Then, in the unknown words acquired by SA6, when there is no unknown word determined to be a "re-learning target unknown word" (that is, when it is determined that all unknown words are "non-target unknown words"), SA6 It is determined that there is no unknown word to be relearned among the unknown words acquired in (NO of SA7), and the process is terminated. On the other hand, in the unknown words acquired by SA6, if there is an unknown word determined to be "unknown word to be relearned", it is assumed that the unknown word to be relearned exists among the unknown words acquired by SA6. Judgment (YES of SA7), and shift to SA8.

Here, for example, in both the title and the text of the news article indicated by the re-learning information, a large number (for example, 20 etc.) of "AB phone XR" are included, and only 3 "BC phones" are included. Therefore, the case where "AB phone XR" appears 5 times or more in the above-mentioned title and the text, while "BC phone" appears less than 5 times will be described. In this case, the part of speech of each unknown word "AB phone XR" and "BC phone" acquired by SA6 is referred to the word information and part of speech information of the morphological analysis word dictionary information of FIG. 7, and both are ". Specify as "noun". Next, regarding "AB Phone XR", the frequency of appearance in the title of the news article indicated by the re-learning information is specified as "20 times", and the frequency of appearance in the text of the news article is also "20 times". After specifying, it is determined that the word is "unknown word to be relearned". On the other hand, regarding "BC phone", after specifying the frequency of appearance in the title and the text as "3 times", it is determined that the word is "non-target unknown word". Then, since it is determined that at least "AB phone XR" is an "unknown word to be relearned", it is determined that there is an unknown word to be relearned among the unknown words acquired by SA6 (). YES of SA7).

=== SA8 ===
In SA8 of FIG. 6, the learning unit 232 registers unknown words and the like. Specific processing is arbitrary, but for example, an unknown word determined to be a "re-learning target unknown word" in SA7 is specified, and the specified unknown word is used as the BERT word dictionary information in the recording unit of FIG. Register by recording additionally.

FIG. 12 is a diagram illustrating the BERT word dictionary information in a state where an unknown word is newly registered. Here, for example, the "AB phone XR" determined to be the "unknown word to be relearned" in SA7 is specified, and the specified "AB phone XR" is used in the BERT word dictionary information of the recording unit of FIG. By additionally recording at the bottom of a certain text file, it is registered as shown in FIG.

In this case, the process corresponding to the change of the layer size is also performed. The "process corresponding to a change in layer size" is a known process related to BERT, and is, for example, information used in a process related to BERT (processing such as re-learning or classification) in the word dictionary information for BERT. It is a process of changing the range of, specifically, a process of changing the setting of the "AB phone XR" newly registered in SA8 so as to be used in the process related to BERT. Specifically, before the addition of "AB Phone XR", 32000 words (that is, 32000 lines of information) are stored in the BERT word dictionary information as initial values, and the 32000 words are stored. It is set to perform processing related to BERT using, and by adding "AB phone XR", 32001 words (that is, 32001) including the "AB phone XR" in the word dictionary information for BERT. Since the line information) will be stored, process related to BERT should be performed using the 32001 words so that all the information including this added "AB phone XR" can be used. Change the settings.

=== SA9 ===
In SA9 of FIG. 6, the learning unit 232 executes the learned BERT model re-learning process. FIG. 13 is a flowchart of the trained BERT model re-learning process. The "learned BERT model re-learning process" is to relearn a learned BERT model (specifically, for example, "pre-learned BERT model" in this embodiment) for unknown words newly registered in SA8. It is a process.

=== SB1 ===
In SB1 of FIG. 13, the learning unit 232 acquires the re-learning information. Specifically, although it is arbitrary, for example, the text of the news article recorded as the re-learning information in the recording unit 22 is acquired.

Here, for example, the text of a news article such as "article number" = "1" to "4" illustrated in FIG. 5 is acquired. That is, "The specifications of the AB phone XR and BC phone released the other day are ... And when comparing these, ... is superior.", And "Services in the 5th generation communication era are superior." It started in xx month of last year, but when this service becomes widespread, it will be ... "and so on."

=== SB2 ===
In SB2 of FIG. 13, the learning unit 232 performs a process of dividing the text of the news article acquired in SB1 into words. Specifically, although it is arbitrary, for example, the process is basically the same as the process of dividing into words of SA5, that is, the process is performed based on the morphological analysis word dictionary information of FIG.

Here, for example, "the other day, release, sa, re, ta, AB phone XR, and BC phone, specs, are (hereinafter omitted)" and the like are divided. The explanation of "The service of the 5th generation communication era started in xx month of last year, but when this service becomes widespread, it will be ..." is omitted. In particular, as illustrated in FIG. 7, since "AB phone XR" has been added, the "AB phone XR" will be divided into one word.

=== SB3 ===
In SB3 of FIG. 13, the learning unit 232 converts the sequence of words divided by SB2 into the sequence of word IDs. Specifically, although it is arbitrary, for example, referring to the text file which is the word dictionary information for BERT in FIG. 12, the line in which the same word as the word divided by SB2 is recorded is specified, and the specified line is specified. Convert to the word ID corresponding to. It should be noted that among the words divided by SB2, there may be words that are not recorded in the BERT word dictionary information of FIG. 12, but in this case, the word ID corresponding to the above-mentioned "UNK" is "ID1". Will be converted to.

Here, for example, the sequence of "the other day, release, sa, re, ta, AB phone XR, and BC phone, specs, is ..." is changed to "ID (the other day), ID (release), ...". The word IDs "ID (sa), ID (re), ID (ta), ID32001, ID4, ID1, ID (no), ID (spec), ID (ha), ..." Are converted. The "ID (the other day)" here indicates the word ID corresponding to the line in which the word "the other day" is stored in the BERT word dictionary information of FIG. 12 (other "(()". The same applies to the expression of the word ID using ")").

In particular, since the "AB phone XR" is registered in the BERT word dictionary information of FIG. 12, as described above, the word ID "ID32001" (FIG. 12) uniquely identifies the "AB phone XR". It will be converted into the word ID corresponding to the 32001th line in the text file of the BERT word dictionary information. On the other hand, the "BC phone" is a word ID corresponding to "UNK" because it is determined to be a "non-target unknown word" in SA7 of FIG. 6 and is not registered in the BERT word dictionary information of FIG. It will be converted to "ID1".

Then, in this SB3, all the texts of the news articles, which are the re-learning information acquired in SB1, are converted into the sequence of word IDs.

=== SB4 ===
In SB4 of FIG. 13, the learning unit 232 learns by performing machine learning again for the learned BERT model recorded in the recording unit 22 of FIG. 1 using the sequence of word IDs converted in SB3. The completed BERT model is updated (specifically, in the present embodiment, for example, the "pre-learned BERT model" is updated to the "unknown language learned BERT model").

Here, for example, "ID (the other day), ID (release), ID (sa), ID (re), ID (ta), ID32001, ID4, ID1, ID (no), ID (spec), ID (ha) ), ... ”, etc., to update the trained BERT model by performing machine learning again for the trained BERT model recorded in the recording unit 22 of FIG. , Record the updated learned BERT model (Specifically, in this embodiment, for example, the "pre-learned BERT model" is updated to the "unknown word learned BERT model", and the updated "unknown word learned" is recorded. Stores the BERT model).

Then, after returning the trained BERT model re-learning process of FIG. 13, the re-learning process of FIG. 6 is terminated.

(Processing-Classification processing)
FIG. 14 is a flowchart of the classification process. The classification process is a process performed by the server device 2, and is generally a process of executing a classification task. The timing of executing this classification process is arbitrary, but for example, after the above-mentioned re-learning process (FIG. 6) is executed, the above-mentioned known machine learning is performed using the information used when generating a task model. This is the timing after the "unknown word learned BERT model" is updated and recorded in the "classified learned BERT model", and the task model is generated and recorded. As an example, the terminal device A predetermined timing (for example, the user classifies via the terminal device 1) after the user of 1 inputs and stores the classification target information of FIG. 1 by transmitting the classification target information of FIG. 1 to the server device 2 via the terminal device 1. It will be started at the timing when the operation to execute the process is performed, etc.), and the explanation will be given from the place where the classification process is started.

Here, for example, in the above-mentioned re-learning process (FIG. 6), the "pre-learned BERT model" is updated to the "unknown language learned BERT model" based on the news article in the communication field, and then the "unknown" is described. The case where the "word-learned BERT model" has been updated to the "classified-learned BERT model" and, as described above, the classified target information is a news article in the communication field will be described as an example.

=== SC1 ===
In SC1 of FIG. 14, the control unit 23 acquires the classification target information. Specifically, although it is arbitrary, for example, the title of the news article recorded as the classification target information in the recording unit 22 of FIG. 1 is acquired.

Here, for example, the news that the title is "Thorough verification of the newly released AB phone XR" and the text is "The rumored AB phone XR is ..." as the classification target information of the recording unit 22 in FIG. An example is an example in which a large number of news articles including articles are stored. In this case, the title such as "Thorough verification of the newly released AB phone XR" is acquired.

=== SC2 ===
In SC2 of FIG. 14, the control unit 23 performs a process of dividing the title of the news article acquired in SC1 into words. Specifically, although it is arbitrary, for example, the same processing as the processing of SB2 in FIG. 13 is performed, that is, the processing is performed based on the word dictionary information for morphological analysis in FIG.

Here, for example, it is divided into "new release, AB phone XR, ..." and the like.

=== SC3 ===
In SC3 of FIG. 14, the control unit 23 converts the sequence of words divided by SC2 into the sequence of word IDs. Specifically, although it is arbitrary, for example, the same processing as the processing of SB3 in FIG. 13 is performed, that is, the processing is performed based on the BERT word dictionary information of FIG.

Here, for example, it is converted into a sequence of word IDs such as "ID (new release), ID (no), ID32001, ID (o), ...". In particular, since the "AB phone XR" is registered in the BERT word dictionary information of FIG. 12, as described above, it is converted into the word ID "ID32001" that uniquely identifies the "AB phone XR". It will be.

=== SC4 ===
In SC4 of FIG. 14, the control unit 23 acquires a sequence of distributed representation numerical information. Specifically, although it is arbitrary, for example, a trained BERT model in which the sequence of word IDs converted by SC3 is recorded in the recording unit 22 of FIG. 1 for each title of each news article (specifically, a book). In the embodiment, for example, it is input to "classification trained BERT model"), and a sequence of distributed expression numerical information output from the trained BERT model is acquired.

Here, for example, "ID (new release), ID (no), ID32001, ID (o), ...", which is a sequence of word IDs converted by SC3, is recorded in the recording unit 22 of FIG. "Num_ID (new), which is a sequence of distributed expression numerical information output from the trained BERT model that has been input to the trained BERT model (specifically, for example," classified and trained BERT model "in this embodiment). (Release), Num_ID (), Num_ID32001, Num_ID (), ... ”and so on. It should be noted that "Num_ID (new release)" is distributed representation numerical information corresponding to "ID (new release)". The same applies to other expressions using "Num_".

=== SC5 ===
In SC5 of FIG. 14, the control unit 23 acquires information indicating a category. Specifically, although it is arbitrary, for example, the sequence of distributed expression numerical information acquired by SC4 is input to the task model recorded in the recording unit 22 of FIG. 1 for each title of each news article, and the task is concerned. The information indicating the category output from the model (that is, the information indicating the category to which the news article of the corresponding title belongs to the distributed representation numerical information input to the task model) is acquired.

Here, for example, "Num_ID (new release), Num_ID (), Num_ID320011, Num_ID (), ...", Which is a sequence of distributed expression numerical information acquired by SC4, is set in the recording unit 22 of FIG. Input to the recorded task model, and acquire information indicating the category of "product introduction" as information indicating the category output from the task model.

=== SC6 ===
In SC6 of FIG. 14, the control unit 23 classifies news articles stored as the classification target information of FIG. Specifically, although it is arbitrary, for example, by classifying each news article into a category based on the information indicating the category acquired by SC5 and transmitting the information indicating the classification result to the terminal device 1, the said article. The processing result can be used on the terminal device 1 side.

Here, for example, a news article whose title is "Thorough verification of the newly released AB phone XR" and whose text is "Rumored AB phone XR is ..." is classified into the "Product introduction" category. In the same way, other news articles are also classified into categories, and information indicating the classification result is transmitted to the terminal device 1 side.

The information indicating the classification result here is arbitrary, but for example, the information specifying each news article (for example, the number or title of the article) and the information indicating the category in which the article is classified are mutually exchanged. The associated information may be used, or list information that identifies news articles belonging to each category may be used.

Then, by processing in this way, the user of the terminal device 1 can grasp the category of each news article based on the information transmitted from the server device 2. This ends the classification process.

(Regarding classification accuracy)
As described above, the learned word is newly added to the BERT word dictionary as shown in FIG. 12 by performing the re-learning process of FIG. 6, and then the learned BERT model is re-learned. The BERT model can uniquely recognize the added unknown word and output the distributed expression numerical information, and the number of words that can be appropriately handled by using the learned BERT model increases. As a result, the learned BERT model and the task In the model, it becomes possible to grasp the context of the sentence more accurately, so that the classification accuracy of the news article is improved.

(About re-learning time)
Further, in the re-learning performed in SB4 of FIG. 13, the words in the BERT word dictionary information other than the unknown words newly stored in the BERT word dictionary information in SA8 of FIG. 7 are already machine-learned. Since it is a target, the amount of information for re-learning used in the re-learning (in the embodiment, the text of the news article) can be reduced, and a relatively small amount of re-learning information is used. Re-learning can be performed in a relatively short time (for example, 12 hours to 24 hours, etc.).

(Effect of embodiment)
As described above, according to the present embodiment, for example, the learned BERT model can be updated by performing machine learning on the unknown word acquired by the acquisition unit 231. Therefore, the learning is performed based on the learned BERT model. It is possible to improve the processing accuracy of the processing. Further, for example, since a newly trained BERT model is not generated, machine learning can be performed using a relatively small amount of second sentence information (the text of a news article), so that the machine learning can be performed. It is possible to shorten the time to do it.

Further, by performing machine learning on the learned BERT model regarding the unknown words determined to be the target of machine learning among the unknown words acquired by the acquisition unit 231, for example, the unknown words can be appropriately narrowed down. Therefore, it is possible to shorten the time for machine learning.

Further, by determining whether or not an unknown word is targeted for machine learning based on a criterion regarding part of speech, for example, it is possible to appropriately determine whether or not it is targeted for machine learning.

Further, by determining whether or not an unknown word is a target of machine learning based on a criterion regarding the frequency of appearance, for example, it is possible to appropriately determine whether or not an unknown word is a target of machine learning.

Further, by acquiring an unknown word from the first sentence information based on an important word or an important phrase specified in the first sentence information (news article title), for example, a useful unknown word can be acquired. , It is possible to improve the processing accuracy of the processing performed based on the trained BERT model.

Further, for example, by acquiring a word other than the word recorded in the word dictionary information (word dictionary information for BERT) among the words included in the first sentence information (title of the news article) as an unknown word, for example. Since useful unknown words can be acquired, it is possible to improve the processing accuracy of the processing performed based on the learned BERT model.

Further, by performing machine learning on at least the unknown words newly recorded in the word dictionary information (word dictionary information for BERT), for example, machine learning can be performed using the second sentence information having a relatively small amount of information. Therefore, it is possible to shorten the time for performing the machine learning.

Further, the first sentence information is information indicating a sentence corresponding to the title of the news article, and the second sentence information is information indicating the sentence corresponding to the body of the news article. Since learning can be performed, it is possible to improve the processing accuracy of the processing performed based on the trained BERT model.

[Variation example with respect to the embodiment]
Although the embodiments according to the present invention have been described above, the specific configuration and means of the present invention shall be arbitrarily modified and improved within the scope of the technical idea of the present invention described in the claims. Can be done. Hereinafter, such a modification will be described.

(About the problem to be solved and the effect of the invention)
First, the problem to be solved by the invention and the effect of the invention are not limited to the above-mentioned contents, and may differ depending on the implementation environment and the details of the configuration of the invention, and only a part of the above-mentioned problems. May be resolved or only some of the above effects may be achieved.

(About distribution and integration)
Further, each of the above-mentioned electrical components is a functional concept and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of dispersion or integration of each part is not limited to the one shown in the figure, and all or part of them may be functionally or physically dispersed or integrated in any unit according to various loads and usage conditions. Can be configured. Further, the "device" in the present application is not limited to a device composed of a single device, but includes a device composed of a plurality of devices.

(About shape, numerical value, structure, time series)
With respect to the components exemplified in the embodiments and drawings, the shape, numerical value, or the mutual relationship of the structure or time series of the plurality of components shall be arbitrarily modified and improved within the scope of the technical idea of the present invention. Can be done.

(About machine learning of task model)
Further, the task model of the above embodiment is generated by performing machine learning as described above, but it is used in the trained BERT model in which the above-mentioned re-learning is performed and in the re-learning. A task model may be generated by performing machine learning using the processing result of the processing performed based on the BERT word dictionary.

(About re-learning process (1))
Further, in the re-learning process of FIG. 6, the unknown word may be acquired by using only the important word or the important word among the important phrases. The processes other than the processes specified below are the same as the processes described in the embodiment (the same applies to "(Re-learning process (Part 2))" described later). Regarding the processing, for example, in SA2 after SA1, only important words or important phrases among important words are acquired. Next, SA3 and SA4 are omitted, and in SA5, a process of dividing into words based on the morphological analysis word dictionary information of FIG. 4 (that is, the morphological analysis word dictionary information to which important phrases are not added) is performed. After this, the process of acquiring the unknown word is performed. Next, in SA6, only the unknown words that match the important words acquired in SA2 are acquired from the unknown words acquired in SA5. Then, after this, SA7 to SA9 are executed.

(About re-learning process (2))
Further, in the re-learning process of FIG. 6, a predetermined word or a predetermined phrase may be registered as the word information of the word dictionary information for morphological analysis. The "predetermined word" is a predetermined word, and is a concept including, for example, a word input to the server device 2 by the user. Further, the "predetermined phrase" is a predetermined phrase, and is a concept including, for example, a phrase input to the server device 2 by the user. Regarding the processing, for example, after executing SA1, SA2 is omitted and SA3 is executed. Next, in SA4, the above-mentioned predetermined word or predetermined phrase is registered by recording it as word information of the word dictionary information for morphological analysis of FIG. In this case, the part-of-speech information and the reading information corresponding to these predetermined words or predetermined phrases are also input to the server device 2 by the user, and this information is registered. Next, in SA5, a process of dividing into words is performed using the word dictionary information for morphological analysis in which a predetermined word or a predetermined phrase is registered in SA4, and then a process of acquiring an unknown word is performed. Next, in SA6, only the unknown word that matches the predetermined word or the predetermined phrase registered in SA4 is acquired from the unknown words acquired in SA5. Then, after this, SA7 to SA9 are executed.

(About omission or change of processing)
Further, any process in the re-learning process of FIG. 6 may be omitted or changed. For example, all the unknown words acquired by SA3 are registered in the BERT word dictionary information, and then the learned BERT model re-learning is performed. It may be configured to perform processing. Alternatively, it may be configured to perform the learned BERT model re-learning process after registering all the unknown words acquired by SA5 in the BERT word dictionary information. Alternatively, among the unknown words acquired by SA3 or SA5, unknown words other than the unknown words excluded by SA6 (that is, unknown words other than unnecessary unknown words such as "!") Are registered in the BERT word dictionary information. Then, it may be configured to perform the trained BERT model re-learning process.

(About the first machine learning)
Further, the "first machine learning" regarding the trained BERT model of the above embodiment may be interpreted as basically the same as the trained BERT model re-learning process.

(Interpretation of terms)
Further, in the above embodiment, it has been explained that the "pre-learned BERT model" may be interpreted as corresponding to the "trained target model", but other interpretations can also be applied. For example, it is assumed that the relearning process of FIG. 6 is executed a plurality of times by exchanging the relearning information of the recording unit 22 of FIG. 1, but in this case, the processing target in the relearning process is the previously learned BERT. It is assumed that the "unknown word learned BERT model" updated by the model re-learning process will be obtained. In this case, it may be interpreted that the previously updated "unknown word learned BERT model" corresponds to the "learned target model".

(About the combination)
Further, the characteristics of the above-described embodiment and the characteristics of the modified example may be arbitrarily combined.

(Additional note)
The information processing system of Appendix 1 is a trained target model generated by performing machine learning on one or more words, and machine learning again on the trained target model that outputs numerical information indicating the characteristics of the words. This is an information processing system for performing the above learning, using the acquisition means for acquiring an unknown word that is not the target of machine learning in the trained target model from the first sentence information and the second sentence information. The completed target model is provided with at least a learning means for performing machine learning on the unknown word acquired by the acquisition means.

In the information processing system described in Appendix 1, whether or not the learning means targets the unknown word acquired by the acquisition means for machine learning based on a predetermined determination criterion. Is determined, and machine learning for the unknown word determined to be the target of machine learning among the unknown words acquired by the acquisition means is performed on the learned target model.

The information processing system of Appendix 3 is the information processing system according to Appendix 2, wherein the learning means obtains the unknown word acquired by the acquisition means based on the criteria regarding the part of the unknown word acquired by the acquisition means. Determine whether or not to target machine learning.

The information processing system of the appendix 4 is the arithmetic system according to the appendix 2 or 3, wherein the learning means appears in the first sentence information or the second sentence information of the unknown word acquired by the acquisition means. Based on the criteria regarding frequency, it is determined whether or not the unknown word acquired by the acquisition means is targeted for machine learning.

The information processing system of Supplementary Note 5 is the information processing system according to any one of Supplementary note 1 to 4, wherein the acquisition means uses important words or phrases in the first sentence information based on a predetermined specific criterion. The unknown word is acquired from the first sentence information based on the specified and specified important word or the important phrase.

The information processing system of Supplementary Note 6 is a storage means for storing word dictionary information in which words uniquely identified by the trained target model are recorded in the information processing system according to any one of Supplementary note 1 to 5. , And the acquisition means acquires words other than the words recorded in the word dictionary information among the words included in the first sentence information as the unknown words.

In the information processing system described in Appendix 6, the learning means newly records the unknown word acquired by the acquisition means in the word dictionary information, and uses the second sentence information. Then, for the trained target model, at least machine learning is performed on the unknown word newly recorded in the word dictionary information.

The information processing system of the appendix 8 is the information processing system according to any one of the appendices 1 to 7, wherein the first sentence information is information indicating a sentence corresponding to the title of a news article, and the second sentence. Information is information indicating a sentence corresponding to the text of a news article.

The information processing program of Appendix 9 is a trained target model generated by performing machine learning on one or more words, and machine learning again on the trained target model that outputs numerical information indicating the characteristics of the words. This is an information processing program for performing the above, using the acquisition means for acquiring an unknown word that is not the target of machine learning in the trained target model from the first sentence information and the second sentence information. , The learned target model is made to function as at least a learning means for performing machine learning on the unknown word acquired by the acquisition means.

(Effect of appendix)
According to the information processing system described in Appendix 1 and the information processing program described in Appendix 9, for example, the trained target model can be updated by performing machine learning on the unknown words acquired by the acquisition means. , It is possible to improve the processing accuracy of the processing performed based on the trained target model. Further, for example, since a newly trained target model is not generated, machine learning can be performed using a relatively small amount of second sentence information, so that the time for performing the machine learning can be shortened. Is possible.

According to the information processing system described in Appendix 2, for example, by performing machine learning on the learned target model regarding the unknown words determined to be the target of machine learning among the unknown words acquired by the acquisition means, for example. Since unknown words can be narrowed down appropriately, it is possible to shorten the time for machine learning.

According to the information processing system described in Appendix 3, for example, whether or not an unknown word is targeted for machine learning is appropriately determined by determining whether or not an unknown word is targeted for machine learning based on the criteria for part of speech. It becomes possible to judge.

According to the information processing system described in Appendix 4, for example, whether or not an unknown word is targeted for machine learning is appropriately determined by determining whether or not an unknown word is targeted for machine learning based on a criterion regarding frequency of appearance. It is possible to make a judgment.

According to the information processing system described in Appendix 5, for example, a useful unknown word is acquired by acquiring an unknown word from the first sentence information based on the important word or important phrase specified in the first sentence information. Therefore, it is possible to improve the processing accuracy of the processing performed based on the trained target model.

According to the information processing system described in Appendix 6, it is useful, for example, by acquiring words other than the words recorded in the word dictionary information among the words included in the first sentence information as unknown words. Since the unknown word can be acquired, it is possible to improve the processing accuracy of the processing performed based on the trained target model.

According to the information processing system described in Appendix 7, at least machine learning is performed on unknown words newly recorded in the word dictionary information, for example, machine learning is performed using a relatively small amount of second sentence information. Therefore, it is possible to shorten the time for performing the machine learning.

According to the information processing system described in Appendix 8, the first sentence information is the information indicating the sentence corresponding to the title of the news article, and the second sentence information is the information indicating the sentence corresponding to the text of the news article. As a result, for example, machine learning can be appropriately performed for unknown words, so that it is possible to improve the processing accuracy of the processing performed based on the trained target model.

1 Terminal device 2 Server device 21 Communication unit 22 Recording unit 23 Control unit 100 Information system 231 Acquisition unit 232 Learning unit

The information processing system according to claim 1 is a trained target model generated by performing machine learning on one or more words in order to solve the above-mentioned problems and achieve the object, and is a trained target model of words. It is an information processing system for performing machine learning again for the trained target model that outputs numerical information indicating features, and stores word dictionary information in which words uniquely identified by the trained target model are recorded. The storage means and the unknown word that is not the target of machine learning in the trained target model and is at least a word that is not recorded in the word dictionary information of the storage means are described in the first sentence. An acquisition means acquired from the information, and a learning means for performing machine learning on the unknown word acquired by the acquisition means at least for the trained target model using the second sentence information related to the first sentence information . The learning means includes a first process of newly recording the unknown word acquired by the acquisition means in the word dictionary information, and a second process of dividing the sentence included in the second sentence information into words. , The third process of converting the words divided in the second process into word identification information for identifying each word based on the word dictionary information stored in the storage means, and the third process. Based on the word identification information converted in, the fourth process of performing machine learning on the trained target model is performed.

The information processing system according to claim 6 is the information processing system according to any one of claims 1 to 5 , wherein the first sentence information is information indicating a sentence corresponding to the title of a news article. The second sentence information is information indicating a sentence corresponding to the body of a news article.
The information processing system according to claim 7 is the information processing system according to any one of claims 1 to 6, wherein in the third process, the learning means includes words divided in the second process. When it is recorded in the word dictionary information, it is converted into the first word identification information which is the word identification information for uniquely identifying the word, and the word divided in the second process is the word dictionary information. When it is not recorded in the word dictionary information, it is converted into the second word identification information which is the word identification information commonly used for the words not recorded in the word dictionary information, and in the fourth process, the learning means is described. Machine learning is performed on the trained target model based on the first word identification information and the second word identification information converted in the third process.

The information processing program according to claim 8 is a trained target model generated by performing machine learning on one or more words, and the trained target model that outputs numerical information indicating the characteristics of the words. An information processing program for performing machine learning again, and a word that is an unknown word that is not the target of machine learning in the trained target model and is uniquely identified by at least the trained target model. The acquisition means for acquiring the unknown word, which is a word not recorded in the word dictionary information stored in the storage means for storing the word dictionary information in which the information is recorded , from the first sentence information, and the first sentence. Using the second sentence information related to the information, the trained target model is made to function as at least a learning means for performing machine learning on the unknown word acquired by the acquisition means, and the learning means is the acquisition means. The first process of newly recording the unknown word acquired by the user in the word dictionary information, the second process of dividing the sentence included in the second sentence information into words, and the word stored in the storage means. Based on the third process of converting the words divided in the second process into word identification information for identifying each word based on the dictionary information, and the word identification information converted in the third process. The fourth process of performing machine learning on the trained target model is performed.

According to the information processing system according to claim 1 and the information processing program according to claim 8 , for example , the trained target model can be updated, so that the processing performed based on the trained target model can be performed. It is possible to improve the processing accuracy. Further, for example, since a newly trained target model is not generated, machine learning can be performed using a relatively small amount of second sentence information, so that the time for performing the machine learning can be shortened. Is possible.

According to the information processing system according to claim 6 , the first sentence information is information indicating a sentence corresponding to the title of the news article, and the second sentence information is information indicating the sentence corresponding to the text of the news article. As a result, for example, machine learning can be appropriately performed for unknown words, so that it is possible to improve the processing accuracy of the processing performed based on the trained target model.

Claims (9)

It is a trained target model generated by performing machine learning on one or more words, and is an information processing system for performing machine learning again on the trained target model that outputs numerical information indicating the characteristics of the words. There,
An acquisition means for acquiring unknown words that are not the target of machine learning in the trained target model from the first sentence information, and
A learning means for performing machine learning on the unknown word acquired by the acquisition means at least for the trained target model using the second sentence information.
Information processing system equipped with. The learning means is
Based on a predetermined determination criterion, it is determined whether or not the unknown word acquired by the acquisition means is targeted for machine learning.
Machine learning for the unknown word determined to be the target of machine learning among the unknown words acquired by the acquisition means is performed on the trained target model.
The information processing system according to claim 1. The learning means determines whether or not the unknown word acquired by the acquisition means is subject to machine learning based on the criteria regarding the part of speech of the unknown word acquired by the acquisition means.
The information processing system according to claim 2. The learning means machine-learns the unknown word acquired by the acquisition means based on the criteria regarding the frequency of appearance of the unknown word acquired by the acquisition means in the first sentence information or the second sentence information. Judging whether or not to be the target of
The information processing system according to claim 2 or 3. The acquisition means identifies an important word or an important phrase in the first sentence information based on a predetermined specific criterion, and based on the specified important word or the important phrase, the unknown word from the first sentence information. To get,
The information processing system according to any one of claims 1 to 4. A storage means for storing word dictionary information in which a word uniquely identified in the trained target model is recorded is provided.
The acquisition means acquires words other than the words recorded in the word dictionary information among the words included in the first sentence information as the unknown words.
The information processing system according to any one of claims 1 to 5. The learning means is
The unknown word acquired by the acquisition means is newly recorded in the word dictionary information, and the unknown word is newly recorded in the word dictionary information.
Using the second sentence information, machine learning is performed on the trained target model at least for the unknown word newly recorded in the word dictionary information.
The information processing system according to claim 6. The first sentence information is information indicating a sentence corresponding to the title of a news article.
The second sentence information is information indicating a sentence corresponding to the body of a news article.
The information processing system according to any one of claims 1 to 7. It is a trained target model generated by performing machine learning on one or more words, and is an information processing program for performing machine learning again on the trained target model that outputs numerical information indicating the characteristics of the words. There,
Computer,
An acquisition means for acquiring unknown words that are not the target of machine learning in the trained target model from the first sentence information, and
A learning means for performing machine learning on the unknown word acquired by the acquisition means at least for the trained target model using the second sentence information.
An information processing program that functions as.

Images