JP3725470B2 - Corpus processing apparatus, method, and program for creating statistical language model - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a corpus processing apparatus, method, and program for creating a statistical language model that prevents leakage of personal information and the like that should be kept secret during speech recognition processing.
[0002]
[Prior art]
In recent years, a Japanese dictation system that can input Japanese sentences by voice has been put into practical use. This system achieves high recognition performance by using not only the acoustic correctness but also information on the appearance probability of a word set called a statistical language model. Here, the appearance probability of the word group constituting the statistical language model is obtained by aggregating statistical information using a statistical technique for a large amount of text called a corpus.
[0003]
Therefore, the appearance probability of the word set in the statistical language model strongly depends on the content of the sentence in the corpus (text) used to create the language model. That is, in the dictation system, it is easy to recognize an expression that appears in a sentence in a text used to create a statistical language model, and conversely, an expression that does not appear is difficult to recognize. For this reason, when a user inputs a sentence that does not appear in the text used to create the statistical language model, the user may misrecognize a notation string that is acoustically close to a word set that appears in the text. Not a few.
[0004]
These techniques are described in detail in the following documents.
[0005]
"Speech recognition using probabilistic models" by Seiichi Nakagawa IEICE {ISBN4-88552-072-X}
“Spoken Language Processing” by Kenji Kitamori Morikita Publishing {ISBN4-627-82380-0}
"Probabilistic language model" by Kenji Kita University of Tokyo Press {ISBN4-13-065404-7}
[0006]
[Problems to be solved by the invention]
In this way, all the expressions appearing in the corpus (text) are directly reflected in the appearance probability of the word set of the statistical language model. Therefore, in the dictation system, expressions that appear in the corpus (text) are likely to appear as recognition results.
[0007]
For this reason, if the corpus (text) used to create the statistical language model contains private personal information, etc., the content of the personal information may appear as a misrecognition result. Information may be revealed to others.
[0008]
For example, when performing a medical speech recognition process, recognition accuracy can be improved by using a medical corpus. However, personal information is often included in a medical corpus, such as a medical record, and there is a problem that personal information may be revealed depending on the result of voice recognition processing.
[0009]
The present invention eliminates notation information relating to confidential information in text used for statistical information aggregation in the process of collecting text for a statistical language model or in the process of creating a statistical language model from the collected text. Another object of the present invention is to provide a corpus processing apparatus, method, and program for creating a statistical language model that can prevent private personal information from being erroneously output in a recognition result.
[0010]
[Means for Solving the Problems]
A corpus processing apparatus for creating a statistical language model according to claim 1 of the present invention is for masking words included in confidential information, and a text analysis unit that morphologically analyzes text data and outputs a morphological analysis result. In accordance with the predetermined mask rule, the confidential information masking part for masking the morphological analysis result, the corpus integrating part for collecting the morphological analysis result masked by the confidential information masking part as a corpus, and the corpus integrating part It has a corpus statistics totaling unit that collects statistical information from the corpus,
A corpus processing device for creating a statistical language model according to claim 2 of the present invention includes: a text analysis unit that performs morphological analysis on text data and outputs a morpheme analysis result; A confidential information replacement unit that replaces the morphological analysis result according to a predetermined replacement rule for replacing the morpheme analysis result, a corpus integration unit that accumulates the morphological analysis result replaced by the confidential information replacement unit as a corpus, and the corpus integration unit And a corpus statistics totaling unit that collects statistical information from the corpus accumulated by.
[0011]
In the first aspect of the present invention, the result of morphological analysis by the text analysis unit is given to the confidential information mask unit. The confidential information masking unit masks the morphological analysis result according to a predetermined mask rule for masking words included in the confidential information. The corpus accumulation unit accumulates the morphological analysis results in which words included in the confidential information are masked as a corpus. The corpus statistics totaling unit collects statistical information based on the accumulated corpus. The confidential information is masked in the statistical information, and the confidential information is prevented from leaking out in the voice recognition process using the statistical information.
[0012]
In claim 2 of the present invention, the morphological analysis result by the text analysis unit is given to the confidential information replacement unit. The secret information replacement unit replaces the morphological analysis result according to a predetermined replacement rule for replacing a word included in the secret information with another word. The corpus accumulation unit accumulates, as a corpus, morphological analysis results in which words included in the confidential information are replaced. The corpus statistics totaling unit collects statistical information based on the accumulated corpus. In the statistical information, the confidential information is replaced with another word, and the confidential information is prevented from leaking out in the speech recognition processing using the statistical information.
[0013]
Note that the present invention relating to an apparatus is also established as an invention relating to a method.
[0014]
Further, the present invention relating to the apparatus is also realized as a program for causing a computer to execute processing corresponding to the present invention.
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a corpus processing apparatus for creating a statistical language model according to the first embodiment of the present invention.
[0016]
In this embodiment, in the process of collecting text for a statistical language model, notation information relating to information to be concealed (hereinafter referred to as concealment information) such as personal information in the text used for statistical information aggregation is removed in advance. This prevents private personal information or the like from being erroneously output in the recognition result of the dictation system or the like.
[0017]
Various data are stored in the database 100. For example, in medical relations such as hospitals, information stored in the database 100 may be information such as medical records and radiographic results. The information in the database 100 includes sentence data including text necessary for creating a statistical language model. In the text data, information that should be kept secret, for example, notation information about personal information may be described.
[0018]
The text extraction unit 101 extracts text information that can be used as a corpus from the database 100 in which data is stored, and supplies the text information to the text analysis unit 102.
[0019]
The text analysis unit 102 outputs the result of morphological analysis of the text extracted by the text extraction unit 101. For example, the text analysis unit 102 divides a sentence in the text into words, and obtains a morphological analysis result including a combination of the divided word (heading) and its part of speech. Note that the morpheme analysis method used here may be any known method that divides text into morphemes composed of combinations of headings and parts of speech.
[0020]
The analysis result of the text analysis unit 102 is supplied to the confidential information mask unit 103. The confidential information mask unit 103 reads the mask rule stored in the storage unit 104, and masks the morphological analysis result from the text analysis unit 102 according to the read mask rule. The mask rule describes information for deleting confidential information such as personal information when creating a corpus.
[0021]
FIG. 2 is an explanatory diagram for explaining the mask rules stored in the storage unit 104.
[0022]
In the example of FIG. 2, the mask rules are stored as information on pairs of parts of speech and headings. FIG. 2 shows an example in which two records are accumulated. * (Asterisk) in FIG. 2 indicates that there are no restrictions on all headings, that is, all words included in the part of speech are masked. The example of FIG. 2 shows a process in which the part of speech masks all headings (words) of the person name: surname and the part of speech masks all headings (words) of the person name: name.
[0023]
For example, by describing “Tanaka” instead of an asterisk in the heading column, it is possible to mask only the word “Tanaka” in the part of speech: surname and surname.
[0024]
In the example of FIG. 2, the mask rule described based on the morpheme part-of-speech information is stored in the storage unit 104. However, the mask rule can be expressed by a headline or a combination of a headline and a part-of-speech. It is.
[0025]
The confidential information mask unit 103 outputs the morphological analysis result masked according to the mask rule to the corpus accumulating unit 105. The corpus accumulating unit 105 gives the input post-mask morpheme analysis result to the storage unit 106 and accumulates it as a confidential information excluded corpus for creating a statistical language model. In this way, the corpus from which the confidential information is removed according to the mask rule is stored in the storage unit 106. The corpus format stored in the storage unit 106 may be a morphological analysis result or may be a simple text format.
[0026]
The corpus statistics totaling unit 107 reads out the corpus accumulated in the storage unit 106, creates a statistical language model, and outputs the statistical language model to the storage unit 108. Note that any of the known methods may be used as the method by which the corpus statistics totaling unit 107 creates a statistical language model (N-gram) from the corpus. The storage unit 108 stores a statistical language model.
[0027]
By using the statistical language model stored in the storage unit 108, voice recognition processing is performed. In this case, since the confidential information is not included in the corpus from which the statistical language model is created, it is unlikely that the confidential information stored in the database 100 will appear in the speech recognition result.
[0028]
Next, the operation of the embodiment configured as described above will be described with reference to FIGS. 3 to 5.
[0029]
Now, description will be made assuming that medical-related information is stored in the database 100. The text extraction unit 101 extracts text from various information stored in the database 100 and outputs it to the text analysis unit 102. The text analysis unit 102 obtains a morpheme analysis result including a combination of a headline and its part of speech from the text by morphological analysis.
[0030]
Now, it is assumed that a part of the text extracted by the text extraction unit 101 is as shown in FIG. FIG. 3 shows that in the extracted text, there is a notation indicating that Ichiro Nihon is a stomach cancer, Jiro Jiro is a stomach ulcer, and Saburo Nihon is a lung cancer.
[0031]
In this case, the text analysis unit 102 obtains the analysis result shown in FIG. 4, for example. That is, as shown in FIG. 4, each extracted word string in FIG. 3 is divided into words and given their parts of speech. For example, “Nichiro Ichiro: Stomach Cancer” in FIG. 3 is composed of four words: Japan / Ichiro /: / Stomach Cancer. “Japan” is a part of speech with a person name: surname, and “Ichiro” is a part of speech with a person name: name. ":" Is a part of speech, and "stomach cancer" is a noun.
[0032]
The analysis result of the text analysis unit 102 is given to the confidential information mask unit 103. The confidential information mask unit 103 reads the mask rule from the storage unit 104. For example, it is assumed that the rule shown in FIG. 2 is described as the mask rule. In this case, the confidential information masking unit 103 masks all words of the person name: surname and part of speech of the person name: name from the analysis result of the text analysis unit 102 and masks the morpheme after masking. The analysis result is output to the corpus accumulating unit 105.
[0033]
For example, assuming that the word masked by the secret information mask unit 103 according to the mask rule is represented by a circle or a black circle, the secret information mask unit 103 outputs the morphological analysis result shown in FIG.
[0034]
As is clear from the comparison between FIG. 5 and FIG. 3 and FIG. 4, the part of speech of the morphological analysis results is masked by ○ or (black circle) for the person name: surname and person name: name word, respectively, and the confidential information mask part 103 is output. That is, in the example of FIG. 3, personal information to be concealed, such as a person's name, among medical-related information included in a hospital chart or the like is prevented from being output as a morphological analysis result.
[0035]
The corpus accumulating unit 105 causes the storage unit 106 to store the morphological analysis result masked by the confidential information masking unit 103. In this way, the corpus from which the confidential information is excluded is stored in the storage unit 106. The corpus statistics totaling unit 107 creates a statistical language model based on the accumulated corpus and stores it in the storage unit 108.
[0036]
Speech recognition processing is performed using the statistical language model stored in the storage unit 108. The statistical language model of the storage unit 108 is created based on a corpus from which confidential information is excluded, and the confidential information is prevented from appearing in the speech recognition result.
[0037]
As described above, in the present embodiment, since confidential information is excluded when creating a corpus based on text extracted from a database, when the created statistical language model is used, the confidential information is excluded. It is possible to prevent information from appearing in the speech recognition result. That is, even when using a database of data such as personal information that must be strictly managed in terms of security for contract and legal reasons, the leakage of confidential information can be completely prevented.
[0038]
FIG. 6 is a block diagram showing a second embodiment of the present invention. In FIG. 6, the same components as those in FIG.
[0039]
In the first embodiment, the confidential information is masked to prevent leakage of the confidential information. As shown in FIG. 5, the original name portion is masked, and the part of speech information is also deleted. That is, since all information of confidential information such as names including part-of-speech information is not used as a morphological analysis result, it may be difficult to accurately convert names and the like during speech recognition processing. Therefore, in the present embodiment, confidential information is used as a morphological analysis result by replacing it with other typical words.
[0040]
This embodiment is different from the first embodiment in that a secret information replacement unit 110 and a storage unit 111 are employed instead of the secret information mask unit 103 and the storage unit 104, respectively.
[0041]
The storage unit 111 stores replacement rules. The secret information replacement unit 110 replaces the word of the secret information in the morpheme analysis result from the text analysis unit 102 with another typical word according to the replacement rule of the storage unit 111 and outputs it to the corpus accumulation unit 105. Yes.
[0042]
Next, the operation of the embodiment configured in this way will be described with reference to the explanatory views of FIGS. 7 and 8 correspond to FIGS. 2 and 4, respectively.
[0043]
FIG. 7 shows an example of the replacement rule stored in the storage unit 111. In the example of FIG. 7, the replacement rule is stored as information on a part of speech, a headline, and a typical word set. FIG. 7 shows an example in which two records are accumulated. Also in FIG. 7, * (asterisk) indicates that there are no restrictions on all headings, that is, all words included in the part of speech are masked. Moreover, the typical word of FIG. 7 has shown the word to replace when the word of a morphological analysis result is the word prescribed | regulated by the part of speech and the headline.
[0044]
In the example of FIG. 7, for example, all headings (words) whose part of speech is personal name: surname are replaced with the word “Suzuki” and the part of speech is represented as personal name: surname. In FIG. 7, the replacement rule based on the morpheme part-of-speech information is described.
[0045]
Now, description will be made assuming that a part of the text extracted by the text extraction unit 101 is as shown in FIG.
[0046]
The morpheme analysis result in this case is, for example, as shown in FIG. The secret information replacement unit 110 reads the replacement rule from the storage unit 111. When the replacement rule is as shown in FIG. 7, the secret information replacement unit 110 replaces all words whose part of speech is personal name: surname in the analysis result of the text analysis unit 102 with “Suzuki” and replaces the part of speech with the personal name. : Last name, part of speech is personal name: replace all words of name: “Taro”, the part of speech is personal name: name, and the morphological analysis result after replacement is output to corpus accumulation unit 105.
[0047]
Thus, the secret information replacement unit 110 outputs the morphological analysis result shown in FIG. As is clear from the comparison between FIG. 4 and FIG. 8, the part of speech in the morphological analysis results is replaced with the typical words “Suzuki” and “Taro” for the personal name: surname and personal name: name words. The secret information replacement unit 110 outputs the information. That is, for example, personal information to be concealed, such as a person's name, among medical-related information included in hospital charts, etc., is not the original person name, but is replaced with a typical person name such as `` Taro Suzuki ''. Personal information is prevented from being output as a morphological analysis result. Also, the part-of-speech information can be used in the speech recognition process because the same part-of-speech information as the word before the replacement is added to the word after the replacement.
[0048]
Other operations are the same as those in the first embodiment.
[0049]
As described above, in the present embodiment, the confidential information is replaced with typical words and output as a morphological analysis result. Therefore, even when the created statistical language model is used, the confidential information is sounded. It can be prevented from appearing in the recognition result. Moreover, since the part-of-speech information is not lost even when the word is replaced, it is possible to perform processing using the part-of-speech information in the speech recognition processing using the statistical language model.
[0050]
FIG. 9 is a block diagram showing a third embodiment of the present invention. In FIG. 9, the same components as those of FIG.
[0051]
In the second embodiment, leakage of confidential information is prevented by replacing the confidential information with typical words. Similarly, in the present embodiment, leakage of confidential information is prevented by replacing words of the confidential information with appropriate hidden characters. When replacing, the same part-of-speech information as the part-of-speech information before replacement is added to the hidden character.
[0052]
This embodiment is different from the second embodiment in that a storage unit 121 is used instead of the storage unit 111. The storage unit 121 stores a replacement rule for replacing with a hidden character having part-of-speech information.
[0053]
Next, the operation of the embodiment configured as described above will be described with reference to the explanatory diagrams of FIGS. 10 to 13 correspond to FIGS. 2 to 5, respectively.
[0054]
FIG. 10 shows an example of the replacement rule stored in the storage unit 121. In the example of FIG. 10, the replacement rule is stored as information on a set of parts of speech, headings, and mask words. FIG. 10 shows an example in which 11 records are accumulated. In FIG. 10, * (asterisk) means all words, for example, * prefecture indicates all headings whose last character is “prefecture”. The mask word in FIG. 10 indicates a word to be replaced with a covert character when the word of the morphological analysis result is a word defined by the part of speech and the heading.
[0055]
In the example of FIG. 10, for example, all headings (words) whose part of speech has “prefecture” as the last character in the place name are replaced with the word “□ prefecture” and the part of speech is used as the place name. . Note that the replacement rules in FIG. 10 are executed in order from the top of the record, for example, and for example, in the upper record such as * city, * road,. The described rule is executed first.
[0056]
In FIG. 10, the replacement rule based on the morpheme part-of-speech information is described, but it can be expressed by a heading or a combination of a heading and a part-of-speech.
[0057]
Now, description will be made assuming that a part of the text extracted by the text extraction unit 101 is as shown in FIG. FIG. 11 shows that in the extracted text, the address of Ichiro Nihon is “1-2-345 Shibaura, Minato-ku, Tokyo” and the address of Jiro Nihon is “6-7 Komukai, Saiwai-ku, Kawasaki City, Kanagawa Prefecture”. This indicates that there was a notation indicating that the address of Saburo Nihon-san was “9-1, Yukimachi, Saiwai-ku, Kawasaki-shi, Kanagawa”.
[0058]
In this case, the text analysis unit 102 obtains an analysis result shown in FIG. 12, for example. That is, as shown in FIG. 12, each extracted word string in FIG. 11 is divided into words, and their parts of speech are attached. For example, “Nihon Ichiro: Shibaura 1-2-345 Minato-ku, Tokyo” in FIG. 11 is 12 pieces in Japan / Ichiro /: / Tokyo / Minato-ku / 1 / − / 2 / − / 3/4/5. “Japan” is part of the name: surname, “Ichiro” is part of speech: name, name: “:” is part of speech, “Tokyo” is part of speech, and “Minato” Is part of speech, "1" is part of speech, "-" is part of speech, "2" is part of speech, "-" is part of speech, and "3" is part of speech. , “4” indicates that the part of speech is a number, and “5” indicates that the part of speech is a number.
[0059]
These morphological analysis results are given to the confidential information replacement unit 110. The secret information replacement unit 110 reads the replacement rule from the storage unit 121 and replaces the secret information according to the replacement rule. When the replacement rule is as shown in FIG. 10, the secret information replacement unit 110 replaces all the words whose part of speech is personal name: surname in the analysis result of the text analysis unit 102 with “○”. ： Last name, part of speech is personal name: Replace all words in the name with “(Kuromaru)”. Replace the part of speech with personal name: name, replace all words with part of speech in the number with “◇”, and change the part of speech into numbers. The place name is also replaced according to the replacement rule of FIG. Thus, the secret information replacement unit 110 outputs the morphological analysis result shown in FIG.
[0060]
As is clear from the comparison between FIG. 12 and FIG. 13, the words specified by the replacement rule in the morphological analysis results are replaced with characters of ○, (black circle), ◇, (black diamond), or □, (black squares) and words replaced with the words “city”, “road”, “fu”, “prefecture”, “city”, “ku”, “town”. For example, the address of “Jiro Jiro” in FIG. 12, “6-7-8 Komukai Koyuki, Kawasaki City, Kanagawa Prefecture” is “□ prefecture (black square) city (black square) ward (black rhombus) ◇ − Replaced by “◇-◇◇◇”. That is, for example, personal information to be concealed, such as personal names and place names, among medical information included in hospital charts, etc., is replaced with a covert character with part-of-speech information added. Output is prevented. Moreover, since the part of speech information that is the same as the word before the replacement is added to the concealed character, the part of speech information can be used in the speech recognition processing.
[0061]
Other operations are the same as those of the second embodiment.
[0062]
Thus, also in this embodiment, the same effect as that of the second embodiment can be obtained.
[0063]
FIG. 14 is a block diagram showing a fourth embodiment of the present invention. In FIG. 14, the same components as those of FIG.
[0064]
In the second and FIG. 3 embodiments, the confidential information is prevented from being leaked by replacing the confidential information with predetermined typical words or hidden characters. However, since the word of confidential information is replaced with a hidden word or a typical word, the morphological analysis result does not include personal names, etc., and when using a statistical language model created based on this morphological analysis result In some cases, it may be difficult to correctly perform voice recognition processing of personal names and the like. Therefore, in this embodiment, by replacing the word of the confidential information with a random word of the same part of speech, it is possible to prevent leakage of the confidential information and to correctly perform voice recognition processing even for an individual name or the like. Is.
[0065]
This embodiment is different from the second embodiment in that a secret information replacement unit 130 and a storage unit 131 are used instead of the secret information replacement unit 110 and the storage unit 111, respectively. The storage unit 131 stores a replacement rule for replacing confidential information with another random word of the same part of speech.
[0066]
Next, the operation of the embodiment configured as described above will be described with reference to FIGS. 15 to 18. 15 to 18 correspond to FIGS. 2 to 5, respectively.
[0067]
FIG. 15 shows an example of the replacement rule stored in the storage unit 131. In the example of FIG. 15, the replacement rule is stored as information on a set of parts of speech, headings, and random words. FIG. 15 shows an example in which 11 records are accumulated. In FIG. 15, * (asterisk) means all words. The random word in FIG. 15 indicates a replacement word when the word of the morphological analysis result is a word specified by the part of speech and the headline, and Rand () indicates a word extracted at random.
[0068]
In the example of FIG. 15, for example, the part of speech 0-9 indicates that the part of speech is replaced by a random number 0-9, which is a number. In FIG. 15, the replacement rule based on the morpheme part-of-speech information is described, but it can be expressed by a heading or a combination of a heading and a part-of-speech.
[0069]
Now, description will be made assuming that a part of the text extracted by the text extraction unit 101 is as shown in FIG. FIG. 16 shows that in the extracted text, some numerical value related to Mr. Ichiro Nihon is “1233-1234”, some numerical value related to Jiro Nihon is “123-4567”, and some numerical value related to Mr. Saburo Nihon is “ This indicates that there is a notation indicating “999-9999”. These numerical values are various information such as various measurement values, amounts, and ages.
[0070]
In this case, the text analysis unit 102 obtains an analysis result shown in FIG. 17, for example. That is, as shown in FIG. 17, each extracted word string in FIG. 16 is divided into words, and their parts of speech are attached. For example, “Nihon Ichiro: 123-1234” in FIG. 16 is made up of 12 words of Japan / Ichiro /: / 1/2/3 /-/ 1/2/3/4/5, and “Japan” is Part-of-speech is a person's first name, “Ichiro” is a part-of-speech name: name, “:” is a part-of-speech symbol, “1”-“3” is a part-of-speech number, “-” is a part-of-speech symbol, It can be obtained from the analysis result that the parts of speech of “1” to “4” are numbers.
[0071]
These morphological analysis results are given to the confidential information replacement unit 130. The secret information replacement unit 130 reads the replacement rule from the storage unit 131 and replaces the secret information according to the replacement rule. When the replacement rule is as shown in FIG. 15, the confidential information replacement unit 130 is the same in which all the words whose part of speech is personal name: surname and personal name: name are randomly extracted from the analysis result of the text analysis unit 102. The same part-of-speech is attached to other words in the part of speech, and for all words whose part-of-speech is a number, the part-of-speech extracted at random is replaced with another number. Thus, the secret information replacement unit 130 outputs the morphological analysis result shown in FIG.
[0072]
As is clear from a comparison between FIG. 17 and FIG. 18, words defined by the replacement rule in the morphological analysis results are replaced with other words of the same part of speech extracted at random. For example, “1233-1234” of “Nihon Ichiro” in FIG. 17 is replaced with “313-6924” of “Ryoko Suzuki”. That is, for example, personal information to be concealed, such as personal names and numerical values, among medical-related information included in hospital charts, etc., is replaced with other words of the same part of speech that are randomly extracted. Outputting as an analysis result is prevented. Moreover, since the word of the same part of speech extracted at random is used as the word after substitution, the morphological analysis result after substitution includes information such as a name and a numerical value. Therefore, when a statistical language model created based on the result of morpheme analysis is used, it is possible to reliably recognize names, numerical values, and the like during speech recognition processing.
[0073]
Other operations are the same as those of the second embodiment.
[0074]
As described above, also in this embodiment, the same effect as in the second and third embodiments can be obtained. Furthermore, since words of the same part of speech that are randomly extracted as replacement words are used, a statistical language model including information such as names and numerical values that are actually used can be created, resulting in reduced speech recognition accuracy. Can be prevented. Further, by extracting the replaced word from the database corresponding to the usage pattern, a statistical language model suitable for the usage pattern can be created, and the speech recognition accuracy can be further improved.
[0075]
FIG. 19 is a block diagram showing a fifth embodiment of the present invention. In FIG. 19, the same components as those in FIG.
[0076]
In the first to fourth embodiments, in the process of collecting the text for the statistical language model, the notation information related to the confidential information in the text used for statistical information aggregation is removed in advance. In contrast, in the present embodiment, in the process of creating a statistical language model from an existing corpus (text after collecting text for a statistical language model), notation information about confidential information in the text used for statistical information aggregation Is to be removed in advance.
[0077]
This embodiment is different from the embodiment of FIG. 1 in that a corpus 140 is used instead of the database 100 and a corpus input unit 141 is used instead of the text extraction unit 101.
[0078]
The corpus 140 is a collection of text collected for statistical language model creation. The corpus 140 may contain confidential information. The corpus input unit 140 outputs the text in the corpus 141 to the text analysis unit 102 with the corpus 141 as a processing target.
[0079]
In the embodiment configured as described above, text is extracted from an existing corpus 140 that has already been created and output to the text analysis unit 102. Thereby, the morphological analysis result obtained by masking the confidential information can be obtained and the corpus excluding the confidential information can be accumulated by the same method as in the first embodiment.
[0080]
Other operations are the same as those of the embodiment of FIG.
[0081]
As described above, in this embodiment, even when an existing corpus is used, it can be converted into a corpus that excludes confidential information, and speech recognition processing using a statistical language model without leaking confidential information. Can also be possible.
[0082]
The fifth embodiment can also be applied to the second to fourth embodiments. FIGS. 20 to 22 are block diagrams when the fifth embodiment is applied to the second to fourth embodiments shown in FIGS.
[0083]
20 to 22 are different from FIGS. 6, 9, and 14 in that a corpus 140 is used instead of the database 100 and a corpus input unit 141 is used instead of the text extraction unit 101.
[0084]
Other configurations and operations are the same as those of the second to fourth embodiments, respectively.
[0085]
As described above, also in the examples shown in FIGS. 20 to 22, the same effects as those of the first to fifth embodiments can be obtained.
[0086]
【The invention's effect】
As described above, according to the present invention, in the process of collecting the text for the statistical language model or the process of creating the statistical language model from the collected text, the personal information in the text used for statistical information aggregation By removing the written information, it is possible to prevent private personal information from being erroneously output in the recognition result.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a corpus processing device for creating a statistical language model according to a first embodiment of the present invention.
2 is an explanatory diagram for explaining mask rules stored in a storage unit 104 in FIG. 1; FIG.
FIG. 3 is an explanatory diagram for explaining the operation of the first embodiment;
FIG. 4 is an explanatory diagram for explaining the operation of the first embodiment.
FIG. 5 is an explanatory diagram for explaining the operation of the first embodiment;
FIG. 6 is a block diagram showing a second embodiment of the present invention.
7 is an explanatory diagram for explaining a replacement rule stored in a storage unit 111 in FIG. 6;
FIG. 8 is an explanatory diagram for explaining the operation of the second embodiment.
FIG. 9 is a block diagram showing a third embodiment of the present invention.
10 is an explanatory diagram for explaining a replacement rule stored in a storage unit 121 in FIG. 9;
FIG. 11 is an explanatory diagram for explaining the operation of the third embodiment.
FIG. 12 is an explanatory diagram for explaining the operation of the third embodiment.
FIG. 13 is an explanatory diagram for explaining the operation of the third embodiment.
FIG. 14 is a block diagram showing a fourth embodiment of the present invention.
15 is an explanatory diagram for explaining a replacement rule stored in a storage unit 131 in FIG. 14;
FIG. 16 is an explanatory diagram for explaining the operation of the fourth embodiment;
FIG. 17 is an explanatory diagram for explaining the operation of the fourth embodiment;
FIG. 18 is an explanatory diagram for explaining the operation of the fourth embodiment.
FIG. 19 is a block diagram showing a fifth embodiment of the present invention.
FIG. 20 is a block diagram showing a modification of the fifth embodiment.
FIG. 21 is a block diagram showing a modification of the fifth embodiment.
FIG. 22 is a block diagram showing a modification of the fifth embodiment.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 100 ... Database, 101 ... Text extraction part, 102 ... Text analysis part, 103 ... Confidential information mask part, 104,106,108 ... Memory | storage part, 105 ... Corpus accumulation | storage part, 107 ... Corpus statistics totaling part.

Claims (12)

A text analysis unit that morphologically analyzes text data and outputs a morphological analysis result;
According to a predetermined mask rule for masking a word included in confidential information, a confidential information masking unit that masks the morphological analysis result;
A corpus accumulation unit that accumulates the morphological analysis results masked by the confidential information mask unit as a corpus;
A corpus processing apparatus for creating a statistical language model, comprising: a corpus statistics totaling unit that collects statistical information from the corpus accumulated by the corpus accumulation unit. A text analysis unit that morphologically analyzes text data and outputs a morphological analysis result;
A secret information replacement unit that replaces the morphological analysis result according to a predetermined replacement rule for replacing a word included in the secret information with another word;
A corpus accumulation unit for accumulating the morphological analysis results replaced by the secret information replacement unit as a corpus;
A corpus processing apparatus for creating a statistical language model, comprising: a corpus statistics totaling unit that collects statistical information from the corpus accumulated by the corpus accumulation unit.   The corpus processing apparatus for creating a statistical language model according to claim 1, wherein the text data given to the text analysis unit is extracted from a predetermined database. .   The corpus processing device for creating a statistical language model according to claim 1, wherein the text data given to the text analysis unit is extracted from a predetermined corpus. .   The corpus processing device for creating a statistical language model according to claim 2, wherein the secret information replacement unit replaces a word included in the secret information with a typical word.   3. The corpus processing device for creating a statistical language model according to claim 2, wherein the secret information replacement unit replaces a word included in the secret information with a covert character to which part-of-speech information is added.   The corpus processing device for creating a statistical language model according to claim 2, wherein the secret information replacement unit replaces a word included in the secret information with a random word.   The mask rule or the replacement rule is a rule for masking or replacing a word having at least one part of speech of a person name, a place name, and a number as a word included in the confidential information. A corpus processing device for creating the statistical language model described in any one of 2 above. A corpus processing method for creating a statistical language model by a computer comprising a text analysis unit, a secret information mask unit, a corpus accumulation unit, and a corpus statistics totaling unit,
A text analysis procedure in which the text analysis unit morphologically analyzes text data and outputs a morphological analysis result;
The confidential information mask unit masks the morphological analysis result according to a predetermined mask rule for masking a word included in confidential information, and
The corpus accumulating unit for accumulating the morphological analysis results masked by the confidential information mask unit as a corpus;
6. A corpus processing method for creating a statistical language model, wherein the corpus statistics totaling unit comprises a corpus statistics totaling procedure for collecting statistical information from the corpus accumulated by the corpus accumulation unit . A corpus processing method for creating a statistical language model by a computer comprising a text analysis unit, a secret information replacement unit, a corpus accumulation unit, and a corpus statistics totaling unit,
A text analysis procedure in which the text analysis unit morphologically analyzes text data and outputs a morphological analysis result;
The secret information replacement unit replaces the morphological analysis result according to a predetermined replacement rule for replacing a word included in the secret information with another word, and
A corpus accumulation procedure in which the corpus accumulation unit accumulates, as a corpus, the morphological analysis result replaced by the confidential information replacement unit ;
The corpus statistic aggregation unit comprises a corpus statistic aggregation procedure for collecting statistical information from the corpus accumulated by the corpus accumulation unit . A corpus processing method for creating a statistical language model, comprising: In a computer having a text analysis unit, a confidential information mask unit, a corpus accumulation unit, and a corpus statistics totaling unit ,
Text analysis processing procedure for outputting morphological analysis results by performing morphological analysis on text data by the text analysis unit ,
By the secret information mask portion, according to a predetermined mask rule for masking the words contained in the confidential information, the confidential information masking procedure for masking the morphological analysis result,
A corpus accumulation processing procedure for accumulating the morphological analysis results masked by the confidential information mask unit as a corpus by the corpus accumulation unit;
A corpus processing program for creating a statistical language model for causing the corpus statistical totaling unit to execute a corpus statistical totaling processing procedure for collecting statistical information from the corpus accumulated by the corpus accumulating unit . In a computer having a text analysis unit, a secret information replacement unit, a corpus accumulation unit, and a corpus statistics totaling unit ,
Text analysis processing procedure for outputting morphological analysis results by performing morphological analysis on text data by the text analysis unit ,
A secret information replacement processing procedure for replacing the morphological analysis result according to a predetermined replacement rule for replacing a word included in the secret information with another word by the secret information replacement unit;
Corpus integration processing procedure for integrating the morphological analysis results replaced by the confidential information replacement unit as a corpus by the corpus integration unit;
A corpus processing program for creating a statistical language model for causing the corpus statistical totaling unit to execute a corpus statistical totaling processing procedure for collecting statistical information from the corpus accumulated by the corpus accumulating unit .

Images