JP2007193350A - Speech correction apparatus, speech correction method and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech recognition apparatus and the method thereof for correctly recognizing an English word from non-native English pronunciation. <P>SOLUTION: A vector data generating part 180 and a label generating part 182 processes speech data of one sentence English speech pronounced by a Japanese to convert it to a label string. A candidate word generating part 184 correlates the label string of one sentence to a first candidate word comprising one or more English words. An analogous word adding part 186 uses a word database 160 to search an English word analogous to the pronunciation of the first candidate word, such as an analogous word "lead" for a first candidate word "read", for example, (it is difficult for a Japanese to discriminate between "l" and "r" in pronunciation), and adds the obtained analogous word to the first candidate word to make it be a second candidate word. A selection part 188 selects one of the second candidate words as a final result of recognition in response to users operation and connects the selected words into English text data for output. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

The present invention relates to a speech identification device and method for identifying each word of the latter language included in the speech of another specific language by a speaker whose native language is a specific language, for example, an English speech spoken by a Japanese The present invention relates to a voice identification device and method for outputting the data as text (text data) indicating a sequence of English words included in the voice.

The present invention also relates to a pronunciation correcting apparatus and method for correcting correct pronunciation by teaching a speaker of correct pronunciation using data (candidate word data) obtained in the process of the speech identification apparatus and method.

  A voice identification device that identifies each word included in the voice spoken by an unspecified speaker and outputs it as text data is used. For example, Japanese Patent Laid-Open Nos. 06-12483, 08-50493, and 09-22297 (references 1 to 3) disclose such voice identification methods.

  For example, using these conventional speech identification methods and creating English text data from English speech pronounced by Japanese using an English speech recognition device that identifies English words from English speech will result in a decrease in the recognition rate. . In English, there are sounds that do not exist in Japanese (th etc.) or sounds that are difficult to distinguish in Japanese (l, r, etc.). This is because the English speech recognition device replaces an incorrect pronunciation with a word as it is. For example, even if a Japanese intends to pronounce “rice” in English, an English voice identification device may identify this voice as “lice” or “louse”.

  Contrary to the example shown above, this problem occurs when an American whose native language is English uses a speech recognition device that creates Japanese text from Japanese speech, or an English- It occurs in various cases, such as when an English-speaking British uses a voice recognition device adjusted for American English, or when a particular person is difficult to pronounce correctly for some reason. sell. However, none of the voice identification methods disclosed in the above documents can solve such a problem.

  Here, if the speaker's English pronunciation improves and comes closer to the native speaker, it goes without saying that the recognition rate by the voice identification device is improved, and it is preferable for the speaker to improve English conversation.

  For example, Japanese Patent Laid-Open No. 4-54956 discloses a learning device that identifies a speaker's English speech and allows the speaker to confirm the identified English speech (Reference 4). Further, for example, Japanese Patent Laid-Open No. 60-123984 discloses an English language learning machine that uses a voice synthesis LSI to let a speaker hear a voice to be learned (Reference 5). In addition, JP-B-44-7162, JP-A-7-117807, JP-A-61-1868, JP-A-8-27588, JP-A-62-211278, JP-A-62-299985 No. 3, 75,869, JP-B-6-27971, JP-B-8-12535, and JP-A-3-226785, for learning pronunciation of foreign languages A learning device is disclosed (References 6 to 15).

However, even with the learning devices disclosed in these documents, it is necessary to compare the proposed pronunciation with your own pronunciation, or because you do not know what is wrong with your pronunciation. A person cannot always obtain a sufficient learning effect.
Japanese Patent Laid-Open No. 06-12383 Japanese Patent Laid-Open No. 08-50493 JP 09-22297 A Japanese Patent Laid-Open No. 4-54956 JP-A-60-123848 Japanese Patent Publication No. 44-7162 JP-A-7-117807 JP 61-18068 A JP-A-8-27588 JP-A-62-111278 JP-A-62-299985 JP-A-3-75869 Japanese Patent Publication No. 6-27971 Japanese Patent Publication No.8-12535 JP-A-3-226785

  The present invention has been made in view of the above-described problems of the prior art, and identifies each word included in the spoken language of a predetermined language by a speaker who is not a native language of the predetermined language (non-native), It is an object of the present invention to provide a speech identification apparatus and method capable of creating accurate text data by replacing words in a predetermined language intended by a speaker.

  In addition, the present invention provides a word intended by a speaker, regardless of whether the pronunciation of the same language has changed due to differences in the spoken region, etc. It is an object of the present invention to provide a voice identification apparatus and method capable of generating accurate text data by converting the data into the above. It is another object of the present invention to provide a voice identification apparatus and method that can compensate for individual differences in pronunciation and always maintain a high identification rate.

  Furthermore, the present invention points out the problem of the speaker's pronunciation by using the data obtained in the process of the voice identification device and method, and allows the speaker to learn the pronunciation of the native speaker. An object of the present invention is to provide a pronunciation correcting apparatus and method for correcting pronunciation. In addition, the present invention can automatically compare the pronunciation of the speaker with the correct pronunciation to point out an error, and further presents detailed information indicating how the speaker should correct the pronunciation An object of the present invention is to provide a pronunciation correcting apparatus and method that can correct the pronunciation.

[First voice identification device]
In order to achieve the above object, a first speech identification device according to the present invention is a speech identification device for identifying each of the words from speech data indicating one or more words included in a spoken voice. Candidate word associating means for associating one or more sets of one or more candidate word combinations (candidate words) obtained by identifying one or more of these voice data with each of the voice data of the words. And 0 or more sets of one or more combinations of words (similar words) that can correspond to the pronunciation of each candidate word are associated with each of the candidate words associated with each of the speech data of the one or more words. Similar word association means, the candidate word associated with each of the one or more voice data of the word, and the similar unit associated with each of the candidate words Select one of the, and an audio data identification means for the identification result of each audio data of the word.

  Preferably, the voice data indicates one or more words included in a spoken language of a predetermined language, and the candidate word associating means assigns each of the voice data of the one or more words to the voice data of these voice data. One or more sets of candidate words of the predetermined language obtained by identifying each one or more are associated, and the similar word associating means associates the candidate words associated with each of the one or more speech data of the words Each candidate is associated with 0 or more pairs of similar words in the predetermined language that can correspond to the pronunciation of each of the candidate words, and the speech data identification means associates each candidate with speech data of one or more of the words Either a word or the similar word associated with each of the candidate words is selected and used as an identification result for each of the voice data of one or more of the words.

  Preferably, the pronunciation of the voice of the predetermined language is made by a speaker who mainly speaks a predetermined language other than the predetermined language, and each of one or more words of the predetermined language is pronounced by the speaker. In this case, zero or more sets of words of the predetermined language that can correspond to one or more of the voice data of the words included in the spoken language of the predetermined language, respectively, one or more of the words of the predetermined language And similar word storage means for storing one or more words in the predetermined language as the similar words, and the similar word associating means includes one or more words in the predetermined language in advance. Each of the candidate words is associated with zero or more sets of the similar words stored in association with each of the candidate words.

  Preferably, the candidate word associating means attaches probability data indicating the probability of each of the candidate words associated with the speech data to each of the candidate words associated with the speech data, and the speech data identifying means Selects only the candidate words for which the value of the probability data falls within a predetermined range, and uses it as the identification result of the speech data of the words.

  Preferably, the candidate word associating means attaches error information indicating a pronunciation error corresponding to each of the similar words to each of the candidate words associated with speech data.

[Operation of voice recognition device]
The speech recognition apparatus according to the present invention uses a language other than a specific language (hereinafter, English is exemplified in this item) as a native language (similarly, Japanese is exemplified), and mainly speaks (similarly, Japanese). A word included in an English spoken voice (speech) pronounced by (example) is identified and replaced with an English word to create text data.

  In the speech identification device according to the present invention, English speech (speech data) by a Japanese input from a microphone or the like and converted into digital data is, for example, for each sound feature (sound pitch, strength, intonation, etc.). Is further converted into vector data quantized to, further referred to as a label, converted into sound data similar to a phonetic symbol, and output to candidate word association means.

[Candidate word association means]
The candidate word associating means processes the voice data converted into the label for one word or a series of a plurality of words, and sets the voice data and the identification result of the voice data as a single (one) candidate. An English word or a combination of English words (collectively referred to as candidate words) is associated.

[Similar word storage means]
Similar word storage means different from accurate English pronunciation, but when a Japanese pronounces English, it can be a single English word or a combination of multiple English words that can correspond to speech data (collectively referring to these words) For example, search dictionary data is stored in which a single English word that can be a candidate word or a combination of a plurality of English words is associated in advance. For example, in this dictionary data, in order to cope with inaccurate English pronunciation by Japanese, a single English word "lead" that can be a candidate word is used, and the similar word "read" It is difficult to take into account l and r. Generally, Japanese is not good at pronunciation of "r"). Note that similar words may not exist for English words. In such a case, similar words are not associated with English words in this dictionary.

[Similar word matching means]
The similar word association means searches the dictionary data stored in the similar word storage means, reads out the similar words associated with the candidate words, and associates the similar words with the candidate words. In the above example, the English word “lead” and the similar word “read” are associated with the voice data corresponding to the English word “read” pronounced by the Japanese.

[Voice data identification means]
The voice data identifying means is, for example, one of a candidate word and a similar word associated with the voice data based on a parsing process for an English word string identified so far or according to a user's selection operation. Is selected as the identification result.

  Each component of the speech identification device according to the present invention sequentially performs the processes described so far on the sequentially input speech data, identifies the English words included in the speech data, and identifies the identified English words Create text data that connects

  In the above description, the English voice by the Japanese has been exemplified. However, when the similar word storage means is different from the American English pronunciation, the voice data By adding a change such as storing dictionary data in which similar words that can correspond to candidate words are stored, the speech identification device according to the present invention is based on English speech by English pronunciation and American pronunciation. Both English speech can be identified and text data can be generated.

[Specified language range]
As described above, the range of the “predetermined language” is defined as a range in which the candidate word association unit can associate the speech data and the word with a sufficient identification rate. Therefore, for example, even in the same language, pronunciations may differ from each other due to regional isolation, and a sufficient identification rate can be obtained with only candidate word association means adjusted to suit one. A plurality of dialects that cannot be obtained (for example, English in countries such as the United States, the United Kingdom, Australia, and South Africa, and Spanish in countries such as Spain and Latin America) are not included in the same range of the “predetermined language”. In addition, for some reason, the pronunciation of a specific person becomes unclear, and the candidate word matching means adjusted to suit the native language (mainly spoken language) of this person cannot provide a sufficient identification rate. The same applies to the case.

[Second voice identification device]
Further, the second speech identification device according to the present invention is based on speech data indicating one or more words of the predetermined language included in the speech of the predetermined language by a speaker who mainly speaks a language other than the predetermined language. A speech identification device for identifying each of one or more words of the predetermined language, wherein each of the speech data of the words of the predetermined language is identified with one or more of these speech data. The obtained word of the predetermined language and the word association means for associating one or more of the words of the predetermined language that the speaker may have spoken or any of them, and the one or more of the words Voice data identifying means for selecting one of the words associated with each of the voice data and identifying each voice data of the one or more words.

[Voice identification method]
A first speech identification method according to the present invention is a speech identification method for identifying each of the words from speech data indicating one or more words included in a spoken voice, and the speech of the one or more words Associating each data with one or more sets of one or more candidate word combinations (candidate words) obtained by identifying one or more of these speech data, and the speech of the one or more words Associating each candidate word associated with each data with zero or more combinations of one or more words (similar words) that can correspond to the pronunciation of each candidate word; and One of the candidate word associated with each of the speech data and the similar word associated with each of the candidate words is selected, and the speech data of the word is selected. And a step of the identification result of, respectively.

  Further, the second speech identification method according to the present invention is based on speech data indicating one or more words of the predetermined language included in the speech of the predetermined language by a speaker who mainly speaks a language other than the predetermined language. A speech identification method for identifying each of one or more words of the predetermined language, wherein each of the speech data of the words of the predetermined language is identified with one or more of these speech data. Associating the obtained word in the predetermined language and / or one or more of the words in the predetermined language that the speaker may have spoken, and voice data of the one or more words Selecting any one of the words associated with each of them, and using each of the voice data of the one or more words as an identification result.

[Sound correction device]
In addition, the speech correction apparatus according to the present invention includes candidate word association means for associating one or more word candidates (candidate words) obtained by identifying speech data indicating words, and the candidates associated with speech data. Similar word associating means for associating each word with zero or more words (similar words) that can correspond to the pronunciation of each of the candidate words, the word indicated by the voice data, and the candidate associated with the voice data Pronunciation corresponding to the same similar word as that indicated by the audio data and outputting pronunciation correction data for correcting the pronunciation of the word indicated by the audio data when the similar words associated with each word match Correction data output means.

[Operation of pronunciation correction device]
In the pronunciation correcting apparatus according to the present invention, the candidate word associating means and the similar word associating means associate speech data with candidate words / similar words as in the above-described speech recognition apparatus according to the present invention.

[Pronunciation data output means]
If the speaker has a correct pronunciation close to native, the word intended by the speaker and the identification result of the voice data are included in the candidate word. On the other hand, if the speaker's pronunciation is incorrect or unclear, the word intended by the speaker is included in the candidate word, but the identification result of the speech data is included in the similar word. Therefore, when a word to be pronounced is shown to the speaker in advance and this word is pronounced, the fact that this word matches the similar word in the identification result of the voice data means that the pronunciation of the user (speaker) is something. It means that there is an error or ambiguity in pronunciation.

  When the word shown to the speaker matches a similar word, the pronunciation correction data output means is associated with the similar word that matches the word shown to the speaker, and corrects the pronunciation error / ambiguity ( For example, the monitor displays image data that shows the movement of the mouth and tongue when the native speaker makes correct pronunciation, and text data that shows in writing what is wrong in the speaker's pronunciation compared to the native speaker. In addition to prompting the speaker to correct pronunciation, learning is assisted so that the speaker's pronunciation approaches that of the native speaker.

[Voice correction method]
In addition, the speech correction method according to the present invention associates one or more word candidates (candidate words) obtained by identifying speech data indicating words, and each of the candidate words associated with speech data 0 or more words (similar words) that can correspond to the pronunciation of each candidate word are associated, the word indicated by the speech data, and the similar word associated with each of the candidate words associated with the speech data Is matched with the same similar word as the word indicated by the voice data, and pronunciation correction data for correcting the pronunciation of the word indicated by the voice data is output.

[First embodiment]
Hereinafter, a first embodiment of the present invention will be described. In the following, for the sake of clarity of explanation and convenience, a case where the voice identification processing according to the present invention is adjusted to identify English spoken by Japanese will be described as a specific example unless otherwise specified.

[Computer 1]
First, with reference to FIG. 1, a computer 1 that realizes a voice identification process according to the present invention will be described. FIG. 1 is a diagram exemplifying a configuration of a computer 1 that realizes voice identification processing according to the present invention. As illustrated in FIG. 1, the computer 1 is, for example, a personal computer having a voice input / output function, and includes a computer main body 10 including a CPU, a memory, and peripheral devices thereof, an output device 100, a magneto-optical (mo). It comprises a storage device 110 such as a disk device, hard disk device or floppy disk device, and an input device 120. The output device 100 includes a monitor 102 such as a CR display device, an audio output speaker 104, a printer 106, and the like. The input device 120 includes a microphone 122, a voice input board 124, a keyboard 126, a mouse 128, and the like.

[Monitor 102]
In the output device 100, the monitor 102 displays a GUI image for operation to the user of the computer 1, text data obtained by the computer main body 10 identifying voice, and the like.

[Speaker 104]
The speaker 104 is used for outputting text data obtained by the computer body 10 identifying voice as voice.

[Printer 106]
The printer 106 is used for outputting a hard copy of text data obtained by the computer main body 10 identifying voice.

[Storage device 110]
The storage device 110 operates according to control by the computer main body 10 and stores text data obtained by the computer main body 10 identifying voice. The storage device 110 stores data necessary for voice identification (hereinafter, such data is collectively referred to as “word data”), a program, and the like, and the stored word data and program are stored in the computer main body 10. Output. The word data stored in the storage device 110 is created by, for example, the voice identification program 16 or supplied by a recording medium such as a floppy disk, and includes a label string table, an index table, a word record, a similar word record, and an error. An information code table (these details excluding label column data will be described later with reference to FIGS. 4 to 7) is included.

[Microphone 122]
The microphone 122 collects the voice of the voice spoken by the user, converts it into an analog audio signal, and outputs it to the audio input board 124.

[Voice input board 124]
The voice input board 124 operates in accordance with the control by the computer main body 10, samples the voice signal input from the microphone 122, converts it into digital voice data corresponding to the waveform of the voice signal, and outputs it to the computer main body 10. Output.

[Keyboard 126, Mouse 128]
The keyboard 126 and the mouse 128, for example, accept user operations on the GUI displayed on the monitor 102, and output them to the computer main body 10 as operation inputs.

[Software 14]
Hereinafter, with reference to FIG. 2, the configuration of software for realizing the voice identification processing according to the present invention will be described. FIG. 2 is a diagram showing a configuration of the software 14 for realizing the voice identification process according to the present invention. In FIG. 2, software components that are not related to the implementation of the voice identification process according to the present invention are omitted.

  As shown in FIG. 2, the software 14 includes a hardware (H / W) support unit 142, an operation system (OS) 148, and an application unit. The hardware support unit 142 includes an audio device driver 144 and a storage device driver 146. The operating system 148 is a general-purpose OS such as OS / 2 (IBM product name) or Windows (Microsoft product name), and includes an audio interface (IF) unit 150 and a storage device interface unit 152. The software 14 includes a voice identification program 16 as an application unit. These components of the software 14 are stored in the storage device 110 and loaded into the memory of the computer main body 10 for execution as necessary.

[Audio device driver 144]
In the hardware support unit 142, the audio device driver 144 controls the audio input board 124 to convert an audio signal input from the microphone 122 into audio data. The audio device driver 144 implements an interface function for outputting audio data input from the audio input board 124 to the audio interface unit 150. Further, the audio device driver 144 performs setting control such as changing the sampling cycle of the audio input board 124 and operation control such as starting and ending sampling according to control of the audio interface unit 150 of the operating system 148.

[Storage Device Driver 146]
The storage device driver 146 controls the operation of the storage device 110 in response to a request (control) from the storage device interface unit 152 of the operating system 148, and stores word data and text data obtained as a result of voice identification. Alternatively, these data stored in the storage device 110 are read out. Further, the storage device driver 146 outputs the word data and text data input from the storage device 110 to the storage device interface unit 152, or these data input from the storage device interface unit 152 as a storage device. An interface function for outputting to the driver 146 is realized.

[Operating system 148]
The operating system 148 performs program execution control in the computer main body 10 in addition to the functions realized by the voice interface unit 150 and the storage device interface unit 152. The operating system 148 also displays text data and a GUI image output from the voice identification program 16 on the monitor 102, converts text data into a voice signal and outputs the voice signal via the speaker 104, and makes a hard copy for the printer 106. For example, processing necessary for performing the above and accepting user operations on the keyboard 126 and the mouse 128 are performed.

[Voice interface unit 150]
In the operating system 148, the voice interface unit 150 controls the voice device driver 144 in response to a request (control) from the voice identification program 16. Further, the voice interface unit 150 realizes an interface function for outputting the voice data input from the voice device driver 144 to the voice identification program 16.

[Storage device interface unit 152]
The storage device interface unit 152 manages the storage area of the storage device 110. Further, the storage device interface unit 152 controls the storage device driver 146 in response to a request (control) from the voice identification program 16, and reads word data and text data requested from the voice identification program 16 from the storage device 110. The read word data and text data are output to the voice identification program 16. Further, the storage device interface unit 152 stores word data and text data input from the voice identification program 16 in a free storage area of the storage device 110 via the storage device driver 146.

[Voice Identification Program 16]
Hereinafter, the voice identification program 16 will be described with reference to FIG. FIG. 3 is a diagram showing the configuration of the voice identification program 16 shown in FIG.

  As shown in FIG. 3, the voice identification program 16 includes a word database unit 160, a control unit 162, and a voice identification unit 18. The voice identification unit 18 includes a vector data generation unit 180, a label creation unit 182, a candidate word creation unit 184, a similar word addition unit 186, and a narrowing-down unit 188. The voice identification program 16 uses these components to display a GUI image for operation, and inputs voice data input from the voice interface unit 150 from the storage device interface unit 152 in accordance with a user operation on the displayed GUI image. The word string obtained as a result of the identification is output via the operating system 148 as text data.

[Control unit 162]
The control unit 162 displays an operation GUI image on the monitor 102 and accepts an operation performed by the user using the keyboard 126 and the mouse 128 of the input device 120 via the operating system 148 for the displayed GUI image. Further, the control unit 162 controls the voice interface unit 150 and the storage device interface unit 152 of the operating system 148 according to the received operation input.

  In addition, the control unit 162 controls the word database unit 160 according to the received operation input, and the label string table and the similar word adding unit 186 used by the candidate word creating unit 184 to associate the voice data with the candidate words. Create or update word data including an index table, a word record, a similar word record, and an error information code table (to be described later with reference to FIGS. 4 to 7) used to associate similar words with candidate words, and a storage device interface The data is stored in the storage device 110 via the unit 152 or the like.

  In addition, the control unit 162 displays candidate words and similar words associated with each part of the audio data in the GUI image, and either the candidate word or the similar word is displayed according to the operation input for these displayed words. Is selected by the narrowing-down unit 188, and the final identification result is obtained. In addition, as an example of a method for displaying candidate words and similar words by the control unit 162, candidate words associated by the candidate word creating unit 184 and the similar word adding unit 186 are highlighted on the monitor 102, so that the user can operate the keyboard 126. The candidate word and the similar word are changed accordingly, and displayed sequentially, or when the user finds an error part of the candidate word displayed on the monitor 102 and clicks with the mouse 128, the candidate of the clicked part For example, a list of candidate words and similar words can be displayed in a window associated with the word.

[Word database section 160]
The word database unit 160 creates or updates word data according to the control of the control unit 162 as described above, and stores and manages the word data in the storage device 110. Further, the word database unit 160 outputs word data (label sequence table) to the candidate word creation unit 184.

  Further, the word database unit 160 searches the word data (index table, word record, similar word record, and error information code table; FIGS. 4 to 7) in response to a request from the similar word adding unit 186, and as a result of the search. The obtained word record corresponding to the first candidate word input to the similar word adding unit 186, the similar word record, and the error information are output to the similar word adding unit 186.

[Word data]
Here, in order to facilitate understanding of the following description, word data (index table, word record, similar word record, and error information table) excluding the label column table will be described with reference to FIGS. .

[Index table]
FIG. 4 is a diagram illustrating data included in the index table of word data. The index table is used by the word database unit 160 to search for word records classified by the first character (A to Z) of the word. As shown in FIG. 4, the index table is configured by associating a pointer indicating the head of the recording area of each of the first characters A to Z with the number of word records having the first characters A to Z, respectively.

[Word record]
FIG. 5 is a diagram illustrating data included in a word record of word data. As illustrated in FIG. 5, the word record includes the first candidate word data [candidate word data (1)] created by the candidate word creation unit 184 in the similar word addition unit 186.
The heading word (TarWord), the pointer to the next heading word (NextP), the number of similar words contained in the word record (#Can), and the similar word record (CanWord) Is done.

  FIG. 5 shows a case where the similar word record is directly associated with the word record. For example, a pointer to the similar word record is associated with the word record, and the similar word record is separated from the word record. The realization method does not ask | require. Further, a plurality of headwords (TarWord) of each word record may be provided so that the similar word adding unit 186 can associate labels corresponding to a plurality of consecutive words with a headword including a plurality of words. Good.

[Similar word record]
FIG. 6 is a diagram illustrating data included in a similar word record of word data. As shown in FIG. 6, the similar word record includes the number of input words (#m; #m is an integer of 1 or more), input candidate words (aWord, aWord-1, aWord-2, ..., aWord-m- 1) The number of output words (#n; #n is an integer of 0 or more), similar words (COWord, COWord-1, .., COWord-n), and an error code (ECode) are associated with each other.

  Among these, the number of input words (#m) refers to the number of candidate words before and after the similar word adding unit 186 associates the first candidate word input from the candidate word creating unit 184 with the similar word. (Hereinafter, the case where the m-1 character behind the first candidate word is referred to will be described as an example).

  The input candidate words (aWord, aWord-1, aWord-2,..., AWord-m-1) are the m first words successively input from the candidate word creation unit 184 to the similar word addition unit 186. Indicates word strings to be compared with candidate words (TarWard, TarWord-1, ..., TarWord-m-1). That is, in the similar word adding unit 186, the p-th first candidate word (TarWord) is not immediately associated with the similar word even if it is input, and the similar word adding unit 186 further receives the (m−1) th After one candidate word is input, each of the p-th to p + m-1th m consecutive first candidate words (TarWard, TarWord-1, ..., TarWord-m-1), Each of the m input candidate words (aWord, aWord-1, aWord-2, ..., aWord-m-1) in the similar word record is compared, and only when they match, the p-th One candidate word (aWord = TarWord) is associated with n similar words (COWord, COWard1, .., COWord-n) following the input candidate word in the similar word record. If there is no similar word, the number of output words (#n) is set to 0, and the similar word is not associated in the similar record.

  The mapping method between the first candidate word and the similar word in the similar word record shown in FIG. 6 will be further described. As a method of mapping (mapping) between the first candidate word and the similar word, for example, the following four methods are conceivable.

[First method]
In order to cope with the case where one word is erroneously identified as another word, a similar word including only one word is associated with the first candidate word including only one word. As an example of the first method, the first candidate word “read” is associated with the similar word “lead” in case “r” of the first candidate word “read” is not pronounced correctly. Can be mentioned. Furthermore, as examples of correspondence when the first method is adopted, “sink” and “think”, “fell” and “fill”, “seat” and “sit”, “better” and “bitter”, “ Examples include the correspondence between "nut" and "not", "fund" and "found", "boat" and "bought", and "coal" and "call".

[Second method]
In order to cope with a case where one word is erroneously identified as another word, a similar word including a plurality of words is associated with the first candidate word including only one word. As an example of the second method, the first candidate word “jumped” and the similar word “jump” are prepared in case “ed” of the first candidate word “jumped” is not correctly pronounced “t”. One example is to associate "," and ". Furthermore, as an example of association in the case of taking the second method, association between “check in” and “chickin” can be cited.

[Third method]
In order to cope with a case where a plurality of words are mistakenly identified as another one word, a similar word including only one word is associated with the first candidate word including the plurality of words. As an example of the third method, the first candidate words “have”, “to” and the similar word “hat” are prepared in case the first candidate words “have”, “to” are connected and pronounced. Are associated with each other. Furthermore, as an example of association in the case of taking the third method, association between “Iwii” and “aisle” can be cited.

[Fourth method]
In order to cope with a case where a plurality of words are erroneously identified as another plurality of words, a similar word including a plurality of words is associated with the first candidate word including the plurality of words. Since the first to third methods can be considered as methods that are limited to the fourth method, the similar word record shown in FIG. 6 is created based on the fourth method, and a plurality of words are recorded. Similar words including a plurality of words are associated with the first candidate word to be included.

  The similar word is selected instead of the candidate word when the pronunciation of the speaker (user) using the computer 1 is inaccurate. Therefore, in the narrowing-down unit 188, when a similar word is finally selected instead of a candidate word, the speaker has made an error corresponding to the selected similar word in English pronunciation. . The error code (ECode) is added to the similar word record from such a viewpoint, and indicates a pronunciation error corresponding to the finally selected similar word in the form of a code.

[Error information code table]
FIG. 7 is a diagram illustrating an error information code table of word data. As shown in FIG. 7, the error information code table includes an error code (ECode; 0, 1, 2,...) And information indicating error contents (for example, “r is pronounced as l”, “l Error information such as “pronunciated as r” and “pronounced th as s”).

[Vector data generator 160]
In the voice identification unit 18, the vector data generation unit 160 (FIG. 3) processes the voice data input from the voice interface unit 150, and each of a plurality of voice characteristics (pitch, strength, intonation, etc.). Is quantized, vector data including numerical values indicating each of these features is generated and output to the control unit 162. For example, when the sampling frequency of the audio data is 11 kHz, the vector data generation unit 160 processes the audio data in units of 1/100 second, quantizes each of a plurality of types of features of the audio data, and includes a plurality of elements. Generate vector data.

[Label creation unit 182]
The label creation unit 182 converts the vector data input from the vector data generation unit 160 into data called a label similar to a phonetic symbol, and outputs the data to the candidate word creation unit 184 for each sentence. The label creation unit 182 associates this conversion process with, for example, a label generated from samples of actual speech of various people (adults, children, men, women, etc.) and a plurality of continuous vector data patterns. The label table is used to select labels corresponding to a plurality of continuous vector data. However, the term “sentence” does not necessarily correspond to an actual sentence in an actual sentence, but simply indicates a processing unit for voice identification.

[Candidate word creation unit 184]
The candidate word creation unit 184 uses a label column table indicating correspondence with English words to indicate one or more continuous label columns corresponding to one or more English words in the audio data. It associates with one or more combinations of one or more English words, and outputs the associated combination of English words to the similar word adding unit 186 as the first candidate word. (Hereinafter, for simplification of explanation, a case where the candidate word creation unit 184 associates each label string corresponding to one English word with a first candidate word including only one English word will be described as an example. .)

  Here, for example, the candidate word creation unit 184 does not convert the sound indicated by the label into alphabet, and converts the alphabet string obtained by the conversion into English words, but directly converts the label string into English words (first Candidate word). That is, for example, when creating “read” as the first candidate word, the candidate word creation unit 184 converts the label string into four alphabets “r”, “e”, “a”, and “d”. Instead of associating the word “read” as the first candidate word after replacement, the word “read” is directly associated with the label string.

  Note that the candidate word creation unit 184 uses the created first candidate words as similar words for each sentence in the format of the input record (InWord) shown in FIG. 8 and the input record matrix (InMatrix) shown in FIG. The data is output to the adding unit 186. FIG. 8 is a diagram illustrating a data structure of an input record (InWord) output from the candidate word creation unit 184 to the similar word addition unit 186. FIG. 9 is a diagram illustrating a data structure of an input record matrix (InMatrix) output from the candidate word creation unit 184 to the similar word addition unit 186.

  As shown in FIG. 8, the candidate word creation unit 184 indicates that the word associated with the label string and the data (InWord) indicating the word length are the i-th word in one sentence, and Data indicating that it is the i-th j-th first candidate word of one sentence is added, an input record (InWord) is created and output to the similar word adding unit 186. However, i and j are integers, i is not more than the maximum number of words (Maxi), and j is not more than the maximum number of candidates (Maxj).

  Further, as shown in FIG. 8, the candidate word creation unit 184 changes the degree of matching between the label sequence input from the label creation unit 182 and the label sequence in the label sequence table corresponding to the selected English word. Then, probability data indicating the probability that the label string indicates the first candidate word is generated, added to the data indicating the word of the input record and its word length, and output to the similar word adding unit 186.

  When the creation of the input record (InWord) for one sentence is completed, the candidate word creation unit 184, as shown in FIG. 9, displays the maximum number of words (Maxi) included in the sentence and the same label string (reading). ), A maximum number of candidates (Maxj) indicating how many first candidate words are associated with each other, and a flag Flg (ij indicating whether the j-th word exists in the i-th word ) Is generated and output to the similar word adding unit 186 together with the input records for one sentence. If the candidate word creation unit 184 cannot select the first candidate word in the label string corresponding to the i-th word, the flag Flg (i1) includes the i-th first word. It is set to a value (for example, 0) indicating that no operation is performed.

[Similar word addition unit 186]
Although the similar word adding unit 186 cannot associate each input record input from the candidate word creating unit 184 with a label string, the first candidate is considered in consideration of English pronunciation habits by the Japanese. An English word that may be similar to the word is added to the first candidate word, and a second candidate word is generated and output to the narrowing unit 188.

  Further, the operation of the similar word adding unit 186 will be described in detail. The similar word adding unit 186 first sequentially outputs the p-th first candidate word included in the input record (InWord) for one sentence input from the candidate word creating unit 184 to the word database unit 160. And request to get word records. The word database unit 160 searches the word record (FIG. 5) using the index table (FIG. 4), acquires a word record in which the word included in the input record (InWord) and the heading word (TarWord) match, It outputs to the similar word addition part 186.

  When the similar word adding unit 186 obtains the word index of the p-th first candidate word, the p-th to p + m−1-th words (InWord-pj, InWord-p + 1.j,... , InWord-p + m-1.j) and m input words (aWord, aWord-1, aWord-) of each similar word record (FIG. 6) added to the word record input from the word database unit 160 2, ..., aWord-m-1), and if they match, n similar words (COWord-1, COWord-2, ..., COWord-n) are A second candidate word is created by adding to the first candidate word.

  The similar word adding unit 186 does not create the second candidate word by adding the similar word to the first candidate word, but replaces the first candidate word with the similar word to replace the second candidate word. It is also possible to change the process to create. In this case, it does not matter whether or not the first candidate word is included in the similar words.

  Further, the similar word adding unit 186 outputs an error code (ECode) corresponding to the similar word record added to the first candidate word to the word database unit 160, and requests acquisition of error information indicated by the error code. . In response to this request, the word database unit 160 searches the error information code table (FIG. 7), acquires error information, and outputs it to the similar word adding unit 186.

  FIG. 10 is a diagram illustrating a data structure of an output record (OutWord) output from the similar word adding unit 186 to the narrowing-down unit 188. FIG. 11 is a diagram illustrating a data structure of an output record matrix (OutMatrix) output from the similar word adding unit 186 to the narrowing-down unit 188.

  The similar word adding unit 186 includes the second candidate word, data indicating the word length, probability data, and error information (or error information code), and the word is the i'th in one sentence. Output indicating that it is the i'th j'th first candidate word of one sentence, and takes the same format as the input record (FIG. 8) as shown in FIG. The data is output to the narrowing down unit 188 in the form of a record (OutWord).

  When the output record (OutWord) for one sentence is completed, the similar word adding unit 186, as shown in FIG. 11, performs the maximum word included in the sentence as in the input record matrix (FIG. 9). Number (Maxi '), the maximum number of candidates (Maxj') indicating the maximum number of second candidate words associated with the same label string (reading), and the i'th word An output record matrix indicating the flag Flg (i′j ′) indicating whether or not the j′-th word exists is created and output to the narrowing-down unit 188 together with an output record for one sentence.

[Refinement part 188]
The narrowing-down unit 188 displays the second candidate word input from the similar word adding unit 186 as an output record on the monitor 102, and for example, syntactic analysis of the word string identified in response to the user's operation or so far Based on the result, one of the second candidate words is selected as a final identification result, and text data in which the selected word is arranged is created and output to the monitor 102, the speaker 104, or the printer 106.

  The creation of text data by the narrowing unit 188 will be further described. For example, n1 second candidate words (OutWord-1.1, OutWord-1.2, ..., OutWord-1.n1) of the first word of one sentence, the second candidate of the second word When there are n2 words or the like, the narrowing-down unit 188 selects one of the second candidate words for the first word and one of the second candidate words for the second word. In addition, one of the second candidate characters of each word is displayed on the monitor 102 in the order of the words.

  When the user clicks the second candidate word displayed in the window of the GUI image using the mouse 128, for example, the control unit 162 highlights the clicked second candidate word, and then the user Each time the same part is clicked, the second candidate word to be displayed is changed. When the user finally selects the second candidate word and performs an operation to that effect on the mouse 128 or the keyboard 126 or shifts to a next word selection operation, the narrowing-down unit 188 displays the last word The selected second candidate word is used as the finally selected identification result. The user repeats this work as necessary, and the narrowing-down unit 188 selects a word according to this work and creates text data.

  Note that when the second candidate word is displayed, the narrowing-down unit 188 monitors only the second candidate word that is likely to be selected as the final identification result in consideration of the context of the word. It is convenient to display the information on the monitor 102 or to display the information on the monitor 102 in the order that the possibility of being selected as the final identification result is high.

  A case where Japanese is identified for convenience will be described as a specific example. The candidate word creation unit 184 identifies the label obtained from the Japanese voice data “Akai Hana” as “Taka Hana (high, flower, high, nose)”, and further adds a similar word addition unit There is a possibility that 186 adds the similar word “red / 罠” to these and outputs these three types as the second candidate words to the narrowing-down unit 188. In such a case, in the narrowing-down unit 188, when the identification result of the first half of the second candidate word is “red”, the identification result of the second half is “flower”, “nose”, and “罠” in this order. It seems to be certain. In such a case, the process of the narrowing-down unit 188 may display only “flower” and “nose” following the word “red”, or display in the order of “flower”, “nose”, and “罠”. By doing so, the user's selection operation can be minimized.

  Further, the processing of the narrowing-down unit 188 corresponds to, for example, probability data that takes a value in a range greater than or equal to a threshold set by the user, using probability data added to the output record input from the similar word adding unit 186. If only the second candidate word is displayed on the monitor 102, only the second candidate word that is likely to be selected as the final identification result is displayed on the monitor 102, and the user's selection operation is performed. Can be less.

[Operation of computer 1]
Hereinafter, the voice identification process in the computer 1 will be described with reference to FIG. FIG. 12 is a flowchart showing the voice identification process according to the present invention in the computer 1. In FIG. 12, for the sake of simplification of explanation, only basic processing is shown in the speech identification processing according to the present invention, and the processing using the above-described probability data or error information is appropriately omitted. Has been.

  As shown in FIG. 12, in step 100 (S100), the vector data generation unit 160 and the label creation unit 182 (FIG. 3) of the voice identification program 16 perform one sentence (unit) of voice data pronounced by a Japanese user. Is processed and converted into a label string. The label string for one sentence generated by the vector data generation unit 160 and the label generation unit 182 is input to the candidate word generation unit 184.

  In step 102 (S102), the candidate word creation unit 184 associates the label string for one sentence input from the label creation unit 182 with the first candidate word, and forms the input record (InWordij) shown in FIG. Are output to the similar word adding unit 186, and the input record matrix (InMatrix) shown in FIG. 9 is generated and output to the similar word adding unit 186.

  In step 104 (S104), the similar word adding unit 186 requests the word database unit 160 to search for a word record (FIG. 5) of the first candidate word included in the input record to be processed. . The word database unit 160 performs a search using the index table (FIG. 4) in response to a request from the similar word addition unit 186, and a word record corresponding to the first candidate word (input record) is obtained as a result of the search. If it is obtained, the obtained word record is output to the similar word adding unit 186 and the process proceeds to S106. If not obtained, the process proceeds to S110.

  In step 106 (S106), the similar word adding unit 186 processes the similar word record (FIG. 6) of the word record input from the word database unit 160, and the similar word corresponding to the first candidate word (input record). To get.

  In step 108 (S108), the acquired similar word is added to the first candidate word to create a second candidate word.

  In step 110 (S110), the similar word adding unit 186 determines whether or not all the processes for the input records included in one sentence have been completed, and if completed, proceeds to the process of S110, and if not completed. Changes the processing target to the next input record and returns to the processing of S104.

  In step 112 (S112), the similar word adding unit 186 outputs the second candidate word created in S108 to the narrowing unit 188 in the output record format shown in FIG. Further, the similar word adding unit 186 creates an output record matrix (FIG. 11) corresponding to the second candidate word and outputs the output record matrix to the narrowing unit 188. The narrowing-down unit 188 displays the input second candidate word in the window of the GUI image on the monitor 102, and outputs the final identification result as English text data in accordance with a user operation.

[Modification]
The process of the narrowing down unit 188 is added to the selected word when the user selects a word added to the second candidate word as a similar word in the similar word adding unit 186 as a final identification result. If the error information is displayed on the monitor 102, the user can know the shortcomings of his / her English pronunciation, and can use the computer 1 as a speech identification device and also as an English pronunciation learning device. . As a display method of error information, a method of displaying a mouth shape for correct pronunciation or synthesizing correct pronunciation and letting the user hear it is conceivable.

  Further, in the first embodiment described above, the process of identifying English text from English pronounced by a Japanese is exemplified, but the speech identification process according to the present invention is not limited to such a case. An American who speaks American English uses a voice recognition device tuned for British English to create text, such as when the pronunciation is different even in the same language, or there is a discrepancy in personal pronunciation It can be widely applied to deal with a decline in the identification rate when it is unknown or unknown.

  As described above, in the speech identification processing according to the present invention, the similar word adding unit 186 adds a similar word to the first candidate word identified by the candidate word creating unit 184 to obtain the second candidate word. Therefore, the voice identification rate is improved. For convenience, the case where Japanese is identified by the computer 1 will be described as a specific example. For example, even if the user intends to pronounce “Akai Hana (red flower)” in Japanese, the pronunciation of “A” becomes unknown for some reason, and the label creating unit 182 causes the “Akai Hana” to be pronounced. In some cases, a label string that does not connect with "Takai Hana" may be generated.

  Even when the candidate word creation unit 184 selects “high / nose” as the first candidate word corresponding to the label string and outputs it to the similar word addition unit 186, the word record is created appropriately. Then, the similar word adding unit 186 can add “red / nose” and “red / flower” to the first candidate word as the first word to be the second candidate word. Therefore, the original “red flower” that was not included in the first candidate word created by the candidate word creation unit 184 is also displayed on the monitor 102, and the user can select from the second candidate words. The correct identification result can be selected.

  In general, speech recognition methods combine pure speech processing with grammatical analysis processing (nouns are easily placed at specific positions in sentences, etc.) and language model processing (probability of aligned words). Therefore, if similar words are added by the speech identification method according to the present invention before these processes are performed, the effect such as a significant improvement in the speech identification rate is obtained.

[Second Embodiment]
In the voice identification program 16 shown in FIG. 3, the candidate word is selected as a result of the voice identification. This means that the pronunciation of the user (speaker) is relatively close to that of the native speaker and the general voice identification that does not apply to the present invention. This means that it is accurate enough to be recognized with a high identification rate even when using a device. On the other hand, if a similar word is selected, the speaker's pronunciation is incorrect or unclear. Means. This also applies to the case where the word intended by the user is included in the similar words in the voice identification program 16.

  Therefore, when the word intended by the user is identified as a similar word, information such as how the pronunciation is wrong for the speaker and how to pronounce correctly (pronunciation correction information) ) Can assist in learning the pronunciation of the speaker and correct the pronunciation of the speaker. Hereinafter, the pronunciation correction method shown as the second embodiment pays attention to this point, and when the speech identification process shown as the first embodiment is modified and it can be determined that the speaker's pronunciation is incorrect, An image showing pronunciation correction information is displayed to the speaker.

[Computer 2]
FIG. 13 is a diagram showing a configuration of the computer 2 that realizes the voice identification process and the pronunciation correction method according to the present invention. Unless otherwise specified, the constituent parts shown in the following drawings are the same as the constituent parts having the same reference numerals shown in the previous drawings. As shown in FIG. 13, the computer 2 adopts a configuration in which the input device 120 of the computer 1 (FIG. 1) is replaced with an input device 130, and the input device 130 has a configuration in which an image input board 132 is added to the input device 120. Take. The image input board 132 is used, for example, for capturing image data used for a pronunciation correction image (FIG. 19).

[Software 20]
FIG. 14 is a diagram showing the software 20 for realizing the voice identification process and the pronunciation correction method according to the present invention. As shown in FIG. 14, the software 20 employs a configuration in which the voice identification program 16 in the software 14 (FIG. 2) is replaced with a voice identification / correction program 22. In addition to the data input / output between each component in the software 14, image data is further input / output in the software 20, and instead of the identification result (text data) in the software 14, the user (speaker) An image (pronunciation correction image) indicating pronunciation correction information for correcting pronunciation is output to the monitor 102 or the like.

[Voice identification / correction program 22]
FIG. 15 is a diagram showing the configuration of the voice identification / correction program 22 shown in FIG. As shown in FIG. 15, the voice identification / correction program 22 includes a voice identification program 16 (FIG. 3) and a pronunciation correction program 24 (the narrowing-down unit 188 is omitted).

[Changes in similar word addition unit 186]
In the speech identification / correction program 22, unlike the speech identification program 16, the similar word addition unit 186 sends similar word records (FIG. 18) to the comparison unit 240 and the pronunciation correction information display unit 242 of the pronunciation correction program 24. Output.

[Changes in control unit 162]
FIG. 16 is a diagram exemplifying a sound generation instruction image displayed by the control unit 162 shown in FIG. FIGS. 17A and 17B are diagrams illustrating examples of correction information indexes generated by the word database unit 160 shown in FIG. 15, and FIG. 17A is a pronunciation correction image for correcting the pronunciation of r. (B) shows a pronunciation correction image for correcting the pronunciation of l. FIG. 18 is a diagram illustrating similar word records generated by the word database unit 160 in the second embodiment.

  As illustrated in FIG. 16, the control unit 162 indicates a word to be pronounced by the user (“read” in FIG. 16), and pronounces a pronunciation instruction image (“read” in FIG. 16) to prompt the pronunciation. The image including the text data “Please see!” Is further generated and displayed on the monitor 102, and the word (correct word) instructing the user to pronounce is output to the comparison unit 240. Further, when the correction information code (CCode; FIG. 18) is input from the pronunciation correction image display unit 242, the control unit 162 outputs the correction information code to the word database unit 160, and the correction information code is The correction information index shown (FIGS. 17A and 17B) is obtained from the word database unit 160. Further, the control unit 162 stores the image data and text data indicated by n entries (n is an integer; n = 8 in FIGS. 17A and 17B) included in the correction information index from the storage device 110. Read out and output to the pronunciation correction image display unit 242.

  FIG. 19 is a diagram illustrating a first pronunciation correction image indicated by the correction information index illustrated in FIG. In FIG. 19, for simplification of illustration, text data corresponding to entries 5 to 7 illustrated in FIG. 17A is omitted. Further, the control unit 162 stores an entry included in the correction information index as illustrated in FIG. 17A, and image data (Image) and text data (Text) indicated by each of the entries as illustrated in FIG. The data is stored in the storage device 110 in association with each other.

  The correction information index illustrated in FIG. 17A is used to correct the pronunciation of the user's r, and is text data indicating a point where the pronunciation should be corrected, and image data indicating the shape of the mouth when the r is pronounced. , Image data indicating the mouth shape when sounding l, text data indicating advice for sounding r, examples of words including r, examples of words including l, examples of words in which r and l appear And entries (entries 1 to 4 and 8) respectively indicating text data indicating examples of sentences in which r and l appear. From the correction information index illustrated in FIG. 17A, a pronunciation correction image as illustrated in FIG. 19 is generated and displayed on the monitor 102.

  Further, the correction information index illustrated in FIG. 17B is used to correct the pronunciation of the user l, and indicates text data indicating a point to be corrected for pronunciation and the shape of the mouth when l is pronounced. Image data, image data indicating the shape of the mouth when sounding r, text data indicating advice for sounding l, examples of words including l, examples of words including r, words in which r and l appear And entries indicating text data indicating examples of sentences in which r and l appear, respectively.

[Changes in word database section 160]
Unlike the software 14, the word database unit 160 creates a correction information index as exemplified in FIGS. 17A and 17B instead of the error information code table (FIG. 7), and stores it as word data. Further store in 110.

  Further, as shown in FIG. 18, the word database unit 160 adds a correction information code (CCode; correction code) indicating one of the correction information indexes to the similar word record instead of the error code (ECode; FIG. 6). And stored in the storage device 110. When the correction information code is input from the pronunciation correction image display unit 242 via the control unit 162, the word database unit 160 corrects the correction information index (FIG. 17) corresponding to the input correction information code (FIG. 18). (A) and (B)) are read from the storage device 110 and output to the control unit 162.

[Comparator 240]
The comparison unit 240 (FIG. 15) compares each similar word included in the similar word record indicated by the similar word record (FIG. 18) input from the similar word addition unit 186 with the correct word input from the control unit 162. Then, it is determined whether the correct word matches any of the similar words. When the correct word matches the similar word, the comparison unit 240 notifies the fact to the pronunciation correction image display unit 242.

[Pronunciation correction image display unit 242]
The pronunciation correction image display unit 242 corrects the correction information index indicated by the correction information code (CCode) added to the similar word record (FIG. 18) when the comparison unit 240 determines that any of the correct words matches the similar word. The control unit 162 is requested to acquire the image data and text data indicated by (FIGS. 17A and 17B). In response to this request, the control unit 162 reads out image data and text data from the storage device 110 and outputs them to the pronunciation correction image display unit 242, and the pronunciation correction image display unit 242 displays these data in FIG. It arrange | positions in the position which attached | subjected code | symbol (a)-(e), illustrated, and produces | generates a pronunciation correction image, and displays it on the monitor 102. FIG.

[Operation of voice identification / correction program 22]
Hereinafter, the operation of the voice identification / correction program 22 will be described. FIG. 20 is a flowchart showing the process (S20) of the voice identification / correction program 22 (FIG. 15) in the second embodiment.

  In FIG. 20, as shown in step 200 (S <b> 200), in response to a user operation input, the control unit 162, for example, as shown in FIG. 16, a pronunciation instruction that prompts the user to pronounce the word “read” The image is displayed on the monitor 102. The control unit 162 outputs the correct word “read” to the comparison unit 240. When the user pronounces “read” in response to the pronunciation instruction image, the vector data creation unit 180, label data creation unit 182, candidate word creation unit 184, and similar word addition unit 186 (FIG. 15) of the voice identification program 16 Is identified and a similar word record is output to the comparison unit 240 and the pronunciation correction image display unit 242.

  Here, as illustrated in the first embodiment, when the user correctly pronounces “read”, the candidate word creation unit 184 outputs candidate words including the word “read” to the similar word addition unit 186, The similar word adding unit 186 outputs a similar word record (FIG. 18) including the word “lead” or the like as a similar word to the comparison unit 240 and the pronunciation correction image display unit 242. On the other hand, for example, if the user cannot distinguish between the pronunciation of “r” and the pronunciation of “l” and makes an incorrect pronunciation of “read”, the candidate word creation unit 184 replaces the word “read”. Candidate words including the word “lead” or the like are output to the similar word adding unit 186, and the similar word adding unit 186 displays a similar word record including the word “read” or the like as a similar word in the comparison unit 240 and pronunciation correction image display. To the unit 242.

  In step 202 (S202), the comparison unit 240 compares the correct word input from the control unit 162 with each similar word included in the similar word record input from the similar word adding unit 186, and the correct word is similar. If it does not match any of the words, the pronunciation correction image display unit 242 is notified of this, and the processing related to the correction / learning of the pronunciation of “read” is terminated. For example, the processing related to the correction / learning of the pronunciation of the next word Proceed to In other cases, the comparison unit 240 notifies the pronunciation corrective image display unit 242 that the correct word matches any of the similar words, and proceeds to the process of S204.

  In step 204 (S204), the pronunciation correction image display unit 242 obtains a correction information code (CCode) from the similar word record. Further, the pronunciation correction image display unit 242 outputs a correction information code to the control unit 162 and requests acquisition of image data and text data used for the pronunciation correction image. In response to a request from the pronunciation correction image display unit 242, the control unit 162 obtains the correction information index illustrated in FIG. 17A from the word database unit 160, and the correction information index entry (entries 1 to 4). 8) The image data and text data indicated by each are read from the storage device 110 and output to the pronunciation correction image display unit 242.

  In step 206 (S206), the pronunciation correction image display unit 242 displays the image data and text data corresponding to the correction information index entries (entries 1 to 4, 8) input from the control unit 162, respectively, in FIG. The pronunciation correction images arranged at the illustrated positions (a) to (e) are generated, displayed on the monitor 102, the processing relating to “read” is terminated, and the processing proceeds to, for example, processing relating to correction / learning of pronunciation of the next word.

[Modification]
In the second embodiment, the case where the pronunciation correction information includes only text data and image data is exemplified, but the pronunciation correction information may include other types of data. For example, the pronunciation correction information may include sound data of correct pronunciation, the pronunciation correction information (FIG. 19) may be displayed on the monitor 102, and the sound data may be output via the speaker 104.

  Further, in the second embodiment, the case where similar words and pronunciation correction information codes are managed in association with each other has been shown. However, the voice identification / correction program is managed so as to be included in the similar word record shown in FIG. 22 may be configured. In the second embodiment, the case where the voice identification / correction program 22 outputs only the pronunciation correction information has been described. However, both the pronunciation correction information and the error information may be output. Further, in the second embodiment, the case where the control unit 162 displays the pronunciation instruction image on the monitor 102 and the pronunciation correction image display unit 242 displays the pronunciation correction information on the monitor 102 is exemplified. The voice identification / correction program 22 may be configured to display either of these images on the monitor 102.

  Further, it is not necessary that all information displayed in the pronunciation correction image (FIG. 19) is registered in the correction information index (FIGS. 17A and 17B). For example, the points to be corrected and the text data for supporting re-sounding shown in FIG. 17A may be registered in the correction information index depending on how to create the voice identification / correction program 22 (FIG. 15), or Instead of being registered in the correction information index, it may be written in advance in the pronunciation correction image.

[Third Embodiment]
Hereinafter, as a third embodiment, a pronunciation learning method to which the pronunciation correction method shown as the second embodiment is applied will be described. This pronunciation learning method improves the operation of the pronunciation identification / correction program 22 shown in FIG. 20, and in addition to the pronunciation instruction image and the pronunciation correction image showing the basics of pronunciation shown in FIGS. The phonetic correction images (FIGS. 21 to 23) for instructing learning are further displayed to facilitate the user's pronunciation learning.

  21 to 23 are diagrams illustrating second to fourth pronunciation correction images used in the pronunciation learning method shown as the third embodiment. In the third embodiment, the pronunciation identification / correction program 22 displays the pronunciation correction images illustrated in FIGS. 21 to 23 in addition to the pronunciation correction images shown in FIGS.

  The second pronunciation correction image shown in FIG. 21 is displayed, for example, when the pronunciation identification / correction program 22 determines that the user has correctly pronounced “read” by looking at the first pronunciation correction image (FIG. 19). And is used to check whether the user correctly pronounces “r” and “l”. Note that the second pronunciation correction image is not displayed until the user correctly pronounces “read” in accordance with the first pronunciation correction image, and the user does not display the word “shown in the second pronunciation correction image”. "write", "raw", "long", "light" are all displayed repeatedly until they can be pronounced correctly. Further, the second pronunciation correction image includes entries 5 and 6 of the correction information index (FIG. 17A) omitted in the first pronunciation correction image.

  The third pronunciation correction image shown in FIG. 22 is displayed, for example, when the pronunciation identification / correction program 22 determines that the user has correctly pronounced each word in the second pronunciation correction image (FIG. 21). Used by the user for further practice to distinguish between "r" and "l". Note that the third pronunciation correction image is not displayed until the user correctly pronounces all the words shown in the second pronunciation correction image, and the sentence “shown in the third pronunciation correction image” “writeletters” and “great trouble” are repeatedly displayed until the user can pronounce correctly. The second pronunciation correction image is the correction information index (FIG. 17) omitted in the first pronunciation correction image. (A)) entry 7 is included.

  The fourth pronunciation correction image shown in FIG. 23 is displayed, for example, when the pronunciation identification / correction program 22 determines that the user has correctly pronounced all the sentences in the third pronunciation correction image (FIG. 22). This is used to confirm that the user can pronounce "r" and "l" separately. Note that the fourth pronunciation correction image is not displayed until the user correctly pronounces all the sentences shown in the third pronunciation correction image, and the sentence “Theriver” shown in the fourth pronunciation correction image is not displayed. "rose several feet and finally overflowed its banks." is displayed repeatedly until the user can pronounce it correctly.

[Operation of voice identification / correction program 22]
Hereinafter, the operation of the voice identification / correction program 22 in the third embodiment will be described. FIG. 24 is a flowchart showing the operation (S30) of the voice identification / correction program 22 (FIG. 15) in the third embodiment. FIG. 25 is a diagram illustrating a learning item list created in the learning item list-up process (S300) shown in FIG.

  As shown in FIG. 24, in step 300 (S300), the speech identification / correction program 22 compares the correct word with the similar word as shown in FIG. 20, and obtains the correct word that matches the similar word. Further, the voice identification / correction program 22 determines the pronunciation (for example, “r”, “th”) that the user is not good at in the obtained correct word, and lists it as a learning item, for example, as illustrated in FIG. To do.

  In step 302 (S302), the speech identification / correction program 22 determines whether or not the learning items listed in the process of S300 remain unlearned. The speech identification / correction program 22 proceeds to the process of S304 when the learning item remains, and ends the process otherwise.

  In step 304 (S304), the voice identification / correction program 22 takes out one of the learning items listed in the process of S300. In step 306 (S306), the speech identification / correction program 22 displays the pronunciation instruction image (FIG. 16) and the pronunciation correction image (FIG. 19), as shown in the second embodiment, and learns pronunciation for the user. Let

  In step 308 (S308), the voice identification / correction program 22 identifies the user's voice, and if the word obtained as a result of the identification matches the candidate word, the voice identification / correction program 22 determines that the user has pronounced correctly and proceeds to S310. In other cases, it is determined that the user has not pronounced correctly, and the process returns to S306.

  In step 310 (S310), the speech identification / correction program 22 determines whether there is an applied problem (second to fourth pronunciation correction images; FIGS. 21 to 23) that is not used for learning. When the application problem remains, the pronunciation identification / correction program 22 proceeds to the process of S312 and otherwise returns to the process of S302.

  In step 312 (S312), the speech identification / correction program 22 displays any one of the second to fourth pronunciation correction images and allows the user to learn pronunciation. In step 314 (S314), the voice identification / correction program 22 determines whether or not the user has pronounced correctly. If the user's pronunciation is correct, the process proceeds to S316. Otherwise, the process of S312 is performed. Return to.

  In step 316 (S316), the voice identification / correction program 22 determines whether or not the user's pronunciation has an error other than the learning items listed in the process of S300. If there is such an error, the voice identification / correction program 22 proceeds to the process of S318, and otherwise returns to the process of S310.

  In step 318 (S318), the speech identification / correction program 22 adds the user's pronunciation error found in the process of S316 to the learning item, and returns to the process of S310.

  As described above, according to the speech identification device and method therefor according to the present invention, it has been made in view of the above-mentioned problems of the prior art, and a speaker whose non-native language is a predetermined language (non-native) ) Can be identified and replaced with words in a predetermined language intended by the speaker to create accurate text data.

  Further, according to the speech identification device and method according to the present invention, even if the pronunciation of the same language changes due to a different spoken area, the speaker in any area Can be converted into words intended by the speaker, and accurate text data can be created. In addition, according to the voice identification device and method according to the present invention, it is possible to always maintain a high identification rate by compensating for individual differences in pronunciation.

  Furthermore, according to the pronunciation correcting apparatus and method thereof according to the present invention, the problem of the pronunciation of the speaker is pointed out using the data obtained in the process of the voice identifying apparatus and method according to the present invention. In addition, the speaker can learn the pronunciation of the native speaker and correct the speaker's pronunciation. Further, according to the pronunciation correcting apparatus and method according to the present invention, it is possible to automatically compare the pronunciation of the speaker with the correct pronunciation to indicate an error, and further, how the speaker can pronounce the pronunciation. It is possible to present detailed information indicating whether or not to correct and correct the pronunciation.

It is a figure which illustrates the structure of the computer which implement | achieves the audio | voice identification process concerning this invention. FIG. 2 is a diagram showing a software configuration for realizing the voice identification process according to the present invention. It is a figure which shows the structure of the audio | voice identification program shown in FIG. It is a figure which illustrates the data contained in the index table of word data. It is a figure which illustrates the data contained in the word record of word data. It is a figure which illustrates the data contained in the similar word record of word data. It is a figure which illustrates the error information code table of word data. It is a figure which shows the data structure of the input record (InWord) which a candidate word preparation part outputs to a similar word addition part. It is a figure which shows the data structure of the input record matrix (InMatrix) which a candidate word preparation part outputs to a similar word addition part. It is a figure which shows the data structure of the output record (OutWord) which a similar word addition part outputs to a narrowing-down part. It is a figure which shows the data structure of the output record matrix (OutMatrix) which a similar word addition part outputs to a narrowing-down part. It is a flowchart figure which shows the audio | voice identification process concerning this invention in a computer. FIG. 13 is a diagram showing the configuration of a computer that implements the speech identification process and the pronunciation correction method according to the present invention. FIG. 14 is a diagram showing software for realizing the voice identification process and the pronunciation correction method according to the present invention. It is a figure which shows the structure of the audio | voice identification and correction program shown in FIG. FIG. 16 is a diagram illustrating a sound generation instruction image displayed by the control unit illustrated in FIG. 15. FIGS. 17A and 17B are diagrams illustrating examples of correction information indexes generated by the word database unit shown in FIG. 15, and FIG. 17A shows pronunciation correction images for correcting the pronunciation of r. (B) shows a pronunciation correction image for correcting the pronunciation of l. FIG. 18 is a diagram illustrating similar word records generated by the word database unit in the second embodiment. It is a figure which illustrates the 1st pronunciation correction image which the correction information index illustrated in Drawing 17 (A) shows. It is a flowchart which shows the process (S20) of the audio | voice identification and correction program (FIG. 15) in 2nd Embodiment. It is a figure which illustrates the 2nd pronunciation correction image used in the pronunciation learning method shown as 3rd Embodiment. It is a figure which illustrates the 3rd pronunciation correction image used in the pronunciation learning method shown as 3rd Embodiment. It is a figure which illustrates the 4th pronunciation correction image used in the pronunciation learning method shown as 3rd Embodiment. It is a flowchart which shows operation | movement (S30) of the audio | voice identification and correction program (FIG. 15) in 3rd Embodiment. FIG. 25 is a diagram illustrating a learning item list created in the learning item list-up process (S300) shown in FIG. 24;

Explanation of symbols

1, 2 ... Computer 10 ... Computer main body 100 ... Output device 102 ... Monitor 104 ... Speaker 106 ... Printer 120, 130 ... Input device 122 ... Microphone 124 ... Voice input board 126 ... Keyboard 128 ... Mouse 132 ... Image input board 110 ... Storage device 14, 20 ... Software 142 ... Hardware support unit 144 ... Audio device Driver 146 ... Storage device driver 148 ... Operating system 150 ... Voice interface unit 152 ... Storage device interface unit 16 ... Voice identification program 160 ... Word database unit 162 ... Control unit 18 ... voice identification unit 180 ... vector data generation unit 182 ... 184 ... Candidate word creation unit 186 ... Similar word addition unit 188 ... Narrowing unit 22 ... Voice identification / correction program 24 ... Pronunciation correction program 240 ... Comparison unit 242 ... -Pronunciation correction image display

Claims (3)

Candidate word associating means for associating one or more word candidates (candidate words) obtained by identifying speech data indicating words;
Similar word associating means for associating zero or more words (similar words) that can correspond to the pronunciation of each candidate word with each of the candidate words associated with voice data;
When the word indicated by the audio data matches the similar word associated with each of the candidate words associated with the audio data, it corresponds to the same similar word as the word indicated by the audio data, A pronunciation correction device comprising: pronunciation correction data output means for outputting pronunciation correction data for correcting the pronunciation of a word indicated by the voice data. Associating one or more word candidates (candidate words) obtained by identifying speech data indicating words;
0 or more words (similar words) that can correspond to the pronunciation of each of the candidate words are associated with each of the candidate words associated with the voice data,
When the word indicated by the audio data matches the similar word associated with each of the candidate words associated with the audio data, it corresponds to the same similar word as the word indicated by the audio data, A pronunciation correction method for outputting pronunciation correction data for correcting the pronunciation of a word indicated by the voice data. A candidate word associating step for associating one or more word candidates (candidate words) obtained by identifying speech data indicating words;
A similar word associating step of associating zero or more words (similar words) that can correspond to the pronunciation of each candidate word with each of the candidate words associated with speech data;
When the word indicated by the audio data matches the similar word associated with each of the candidate words associated with the audio data, it corresponds to the same similar word as the word indicated by the audio data, A computer-readable recording medium storing a program for causing a computer to execute a pronunciation correction data output step of outputting pronunciation correction data for correcting pronunciation of a word indicated by the voice data.

Images