JP2019035818A - Vocalization utterance learning device and microphone - Google Patents

Abstract

To make it possible to improve language learning efficiency of a user.SOLUTION: A vocalization utterance learning device 10 that makes a user learn languages includes: an input section 132 for inputting respective air passage voice and bone conduction voice of a first voice; and a creation section 134 for converting a rhythm of the first voice into a prescribed rhythm to create a second voice while sound quality of the first voice is maintained based on the input air passage voice and bone conduction voice.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an utterance / utterance learning apparatus and a microphone.

  2. Description of the Related Art Conventionally, an utterance / utterance learning apparatus that creates a model voice based on input text is known.

  In this regard, Patent Document 1 discloses a technique for creating user voice based on voice uttered by the user.

Japanese Patent No. 3701850

  However, when the user himself / herself who uttered the voice of the user created by the technique described in Patent Document 1 hears it, the user feels it is different from his / her voice that he / she always listens to, and feels uncomfortable. End up. If the user feels uncomfortable, learning of languages such as Japanese and English may be hindered and the language learning efficiency may be reduced.

  The present invention has been made in view of such problems, and an object thereof is to provide a speech / speech learning device and a microphone that can improve the language learning efficiency of a user.

  First, the present inventor examined the reason why the user himself felt uncomfortable when listening to the user's voice. The sound emitted from the user's mouth propagates through the air and reaches both ears. This sound transmitted through the air is called “airway sound”. On the other hand, the sound of the “bone-conducted speech” that transmits the vibration of the vocal cords when speaking out directly through the skull is also transmitted to your ear. People other than the user do not receive this bone-conducted voice and only listen to the airway voice. The recorded sound is also airway voice. On the other hand, the user himself / herself listens to both the airway sound and the bone conduction sound, and as a result, the sound is recognized differently from the sound heard by others and the recorded sound. I learned to remember. Therefore, the present inventor considered using not only airway speech but also bone conduction speech for language learning.

  The utterance / utterance learning apparatus according to the first aspect of the present invention is an utterance / utterance learning apparatus that allows a user to learn utterances of language speech, and inputs an airway voice and a bone-conducted voice constituting the first voice, respectively. The second voice is created by mixing the input airway voice and the bone conduction voice at a ratio constituting the voice that the user is listening to while maintaining the sound quality of the first part and the first voice. An utterance / utterance learning device comprising:

  According to this configuration, when the user who utters the first sound hears the created second sound, the sound quality of the second sound feels equivalent to the sound quality of his / her own sound he / she always listens to. It is possible to suppress feeling uncomfortable. If the sense of discomfort can be reduced, language learning efficiency can be improved.

  In the utterance / utterance learning device according to the second aspect of the present invention, the creation unit modifies transmission characteristics and creates a voice closer to a voice heard by a speaker as the second voice.

  In the utterance / utterance learning device according to the third aspect of the present invention, the input unit further includes the airway speech and the bone guide that are input when a predetermined manuscript for creating synthesized speech is read by a user. Based on the voice, a phoneme and a phoneme string corresponding to the voice that the speaker is listening to are created, and a synthesized voice along the text data inputted for learning is created as the second voice.

  In the utterance / speech learning device according to the fourth aspect of the present invention, the input unit further inputs a determination operation of one mixing ratio among mixing ratios of the airway speech and the bone conduction speech, and the creation unit includes: Based on the one mixing ratio, the inputted airway sound and bone conduction sound are mixed.

  In the utterance / utterance learning device according to the fifth aspect of the present invention, the input unit inputs the bone-conducted speech and is configured to be disposed in the user's external auditory canal, and the airway speech. And an airway microphone installed outside the bone-conduction microphone.

  A microphone according to a sixth aspect of the present invention inputs a bone conduction sound of a user, inputs a bone conduction microphone configured to be placed in the user's external auditory canal, inputs a user's airway sound, and outside the bone conduction microphone. Collect voices that the user is listening to, including an airway microphone on the wall.

  ADVANTAGE OF THE INVENTION According to this invention, a user's language learning efficiency can be improved.

FIG. 1 is a diagram illustrating an example of a hardware configuration of a Japanese language learning device 10 as an example of an utterance / utterance learning device according to the first embodiment. FIG. 2 is a diagram showing the arrangement of the airway microphone 116 and the bone conduction microphone 118 shown in FIG. FIG. 3 is a block diagram illustrating an example of a functional configuration (functional unit) of the Japanese language learning device 10 according to the first embodiment. FIG. 4 is a flowchart illustrating an example of a preparation process executed by the Japanese language learning apparatus 10 according to the first embodiment. FIG. 5 is a flowchart showing an example of the flow of Japanese language learning processing executed by the Japanese language learning device 10 according to the first embodiment. FIG. 6 is a flowchart showing an example of the flow of another preparation process executed by the utterance / utterance learning apparatus according to the second embodiment in addition to the preparation process shown in FIG. FIG. 7 is a flowchart illustrating an example of a language learning process performed by the utterance / utterance learning apparatus according to the second embodiment.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. In addition, in each figure, what attached | subjected the same code | symbol has the same or similar structure.

――― First embodiment ――――
First, the utterance / utterance learning apparatus according to the first embodiment will be described. In the first embodiment, a language learning device, particularly a Japanese language learning device will be described as an example of the utterance / utterance learning device.

<Hardware configuration>
FIG. 1 is a diagram illustrating an example of a hardware configuration of a Japanese language learning device 10 as an example of an utterance / utterance learning device according to the first embodiment. As shown in the figure, the Japanese language learning device 10 includes a central processing unit (CPU) 102, a random access memory (RAM) 104, a read only memory (ROM) 106, a drive device 108, and a network interface (Interface). ) 110, input device 112, display device 114, airway microphone 116, bone conduction microphone 118, and speaker 120. These components are connected to each other via a bus so as to be able to transmit and receive data.

  The CPU 102 is a control unit that controls each component connected to the CPU 102, calculates and processes data. The CPU 102 is an arithmetic unit that executes an application program stored in the RAM 104 or the ROM 106 for performing Japanese language learning processing (including voice creation processing) and preparation processing for the Japanese language learning processing. The CPU 102 receives data from the input device 112, the network I / F 110, and the like, calculates and processes the data, and then outputs the calculation result to the display device 114 and the storage device.

  The RAM 104 is, for example, a main storage unit. The RAM 104 is a storage device that stores or temporarily stores programs and data such as an OS (Operating System) and application software that are basic software executed by the CPU 102.

  The ROM 106 is a storage device that stores data related to application software, for example.

  The drive device 108 reads a program and data from a recording medium 108A, for example, a CD-ROM or an SD card, and installs or downloads the program or data in a storage device.

  In addition, a predetermined program is stored in the recording medium 108A, and the program stored in the recording medium 108A is installed in the Japanese language learning device 10 via the drive device 108. The installed predetermined program can be executed by the Japanese language learning device 10.

  The network I / F 110 is an interface between a peripheral device having a communication function and the Japanese language learning device 10. The network I / F 110 is connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network) constructed by a data transmission path such as a wired and / or wireless line.

  The input device 112 includes a keyboard having cursor keys, numeric input, various function keys, and the like, a mouse and a slide pad for selecting keys on the display screen of the display device 114, and the like. The input device 112 is a user interface for a Japanese learner (user) to give an operation instruction to the CPU 102 or to input data.

  The display device 114 is configured by an LCD (Liquid Crystal Display) or the like, and performs display according to display data input from the CPU 102.

  The airway microphone 116 is a sound input device that picks up the airway voice of the voice uttered by the user, converts it into an electrical signal, and inputs it into the Japanese language learning device 10.

The bone-conduction microphone 118 is a sound input device that collects bone-conduction speech uttered by the user, converts it into an electrical signal, and inputs it into the Japanese language learning device 10. The location where the bone-conduction microphone 118 is placed when picking up bone-conducted speech includes the user's ear canal, the top of the head, the forehead, the temporal bone, and the like. Among these, the bone-conducting microphone 118 is placed in the ear canal near the user's hearing device, which can be stably placed and can pick up the sound closest to the bone-conducting sound heard by the hearing device. Is preferred. In other words, as shown in FIG. 2, the bone conduction microphone 118 is preferably configured in shape, size, and the like so that it can be inserted and placed in the user's external auditory canal. FIG. 2 shows a microphone 119 configured with an airway microphone 116 and a bone conduction microphone 118 as a set. The microphone 119 is provided with a disk-shaped fixing portion 117 that fits into the ear canal. An airway microphone 116 is provided inside the fixed portion 117. On the other hand, a bone-conduction microphone 118 extending from the fixing portion 117 in the thickness direction is provided. The bone conduction microphone 118 is disposed in the external auditory canal, and the airway microphone 116 is disposed outside the external auditory canal.
That is, the airway microphone 116 is provided on the outer wall of the bone conduction microphone 118. The bone-conduction microphone 118 includes a piezo element that can collect bone-conduction sound. The bone-conduction microphone 118 disposed in the external auditory canal can collect sounds close to the bone-conduction sound heard by the user (user) even when the entire head is considered to be transmitted to the auditory sense. Further, when the airway microphone 116 is installed on the outer wall of the bone-conduction microphone 118, it is possible to capture the airway sound that is closest to the sound that the sound emitted from the oral cavity actually reaches its own ear.

  The speaker 120 is a sound output device that outputs sound based on an electrical signal to the outside of the Japanese language learning device 10.

<Functional configuration>
FIG. 3 is a block diagram illustrating an example of a functional configuration (functional unit) of the Japanese language learning device 10 according to the first embodiment. As illustrated, the Japanese language learning device 10 includes a storage unit 130, an input unit 132, a creation unit 134, an output unit 136, and an evaluation unit 138.

  The storage unit 130 is realized by, for example, the RAM 104, the ROM 106, the recording medium 108A, and the like. The storage unit 130 stores a plurality of model phonemes 130A created based on speech uttered by a model whose native language is Japanese, for example. In addition, the storage unit 130 stores a mixing ratio 130B for mixing the airway sound and the bone conduction sound. Note that the mixing ratio 130B is associated with each user, and is set to a different value.

  The input unit 132 is realized by the airway microphone 116 or the bone conduction microphone 118, for example. The input unit 132 inputs the airway sound and the bone conduction sound of the sound (first sound) uttered by the user.

The creation unit 134 is realized, for example, when the CPU 102 executes an application program stored in the RAM 104 or the ROM 106. The creation unit 134 creates the second voice by mixing the airway voice and the bone conduction voice input by the input unit 132 at a ratio constituting the voice that the user is listening to while maintaining the sound quality of the first voice. To do. The “ratio constituting the voice the user is listening to” on the left is, for example, a ratio designated or selected in advance by the user. Furthermore, in the present embodiment, the creation unit 134 converts the prosody of the first speech into a predetermined prosody when creating the second speech. The “predetermined prosody” on the left includes, for example, the prosody of speech uttered by a model whose native language is Japanese, and the prosody based on the speech of the model. Prosody includes pitch, speaking speed, intonation and the like.
Note that the creation unit 134 may modify the transmission characteristic of the first voice and create a voice closer to the voice heard by the user as the second voice.

  The output unit 136 is realized by the speaker 120, for example. The output unit 136 outputs a model voice based on the model phoneme 130A, a second voice, and the like.

  The evaluation unit 138 is realized by the CPU 102 executing an application program stored in the RAM 104 or the ROM 106, for example. The evaluation unit 138 evaluates the voice input by the input unit 132 based on a predetermined evaluation criterion.

<Preparation process>
FIG. 4 is a flowchart illustrating an example of a preparation process executed by the Japanese language learning apparatus 10 according to the first embodiment. This preparation process is started, for example, when the Japanese language learning apparatus 10 is activated, at a predetermined time, or when a user gives an instruction.

(Step SP10)
The input unit 132 inputs an airway voice and a bone conduction voice, which are voices uttered by the user. And a process transfers to the process of step SP12.

(Step SP12)
The creation unit 134 provisionally determines a mixing ratio when mixing the airway sound and the bone conduction sound. At this time, the creating unit 134 may display on the display device 114 a selection screen for selecting a provisionally determined mixture ratio and an input screen for inputting the mixture ratio. Further, the mixing ratio may be automatically and temporarily determined to an optimum value or the like. In the first embodiment, the creation unit 134 automatically determines a different mixing ratio each time step SP12 is repeated. And a process transfers to the process of step SP14.

(Step SP14)
The creating unit 134 creates a mixed sound by mixing the input airway sound and the bone conduction sound at (based on) the provisionally determined mixing ratio. And a process transfers to the process of step SP16.

(Step SP16)
The output unit 136 outputs the created mixed sound. And a process transfers to the process of step SP18.

(Step SP18)
The creation unit 134 determines whether or not there is an operation for determining one mixing ratio (provisionally determined mixing ratio) by the user. If a positive determination is made, the process proceeds to step SP20. If a negative determination is made, the process returns to step SP12.

(Step SP20)
The creation unit 134 finally determines the one mixing ratio that has been determined as the mixing ratio 130 </ b> B and stores it in the storage unit 130.

<Japanese learning process>
FIG. 5 is a flowchart showing an example of the flow of Japanese language learning processing executed by the Japanese language learning device 10 according to the first embodiment. This Japanese language learning process is started in response to a user instruction, for example.

(Step SP30)
The input unit 132 inputs text created by the user or text prepared for Japanese language learning in advance. When inputting the text created by the user, the input unit 132 inputs according to the user's operation. In addition, when inputting text prepared for learning Japanese, the input unit 132 acquires and inputs from the storage unit 130 or the Internet. And a process transfers to the process of step SP32.

(Step SP32)
The creation unit 134 creates model speech along the text based on the input text and the model phoneme 130A. Then, the output unit 136 outputs the created model sound. And a process transfers to the process of step SP34.

(Step SP34)
The input unit 132 inputs the first voice airway voice and bone conduction voice uttered by the user along the model voice. And a process transfers to the process of step SP36. In step SP34, the evaluation unit 138 evaluates the first voice based on the input airway voice and bone conduction voice. If the evaluation is equal to or higher than a predetermined value, the Japanese language learning process shown in FIG. May be.

(Step SP36)
The creation unit 134 converts the prosody of the first sound into a predetermined prosody while maintaining the sound quality of the first sound, based on the input airway sound and bone-conducted sound of the first sound, and converts the second sound into a predetermined prosody create. In the first embodiment, the creating unit 134 mixes the input airway sound and bone conduction sound based on the mixing ratio 130B to create a second sound. In the first embodiment, the creation unit 134 converts the prosody of the first speech into the prosody of the model speech. And a process transfers to the process of step SP38.

(Step SP38)
The output unit 136 outputs the created second sound. And a process transfers to the process of step SP40. Note that the output unit 136 may also output a voice such as “If you speak, it is easy to understand if you speak” before outputting the second voice.

(Step SP40)
The input unit 132 inputs the third voice uttered by the user along the second voice. At this time, the input unit 132 may input only the third voice of the airway voice. And a process transfers to the process of step SP42.

(Step SP42)
The evaluation unit 138 recognizes the input third voice, compares it with the input text, and evaluates whether the voice is correctly recognized, in other words, whether the voice is correctly spoken. In addition, the evaluation unit 138 analyzes the prosody (sound) of the third voice, and evaluates the degree of ease of transmission (clarity). Then, the evaluation unit 138 stores the evaluation result in the storage unit 130. The evaluation unit 138 may display the evaluation result on the display device 114 or the like. When displaying on the display device 114, the evaluation unit 138 displays advice such as “Let's emphasize the intonation!” Or “Let's speak with a clear voice break!” May be. And a process transfers to the process of step SP44.

(Step SP44)
The evaluation unit 138 determines whether or not the evaluation is greater than or equal to a predetermined value. If an affirmative determination is made, the process ends. If a negative determination is made, the process returns to step SP38. If a negative determination is made, instead of returning to step SP38 and outputting the second voice, the model voice that has been processed to clarify the inflection emphasis or utterance break is added to the third voice input in step SP40. It may be output.

  As mentioned above, according to 1st Embodiment, when the user who uttered the 1st audio | voice hears the produced 2nd audio | voice, the sound quality of the 2nd audio | voice feels equivalent to the sound quality of oneself who has always heard. Thus, it is possible to suppress feeling uncomfortable. If this sense of incongruity can be reduced, the user can learn the language smoothly, and can change the prosody of his / her voice according to the prosody of the second voice, so that his / her voice approaches the second voice. I feel a sense of accomplishment that I have approached two voices, and now I am trying to practice Japanese utterances, improving the language learning efficiency.

  In addition, according to the first embodiment, since the input airway sound and bone conduction sound are mixed to create the second sound, when the user hears the created second sound, the second sound The sound quality becomes more equal to the sound quality that I always listen to, and I can suppress the feeling of strangeness more.

  In addition, according to the first embodiment, the user can determine and operate one mixing ratio for actually mixing out of the mixing ratios of the airway sound and the bone conduction sound, so the user listens to the mixed sound while changing the mixing ratio, When the mixed sound is considered to be the same as one's own sound, an appropriate one mixing ratio can be determined and operated. As a result, when the user hears the created second sound, the sound quality of the second sound becomes more equal to his own sound quality that he / she always listens to, and the user feels more uncomfortable. Can be suppressed.

Second Embodiment
Next, a speech / speech learning apparatus according to the second embodiment will be described. The second embodiment is different from the first embodiment in that a synthesized voice is created as the second voice. Similar to the first embodiment, the second embodiment includes a storage unit 130, an input unit 132, a creation unit 134, an output unit 136, and an evaluation unit 138.

  FIG. 6 is a flowchart showing an example of the flow of another preparation process executed by the utterance / utterance learning apparatus according to the second embodiment in addition to the preparation process shown in FIG. The other preparation processing is started, for example, when the Japanese language learning device 10 is activated, at a predetermined time, or when a user gives an instruction.

(Step SP50)
The input unit 132 inputs an airway voice and a bone-conducted voice, which are voices obtained by the user uttering (reading) a predetermined manuscript for creating a synthesized voice for about 60 to 90 minutes, for example. And a process transfers to the process of step SP52.

(Step SP52)
The creation unit 134 analyzes the input airway speech and bone conduction speech. And a process transfers to the process of step SP54.

(Step SP54)
The creation unit 134 mixes the input airway voice and bone conduction voice based on the analysis result based on the mixing ratio 130 </ b> B, and creates a plurality of user phonemes (and phoneme strings) corresponding to the user's listening voice.

  FIG. 7 is a flowchart illustrating an example of a language learning process performed by the utterance / utterance learning apparatus according to the second embodiment. This language learning process is started in response to a user instruction, for example.

(Step SP60)
The input unit 132 inputs text for learning created by the user or text prepared for Japanese language learning in advance. When inputting the text created by the user, the input unit 132 inputs according to the user's operation. In addition, when inputting text prepared for learning Japanese, the input unit 132 acquires and inputs from the storage unit 130 or the Internet. And a process transfers to the process of step SP62.

(Step SP62)
The creation unit 134 creates a synthesized speech as the second speech along the text based on the input text and the user phoneme. And a process transfers to the process of step SP64. The creation unit 134 performs integrated Fourier analysis on the created synthesized speech together with the input airway speech, calculates an inverse difference characteristic, and gives the inverse synthesized characteristic to the synthesized speech, so that the user always listens. You may adjust to synthetic speech.

(Step SP64)
The output unit 136 outputs the created synthesized speech. And a process transfers to the process of step SP66.

(Step SP66)
The input unit 132 inputs the third voice uttered by the learner along the synthesized voice. At this time, the input unit 132 may input only the third voice of the airway voice. And a process transfers to the process of step SP68.

(Step SP68)
The evaluation unit 138 recognizes and evaluates the input third voice, and stores the evaluation result in the storage unit 130. The evaluation unit 138 may display the recognition result and the evaluation result on the display device 114. And a process transfers to the process of step SP70.

(Step SP70)
The evaluation unit 138 determines whether or not the evaluation is greater than or equal to a predetermined value. If an affirmative determination is made, the process ends. If a negative determination is made, the process returns to step SP64.

  As described above, according to the second embodiment, in addition to the same effects as those of the first embodiment, the process of outputting the model sound as shown in step SP32 in FIG. 5 can be omitted.

<Modification>
The present invention is not limited to the above embodiment. In other words, the above-described embodiment with appropriate design adjustments by those skilled in the art is also included in the scope of the present invention as long as it includes the features of the present invention. Moreover, each element with which embodiment mentioned above is provided can be combined as long as it is technically possible, and the combination of these is also included in the scope of the present invention as long as it includes the features of the present invention.

  For example, in the first embodiment, the case where the Japanese language learning apparatus 10 includes the airway microphone 116 and the bone conduction microphone 118 has been described, but instead of these, a bone conduction / airway integrated microphone may be provided.

  In the first embodiment, the case where the output unit 136 and the evaluation unit 138 are provided in the Japanese language learning device 10 has been described. However, the output unit 136 and the evaluation unit 138 may be provided in a device outside the Japanese language learning device 10. In this case, the Japanese language learning device 10 transmits the second voice to an external device.

10 ... Japanese language learning device (speech / utterance learning device), 132 ... input unit, 134 ... creation unit

Claims (6)

An utterance / speech learning device that allows a user to learn speech of speech,
An input unit for inputting the airway voice and the bone conduction voice constituting the first voice;
A creation unit that creates the second voice by mixing the airway voice and the bone conduction voice that are input at a ratio that constitutes the voice that the user is listening to while maintaining the sound quality of the first voice;
A speech / speech learning device comprising: The creation unit corrects transmission characteristics and creates a voice closer to the voice that the user is listening to as the second voice.
The utterance / utterance learning apparatus according to claim 1. The input unit further includes:
Based on the airway speech and the bone conduction speech that were input when a predetermined manuscript for creating synthesized speech was read by the user, a phoneme and a phoneme sequence corresponding to the speech that the user is listening to are created. As the second voice, a synthesized voice along the text input for learning is created.
The utterance / utterance learning apparatus according to claim 2. The input unit further inputs a determination operation of one mixing ratio of mixing ratios of the airway sound and the bone conduction sound,
The creating unit mixes the input airway sound and the bone conduction sound based on the one mixing ratio.
The utterance / utterance learning apparatus according to claim 2. The input unit inputs the bone conduction sound and is configured to be disposed in a user's external auditory canal; and the airway provided to the outer wall of the bone conduction microphone by inputting the airway sound. Including a microphone,
The utterance / utterance learning apparatus according to claim 1. A bone-conduction microphone configured to input a user's bone-conducted sound and be placed in the user's external auditory canal;
The user inputs the airway voice, and the airway microphone provided on the outer wall of the bone conduction microphone;
A microphone that collects the sound the user is listening to, including