JP2015011348A - Training and evaluation method for foreign language speaking ability using voice recognition and device for the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a method, in relation to a foreign language speaking ability training and evaluation method using voice recognition, for training and evaluating foreign language speaking ability by a user by using a computer or other measurement devices.SOLUTION: The method includes: a step S310 for inputting a foreign language voice signal generated by a first user; a step S320 for recording the voice signal that has been input; a step S330 for reproducing the voice signal that has been recorded to provide the signal to a second user; a step S340 for inputting text data corresponding to the voice signal that has been provided by the second user; a step S350 for comparing the text data that has been input with the voice signal that has been recorded to measure a degree of accuracy; and a step S360 for providing an evaluation result with the degree of accuracy that has been measured to the first user.

Description

  The present invention relates to a method for training and evaluating a foreign language speaking ability using speech recognition, and more particularly, to a method for a user to train and evaluate a foreign language speaking ability by using a computer or other measuring device.

  As the importance of speaking ability increases in many foreign language education processes, including English, the educational demand for this is increasing, and there is a large scale such as the National English Abilities Test (NEAT). There will also be an increase in speaking ability evaluation tests, thereby increasing the demand for training and evaluation equipment.

  Fig. 1 shows an example of a conventional foreign language speaking training or evaluation method. (A) When reading a predetermined script (b), (b) Speaking freely and the speech recognition system automatically recognizes the utterance content. It is drawing which shows the case where it does.

  First, as shown in FIG. 1A, when a script is determined in advance in a foreign language speaking training and evaluation method using a computer and other auxiliary devices, the user views the script and A speech signal is generated by reading according to the above. It has been a general method to record the voice signal generated in this way and compare the recorded signal with a pre-stored script to quantitatively evaluate the speaking ability.

  However, in such a case, since the script is prepared and read in advance, there is a difference from the actual speaking, and the effectiveness of the test is reduced.

  On the other hand, in order to solve the above-described problems, as shown in FIG. 1B, the automatic speech recognition system is used to automatically recognize the content spoken by the user, and using this, the speaking ability However, in the current speech recognition technology, if the recognition result is inaccurate, and if it is a slight recognition malfunction, the final evaluation result may change greatly. There was a problem that evaluation was impossible.

JP 2008-242437 A

  The present invention has been made to solve the above-described problems, and the present invention records the voice after the user speaks freely without a script first, and prompts the user immediately or after a certain period of time. The user listens to this and types what he / she uttered directly using an input device such as a keyboard, and compares the typed text and the recorded uttered text with each other to evaluate the speaking ability. Provide a way to do.

  Through this process, all the above-mentioned problems can be solved, and the user creates the text while reconfirming the contents spoken by himself. The point will be recognized by itself, and the educational effect can be further enhanced.

  In order to achieve the technical problem of the present invention, in an embodiment of the present invention, a foreign language speech signal uttered by a first user is input in a method for training and evaluating a foreign language speaking ability using speech recognition. A step of recording the input audio signal, a step of reproducing the recorded audio signal and providing it to a second user, and text data corresponding to the provided audio signal is stored in the second user. A step of measuring the accuracy by comparing the input text data with the recorded voice signal, and providing an evaluation result based on the measured accuracy to the first user. Steps.

  Preferably, the first user is a person to be evaluated in foreign language speaking ability training and evaluation using the voice recognition, and the second user is the same user as the first user.

  The step of reproducing the recorded audio signal and providing it to the second user is provided to the second user immediately after recording the input audio signal or after a predetermined n hours (n is a positive real number). It is preferable to do.

  The step of inputting text data corresponding to the provided speech signal includes inputting the whole text corresponding to the provided speech signal or an error in the text of the speech signal recognized by the speech recognition module. It is preferable that correction data for a part of text including “” is input.

  The step of measuring the accuracy is a step of converting the recorded voice signal from an analog voice signal to voice data which is a digital voice signal, and a character string in which the input text data is displayed by phonetic symbols. The step of converting to a pronunciation string, the step of aligning the converted pronunciation string with the converted voice data, and comparing the aligned pronunciation string and the voice data to determine the accuracy of the pronunciation string unit Preferably measuring.

  The step of quantitatively measuring the accuracy measures the accuracy based on whether or not the speech feature is included in the speech data, using the phonetic feature of the phonetic sequence. More preferably.

  In the step of quantitatively measuring the accuracy, the signal corresponding to each phoneme in the phonetic string is defined as a model, and a difference between the speech data and the defined model is calculated as a score. More preferably, the accuracy is measured.

  On the other hand, in order to solve the technical problem of the present invention, another embodiment of the present invention is a foreign language speaking ability training and evaluation apparatus using speech recognition, wherein a foreign language speech signal uttered by a first user is An input audio signal receiving unit, a recording unit for recording the input audio signal, an audio signal reproducing unit for reproducing the recorded audio signal and providing it to a second user, and the provided audio signal A text receiving unit in which text data corresponding to the second user is input, an accuracy measuring unit that measures the accuracy by comparing the input text data and the recorded voice signal, and the measurement And an evaluation unit that provides the first user with an evaluation result based on the accuracy obtained.

  Preferably, the first user is a person to be evaluated in foreign language speaking ability training and evaluation using the voice recognition, and the second user is the same user as the first user.

  The audio signal reproduction unit preferably provides the second user immediately after recording the input audio signal or after a predetermined n hours (n is a positive real number).

  The text receiving unit receives the whole text corresponding to the provided speech signal, or correction data for a partial text including an error in the text of the speech signal recognized by the speech recognition module. Preferably it is entered.

  The accuracy measuring unit includes an AD converter that converts the recorded audio signal from an analog audio signal to audio data that is a digital audio signal, and a pronunciation that is a character string in which the input text data is represented by phonetic symbols. A phonetic string conversion unit for converting to a string; an alignment unit for aligning the converted phonetic string with the converted voice data; and comparing the sorted phonetic string with the voice data to accurately determine the pronunciation string unit. And a sub-accuracy measuring unit that quantitatively measures the degree.

  The sub-accuracy measuring unit may further measure the accuracy based on whether or not the audio feature is included in the audio data using a phonetic feature of the pronunciation sequence. preferable.

  The sub-accuracy measurement unit defines the signal itself corresponding to each phoneme of the phonetic sequence as a model, and calculates the difference between the speech data and the defined model as a score, thereby calculating the accuracy. It is more preferable to measure.

  With the method for training and evaluating foreign language speaking ability using speech recognition according to the present invention, the user is guided to speak freely instead of reading a predetermined script, thereby training the actual speaking ability. And can evaluate this, and automatically compares the text entered by the user with the voice signal uttered to prevent erroneous evaluation results due to malfunctions in the existing speech recognition system. Can do.

  In addition, since a method of aligning utterance contents and character strings in units of phonemes using a speech recognition technique at the time of evaluation can be applied, fluency evaluation, pronunciation evaluation, and the like for a user's utterance can be performed more accurately.

It is an example of a conventional foreign language speaking training or evaluation method, and shows (a) when reading a predetermined script, (b) uttering freely, and the speech recognition system automatically recognizes the utterance content. It is a drawing. 6 is an exemplary diagram illustrating an evaluation result obtained by freely speaking using a foreign language speaking ability training and evaluation device according to an embodiment of the present invention. 4 is a flowchart illustrating a foreign language speaking ability training and evaluation method using speech recognition according to an embodiment of the present invention. FIG. 4 is a functional block diagram illustrating a foreign language speaking ability training and evaluation device using speech recognition according to an embodiment of the present invention. 4 is a diagram illustrating a method of measuring accuracy by comparing recorded voice data and text data according to an exemplary embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily implement the embodiments. However, the invention can be implemented in a variety of different forms and is not limited to the embodiments described herein.

  In order to clearly describe the present invention, portions not related to the description are omitted from the drawings, and portions that are the same or similar throughout the specification are given the same or similar drawings.

  In addition, throughout the present specification, when a component includes a certain component, the component may include another component without excluding the other component unless otherwise stated to the contrary. Means you can.

  Further, in adding reference numerals to the components of each drawing, the same components are given the same reference numerals as much as possible even if they are displayed in other drawings. Further, in describing the present invention, when it is determined that a specific description related to a known configuration or function is unclear, the detailed description thereof will be omitted.

  In describing the constituent elements of the present invention, terms such as first, second, A, B, (a), and (b) can be used. Such terms are used to distinguish the constituent elements from other constituent elements, and the essence, order, or order of the constituent elements are not limited by the terms. When a component is described as being “coupled”, “coupled” or “connected” to another component, the component can be directly coupled or connected to another component, and each component It should be understood that other components may also be “coupled”, “coupled” or “connected” between the elements.

  The present invention relates to a method for training and evaluating a foreign language speaking ability using a speech recognition system, and automatically converts the content spoken by a user into characters by using speech recognition technology to convert foreign language utterance content. In order to compensate for errors in speech recognition technology in the process of evaluating in, the user who uttered input the utterance content with characters using an input device such as a keyboard, and the input character data and recorded voice data The present invention relates to a method and apparatus for self-training by comparing a user's foreign language ability and providing the user with an accurate evaluation result.

  Hereinafter, a method for training and evaluating a foreign language speaking ability using speech recognition according to the present invention will be described in more detail with reference to various drawings.

  FIG. 2 is an exemplary drawing for freely speaking and receiving evaluation results using a foreign language speaking ability training and evaluation apparatus according to an embodiment of the present invention.

  As described briefly earlier, in the case of a foreign language speaking ability evaluation system using conventional speech recognition, when a user reads a predetermined sentence, the user's pronunciation ability or intonation is evaluated. Or, when the user speaks freely, automatic speech recognition is attempted, and whether or not the recognized result matches the grammar and the foreign language method is evaluated.

  In the former case, it is impossible to evaluate the ability to speak freely, and in the latter case, a consistent evaluation is impossible due to an error in the automatic speech recognition system.

  Therefore, as shown in FIG. 2, in the present invention, after the user utters freely, this is recorded and reproduced by the training and evaluation device, and the user hears the recorded utterance content and Based on this, the evaluation device measures the accuracy with respect to the previously recorded voice data, and evaluates the user's speaking ability evaluation result (result). )I will provide a.

  By using the method as described above, the user can freely train foreign language speaking without a prescribed script, and prevent the possibility of an error in the evaluation system due to a malfunction of the existing automatic speech recognition system. it can.

  A more detailed operation process will be described with reference to FIGS. 3 and 4 below.

  FIG. 3 is a flowchart illustrating a foreign language speaking ability training and evaluation method using speech recognition according to an embodiment of the present invention.

  Referring to FIG. 3, the training and evaluation method of the present invention includes a step (S310) in which a foreign language speech signal uttered by a first user is input, a step (S320) in which the input speech signal is recorded, The step of reproducing the recorded voice signal and providing it to the second user (S330), the step of inputting text data corresponding to the provided voice signal from the second user (S340), and the inputted text Comparing data with the recorded audio signal to measure accuracy (S350), and providing an evaluation result based on the measured accuracy to the first user (S360).

  Here, the first user and the second user described separately may be the same user or other users. The second user may be an automated device (eg, an automatic speech recognition system).

  Usually, in the case of English speaking ability evaluation, the user (ie, the evaluated person) speaks freely in the given environment, and the evaluator records this and then listens to it after the fact, how fluent It will be quantitatively evaluated whether the voice is spoken.

  The present invention relates to making the step of listening to and evaluating the audio file recorded in this process automatically performed by an automatic device such as a computer, and is roughly divided into five steps. ) A step where the evaluated person speaks freely according to the given environment or mission, (2) a step of recording the content to be uttered, (3) a step of letting the evaluated person hear the recorded content, (4) recording A step in which the person to be evaluated listens to the received content and converts and inputs it into text using an input device such as a keyboard, and (5) the spoken ability is automatically evaluated by comparing the input text with the recorded data. It can consist of steps.

  In the step of the above-mentioned evaluated person listening to the recorded content and converting it into text using an input device such as a keyboard, the evaluated person may create the entire text from the beginning to the end, or voice recognition. Of the contents recognized by the module, only a part containing errors by the evaluator may be corrected.

  Through these many steps, the user not only has the opportunity to reconfirm the content he uttered, but can also prevent recognition and conversion malfunctions of the entire speech recognition system, so The accuracy and consistency in evaluating the speaking ability itself, such as intonation, can be greatly improved.

  Hereinafter, the signal and data transmission process between the detail modules constituting the training and evaluation apparatus will be described in detail.

  FIG. 4 is a functional block diagram illustrating a foreign language speaking ability training and evaluation apparatus using speech recognition according to an embodiment of the present invention.

  Referring to FIG. 4, the training and evaluation apparatus 400 of the present invention includes an audio signal receiving unit 410 to which a foreign language audio signal uttered by a first user is input, a recording unit 420 that records the input audio signal, An audio signal reproducing unit 430 that reproduces a recorded audio signal and provides it to a second user, a text receiving unit 440 that inputs text data corresponding to the provided audio signal by the second user, and an input An accuracy measuring unit 450 that measures the accuracy by comparing text data with the recorded audio signal, and an evaluation unit 460 that provides an evaluation result based on the measured accuracy to the first user.

  The data transmission process between the detailed modules constituting the training and evaluation apparatus 400 will be described. First, a voice uttered in a foreign language by the person to be evaluated is input to the voice signal receiving unit 410.

  The audio signal receiving unit 410 transmits this to the recording unit 420, and the recording unit 420 records the input audio signal and transmits it to the accuracy measuring unit 450 for subsequent accuracy measurement and evaluation. .

  On the other hand, the recorded audio signal is provided to the evaluated person again by the audio signal reproducing unit 430. At this time, the recorded audio signal may be provided to the evaluated person immediately after recording the audio signal, and may be provided for a predetermined n hours (n May be provided to the evaluated person after a positive real number).

  As described above, the recorded voice signal can be provided to the person to be evaluated, and can be used as a basic material for evaluation after being provided to another user (or a voice recognition system). it can.

  Thereafter, the evaluator who listened to the recorded voice inputs the whole text corresponding to the provided voice signal, or an error is detected in the whole text of the voice signal recognized by the voice recognition system. The script text is input by inputting correction data for the included partial text, and the text receiving unit 440 in the training and evaluation apparatus 400 receives the script text.

  Then, the text receiving unit 440 transmits the text data to the accuracy measuring unit 450 and uses it as a basic material for voice signal evaluation.

  The detailed module constituting the accuracy measurement unit 450 will be described. The accuracy measurement unit 450 converts an audio signal recorded from an analog audio signal into audio data that is a digital audio signal, and an AD converter 451 input thereto. A pronunciation string conversion unit 453 that converts text data into a pronunciation string that is a character string displayed as a phonetic symbol; an alignment unit 452 that aligns the converted pronunciation string with the converted voice data; A sub-accuracy measuring unit 454 that compares the generated pronunciation sequence with the audio data to quantitatively measure the accuracy of the pronunciation sequence unit.

  Finally, the final evaluation result for the utterance is fed back to the person to be evaluated through the evaluation unit 460 according to the measurement result of the accuracy measurement unit 450 performing the above role.

  FIG. 5 is a diagram for explaining a method of measuring accuracy by comparing voice data and text data recorded according to an embodiment of the present invention.

  The following method can be used as a specific method for measuring and evaluating the speaking ability using the voice signal (or converted voice data) uttered by the user and the text data input by the user. .

  First, text data input by the user is converted into a pronunciation string. The pronunciation string means a character string in which the English sentence is displayed with a phonetic symbol.

  After that, such a pronunciation string is aligned with voice data uttered by the user (forced to align).

  The above process is a process of accurately matching a section corresponding to the pronunciation string in the voice data by using a general voice recognition system.

  Here, as described above, the user himself / herself changes the content of the utterance into the character string (text), so that the error in the alignment process can be greatly reduced.

  When the pronunciation string and the voice signal are aligned, the voice signal is analyzed in units of the pronunciation string, and it becomes possible to quantitatively measure how accurately the user has uttered the pronunciation string.

  In this step, it can be used as a means for measuring the accuracy of whether or not the feature is included in the speech signal using the phonetic feature of the pronunciation string. For example, in the case of voiced sounds such as / b /, / d /, and / g /, it can be used whether or not the characteristics of the voiced sound are in the audio signal.

  In addition to the individual phonetic features, the speech signal itself corresponding to each phoneme is defined as a model by another method, and the difference between the current user signal and the stored model is calculated as a score. There is also a method.

  Referring to FIG. 5, the input text 520 is converted into a pronunciation string 530, and after being aligned with the recorded voice data 510, the model of the voice signal in the section corresponding to each pronunciation string (541 b to 543 b). An evaluation result can be calculated 550 by calculating the difference between (541a to 543a) with each score and totaling the total.

  Thus, using the foreign language speaking ability training and evaluation method according to the present invention, the actual speaking ability can be trained and guided by guiding the user to speak freely rather than reading the prescribed script. In addition to being able to evaluate, the sentence inputted by the user and the voice signal uttered are automatically compared to prevent an error of the evaluation system due to an automatic voice recognition malfunction.

In addition, there is an effect that fluency evaluation and pronunciation evaluation for user utterance can be performed more accurately by applying a method of aligning utterance contents and character strings in units of phonemes using a speech recognition technique.
In the above description, it has been described that all the components constituting the embodiment of the present invention are combined into one or operate by being combined, but the present invention is not limited to such an embodiment. In other words, all the components can be selectively combined and operated within the scope of the present invention. In addition, all of the components can be realized as one independent hardware, and some or all of the components are selectively combined and combined with one or more hardware. It can also be realized as a computer program having program modules that perform some or all of the functions. Codes and code segments constituting the computer program can be easily inferred by those skilled in the art of the present invention. Such a computer program is stored in a computer-readable storage medium, and can be read and executed by the computer to implement the embodiments of the present invention. The computer program storage medium may include a magnetic recording medium, an optical recording medium, and the like.

  In addition, terms such as “including”, “constituting”, or “having” described above mean that the component can be included unless otherwise stated, and exclude other components Rather, it should be construed that it can further include other components. All terms, including technical or scientific terms, unless defined differently have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Commonly used terms such as those defined in the dictionary should be construed as consistent with the contextual meaning of the related art and are ideal unless explicitly defined in the present invention, Or it is not overly interpreted in a formal sense.

  The above description is merely illustrative of the technical idea of the present invention, and a person having ordinary knowledge in the technical field to which the present invention belongs can be used without departing from the essential characteristics of the present invention. It will be understood that various modifications and variations are possible. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but are for explanation, and the scope of the technical idea of the present invention is not limited by such examples. The protection scope of the present invention should be construed by the claims of the present invention, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the right of the present invention.

Claims (14)

In foreign language speaking ability training and evaluation method using speech recognition,
Inputting a foreign language voice signal uttered by the first user;
Recording the input audio signal;
Replaying and providing the recorded audio signal to a second user;
Text data corresponding to the provided audio signal is input by the second user;
Comparing the input text data and the recorded audio signal to measure accuracy;
Providing an evaluation result based on the measured accuracy to the first user;
A foreign language speaking ability training and evaluation method using speech recognition characterized by comprising:   The first user is an evaluated person in a foreign language speaking ability training and evaluation using the voice recognition, and the second user is the same user as the first user. A foreign language speaking ability training and evaluation method using the speech recognition according to 1. Replaying the recorded audio signal and providing it to the second user comprises:
2. The foreign user using voice recognition according to claim 1, wherein the input voice signal is provided to the second user immediately after recording or after a predetermined n hours (n is a positive real number). Language speaking ability training and evaluation methods. The step of inputting text data corresponding to the provided audio signal includes:
The entire text corresponding to the provided speech signal is input, or correction data for a partial text including an error in the text of the speech signal recognized by the speech recognition module is input. A foreign language speaking ability training and evaluation method using speech recognition according to claim 1. Measuring the accuracy comprises:
Converting the recorded audio signal from an analog audio signal to audio data which is a digital audio signal;
Converting the input text data into a phonetic string that is a character string displayed with phonetic symbols;
Aligning the converted pronunciation sequence with the converted audio data;
The foreign language using speech recognition according to claim 1, further comprising the step of comparing the aligned pronunciation sequence and speech data to quantitatively measure the accuracy of the pronunciation sequence unit. Speaking ability training and evaluation methods. The step of quantitatively measuring the accuracy includes:
6. The accuracy of claim 5, wherein the accuracy is measured based on whether the phonetic feature is included in the voice data using a phonetic feature of the phonetic sequence. Foreign language speaking ability training and evaluation method using speech recognition. The step of quantitatively measuring the accuracy includes:
A signal corresponding to each phoneme of the phonetic sequence is defined as a model, and the accuracy is measured by calculating a difference between the speech data and the defined model as a score. A foreign language speaking ability training and evaluation method using voice recognition according to claim 5. In foreign language speaking ability training and evaluation equipment using speech recognition,
An audio signal receiving unit to which a foreign language audio signal uttered by the first user is input;
A recording unit for recording the input audio signal;
An audio signal reproduction unit that reproduces the recorded audio signal and provides it to a second user;
A text receiving unit in which text data corresponding to the provided audio signal is input by the second user;
An accuracy measurement unit that measures the accuracy by comparing the input text data and the recorded voice signal;
An evaluation unit that provides the first user with an evaluation result based on the measured accuracy;
A foreign language speaking ability training and evaluation device using speech recognition, characterized by comprising:   The first user is an evaluated person in a foreign language speaking ability training and evaluation using the voice recognition, and the second user is the same user as the first user. A foreign language speaking ability training and evaluation device using the voice recognition according to 8. The audio signal reproduction unit is
9. The foreign language using voice recognition according to claim 8, wherein the input voice signal is provided to the second user immediately after recording or after a predetermined n hours (n is a positive real number). Speaking ability training and evaluation equipment. The text receiving unit
The entire text corresponding to the provided speech signal is input, or correction data for a partial text including an error in the text of the speech signal recognized by the speech recognition module is input. A foreign language speaking ability training and evaluation device using speech recognition according to claim 8. The accuracy measuring unit includes:
An AD converter that converts the recorded audio signal from an analog audio signal to audio data that is a digital audio signal;
A phonetic string conversion unit that converts the input text data into a phonetic string that is a character string displayed with phonetic symbols;
An alignment unit for aligning the converted pronunciation sequence with the converted audio data;
The speech recognition according to claim 8, further comprising: a sub-accuracy measurement unit that quantitatively measures the accuracy of the phonetic string unit by comparing the aligned phonetic strings and voice data. Foreign language speaking ability training and evaluation device used. The sub-accuracy measuring unit is
The method according to claim 12, wherein the accuracy is measured based on whether or not the phonetic feature is included in the voice data using a phonetic feature of the phonetic sequence. Foreign language speaking ability training and evaluation device using speech recognition. The sub-accuracy measuring unit is
The accuracy is measured by defining a signal corresponding to each phoneme in the phonetic sequence as a model, and calculating a difference between the speech data and the defined model by a score. Item 13. A foreign language speaking ability training and evaluation device using speech recognition according to Item 12.

Images