KR101973791B1 - Method for correcting voice - Google Patents

Abstract

The embodiments of the present invention provide a speech evaluation and learning method in which a speech learning content app installed in a learner terminal performs speech training for a learner, the speech evaluation and learning method comprising: a speech training content selection process in which a speech training content to be used for speech training is selected from a learner; A spontaneous speech test process for providing a spontaneous speech test screen of the selected speech training contents to allow a spontaneous speech test to be performed; And an empirical test screen for performing a test on a mimetic word following the speech model of the language model used in the speech training content is provided so as to perform the mimetic word test if the self- An empirical test process; And a speech conversion test screen for testing imitation of the speech-converted language model generated by reflecting the result of the imitation test is provided so as to perform the speech conversion test when the imitation test result is less than a preset imitation evaluation criterion A speech conversion test process; And a speech learning process for providing learner customized learning data reflecting the test results of the speech test, the speech test, and the speech conversion test so that the learner's speech learning is performed.

Description

{Method for correcting voice}

The present invention relates to a speech evaluation and learning method, and a speech evaluation and learning method that provides speech learning.

In the treatment and education methods of general developmental disorders, treatment and education through treatment specialists and specialists can provide professional treatment for persons with disabilities, There are disadvantages such as lack of accessibility to use, lack of time for treatment, and waiting time for treatment. In addition, treatment and education through private treatment facilities and welfare facilities have advantages such as low cost of treatment but shortcomings such as lack of expertise of therapists, inferiority of facilities, lack of connection with professional organizations, and waiting time for treatment. In addition, treatment and education using PC treatment S / W program has advantages of reduction of economic burden, accessibility and convenience, but it is difficult to apply the same method as learning method of phonemic point which can not take into consideration individual characteristics of learners There are disadvantages.

In contrast to the importance of early detection and early intensive intervention in language development, such conventional developmental disability treatment and education methods are inadequate for accessibility, delayed treatment due to increase in waiting time, lack of time for language therapy due to economic burden, It is difficult to provide immediate feedback on the result of the treatment, and there is a problem in that it lacks professional evaluation and advice.

On the other hand, language education for the children of multicultural families, which are growing rapidly in recent years, is necessary, but most of them are only possible through visiting specialist institutions. In addition, although the number of people with disabilities is increasing every year, proportionately there is a lack of welfare facilities and treatment / education facilities.

As a result of the lack of such treatment / education institutions, there is a concern that the quality of the treatment / education service is deteriorated and the treatment / education time is delayed or the treatment / education period is prolonged due to lack of time for treatment / education.

Thus, although language learning contents are provided in recent years, there is a problem that the effectiveness of language learning is inferior. In other words, the existing language learning contents are produced and provided to the user. Most of the recording scripts are provided only to the user with one voice and one pronunciation speed, and are limited to the method of lecturing and modeling the pronunciation method of the language model .

In addition, although the pronunciation speed is mostly normal pronunciation and speed of the native language user, even if the convenient repetition function of the computer and the Internet is utilized for the language learning, there is a problem that the effect of the language learning can not be expected only by providing such one type of contents.

For learners who need language therapy and education, there are reasons that pronunciation is poor, but there are also characteristics that lack hearing ability. In other words, lack of dexterity and language discrimination ability of language, lack of hearing information processing ability such as hearing aids, auditory memory width, and so on, the pronunciation accuracy varies according to the presented speech speed. Therefore, based on the learner's speech perception level based on the linguistic information processing process, learners can grasp the characteristics of the speech stimulation that can accurately hear the language and adjust it to help the learner accurately recognize and speak the language We need a system. Also, speaking learning should be fun and natural, so that learners can learn enough language until they are mastered. Therefore, various learning games performed by learner 's linguistic reaction can expect language learning effect and voluntary promotion effect through iterative practice.

Korean Patent Publication No. 10-2010-0005177

The technical problem of the present invention is to provide a learner who is required to diagnose and treat language (an infant with slow language development, a language disorder including hearing loss and articulation disorder, an elderly person with language problem due to hearing loss, And to provide a speech evaluation and learning method.

The embodiments of the present invention provide a speech evaluation and learning method in which a speech learning content app installed in a learner terminal performs speech training for a learner, the speech evaluation and learning method comprising: a speech training content selection process in which a speech training content to be used for speech training is selected from a learner; A spontaneous speech test process for providing a spontaneous speech test screen of the selected speech training contents to allow a spontaneous speech test to be performed; And an empirical test screen for performing a test on a mimetic word following the speech model of the language model used in the speech training content is provided so as to perform the mimetic word test if the self- An empirical test process; And a speech conversion test screen for testing imitation of the speech-converted language model generated by reflecting the result of the imitation test is provided so as to perform the speech conversion test when the imitation test result is less than a preset imitation evaluation criterion A speech conversion test process; And a speech learning process for providing learner customized learning data reflecting the test results of the speech test, the speech test, and the speech conversion test so that the learner's speech learning is performed.

The speech training content selection process may be characterized by selecting a learning target level of speech to be used for speech training.

Wherein the spontaneous speech test is a spontaneous speech display step of displaying a picture or a letter capable of generating phonemes, syllables, words, and sentences of a spontaneous word corresponding to the learning target level; A spontaneous speech preprocessing process for eliminating noise by inputting speech of a learner who speaks spontaneous speech; A spurious response evaluation process in which a response evaluation is performed on the noise of the noise canceled spontaneous speech; A pronunciation evaluating step of providing a feedback result by proceeding pronunciation evaluation when the result of the self-verbal response evaluation meets the self-verbal evaluation criterion; And an empirical test movement process of providing the feedback of the spontaneous word test failure and proceeding to the empirical word test when the result of the spurious word reaction evaluation is less than the spurious word evaluation criterion.

Wherein the mimic word test process includes: displaying a mimetic word by displaying a phonemic, a syllable, a word, and a sentence of a mimetic word corresponding to the learning target level as a photograph or a letter and following a speech of a language model; A preprocessing process of removing a noise by inputting a voice of a learner who speaks a mimetic word; An empirical response evaluation process for evaluating the response to the voice of the noise canceled mimic word; A pronunciation evaluation process of performing a pronunciation evaluation and providing a feedback result when the pronunciation evaluation meets the pronunciation evaluation result, And a voice conversion test movement step of providing feedback of a pronunciation test failure and proceeding to the voice conversion test when the result of the voice response evaluation is less than the voice recognition evaluation criterion.

The voice conversion test process may include: a voice conversion generation process of generating and presenting a voice conversion of a learning target level by reflecting the result of the speaker test; A speech conversion presentation process for presenting a speech of a speech-converted language model; A speech conversion pre-processing step of inputting speech of a learner who speaks through speech conversion and removing noise; A voice conversion pronunciation evaluation feedback providing step of performing a reaction evaluation on the learner's voice with noise removed and proceeding with pronunciation evaluation if the dictionary meets the predetermined voice conversion evaluation criteria to provide a feedback result; And a voice conversion reaction evaluation feedback providing step of providing feedback of the voice conversion test failure when the voice conversion evaluation criterion is not satisfied as a result of the evaluation of the learner voice following the voice conversion.

The voice conversion test process repeats the voice conversion generation process, the voice conversion preprocess process, the voice conversion pronunciation evaluation feedback providing process, and the voice conversion reaction evaluation feedback providing process until the voice response of the learner meets predetermined criteria Can be characterized.

In the non-verbal response evaluation, the reaction rate, the screen transition, and the gaze treatment are evaluated. The pronunciation evaluation includes a phoneme level, a syllable level, a word level , Pronunciation evaluation based on the language model for the sentence level, and assessment of the learners' learning target standards and learning levels.

Wherein the speech learning process comprises: providing the learner-customized learning material; A step of receiving learner information including a voice, a mouth shape, a tongue shape, and breathing of a learner learning the learner customized learning material; Removing noise from the learner information; A step of learning evaluation of whether or not learner information whose noise has been removed meets a predetermined learning evaluation standard; Repeating the learner information input and the learning evaluation by modifying the learner customized learning material when the learning evaluation result is less than the learning evaluation standard; And a speech learning game progress process in which the speech learning game is progressed and learned when the learning evaluation result meets the learning evaluation standard.

The speech learning game is characterized in that the learner is made up of already learned speech or speech being learned, and the win or loss of the game is determined by the learner's speech response.

According to the embodiment of the present invention, a learner who needs language diagnosis and treatment can perform speech training efficiently by providing speech training training.

1 is a perspective view of a learner terminal capable of evaluation and learning speech according to an embodiment of the present invention;
2 is a block diagram of a configuration of a learner terminal according to an embodiment of the present invention;
3 is a flowchart illustrating a speech evaluation and learning process according to an embodiment of the present invention.
4 is a flowchart illustrating a spontaneous speech test process according to an embodiment of the present invention.
FIG. 5 is a diagram illustrating a preprocessing module according to an embodiment of the present invention. FIG.
FIG. 6 is a diagram illustrating a feedback module according to an embodiment of the present invention, and FIG. 7 is a diagram illustrating a speech evaluation module according to an embodiment of the present invention.
FIG. 8 is a flow chart illustrating an empirical test process according to an embodiment of the present invention; FIG.
9 is a diagram illustrating a voice conversion test process according to an embodiment of the present invention.
FIG. 10 is a diagram illustrating a voice conversion module according to an embodiment of the present invention. FIG.
11 is a flowchart illustrating a speech learning process according to an embodiment of the present invention.
FIG. 12 is a diagram illustrating a speech learning module according to an embodiment of the present invention. FIG.
FIG. 13 is an exemplary illustration of a speech learning game screen according to an embodiment of the present invention. FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The advantages and features of the present invention, and how to achieve them, will be apparent from the following detailed description of embodiments thereof taken in conjunction with the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the exemplary embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of the invention to those skilled in the art. And the present invention is only defined by the scope of the claims. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

FIG. 1 is a perspective view of a learner terminal capable of evaluation and learning speech according to an embodiment of the present invention, and FIG. 2 is a block diagram of a learner terminal according to an embodiment of the present invention.

In the present invention, when a learner speaks, the learner terminal 10 selects and edits only improper pronunciations so that the learner can hear the correct pronunciation, hears the learner's speech again, It is a language learning tool that ultimately corrects pronunciation problems.

To this end, the learner terminal 10 of the present invention is provided with an application (APP) for training learners who need to learn speech, such as a language disorder and a developmental disorder, and performs speech training through such an application .

The learner terminal 10 of the present invention will be described by taking a smart phone as an example in the drawing of Fig. 1, but it is possible to use not only a smart phone but also a desktop PC, a tablet PC, a slate PC A notebook computer, a digital broadcasting terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a navigation, and the like. Of course, the terminal to which the present invention can be applied is not limited to the above-described types, and it is of course possible to include various terminals.

2, the learner terminal 10 of the present invention includes a terminal communication unit 11, a terminal memory 12, a terminal display unit 13, a terminal input unit 14, a terminal control unit 15).

The terminal communication unit 11 is a module that performs a function of communicating through a mobile communication network. When performing mobile communication such as 3G or 4G, the terminal communication unit 11 includes an RF transmitter (not shown) for up-converting and amplifying the frequency of a radio- An RF receiver (not shown) for low-noise amplifying the received radio signal and down-converting the frequency, and the like.

The terminal memory 12 is a storage body in which the speech training contents (spontaneous speech, mimic speech, voice conversion, etc.) of the present invention, screen graphic interface (GUI) Such a memory is a module capable of inputting and outputting information such as a flash memory, a CF card (Compact Flash Card), and an SD card (Secure Digital Card) There may be.

The terminal display unit 13 is a module for displaying a spontaneous speech test screen, a mimetic word test screen, a speech conversion test screen, and a learner-customized learning material.

The terminal input unit 14 is a module that receives input from a learner through a voluntary language test screen, an empirical test screen, a voice conversion test screen, and a learner-customized learning material displayed through the terminal display unit 13. The terminal input unit 14 and the terminal display unit 13 may be implemented as a single type of touch screen panel. A touch screen panel is a display window provided on a front surface of a terminal to provide a touch screen screen capable of simultaneously performing input and display, and includes a graphical user interface (GUI) for communicating with a learner Display.

The terminal control unit 15 is implemented by an MCU (Main Control Unit) that controls each function module of the learner terminal 10 and is a module in which the speech learning content app of the present invention is installed. For reference, the learner terminal 10 implemented with a smart phone or the like can install, add or delete hundreds of various applications (application programs) as desired by the user, And can implement an appropriate interface through various applications. Therefore, you can download the speech learning content app from Google Market, Apple Store, etc. and install it on the smartphone.

FIG. 3 is a flow chart illustrating a speech evaluation and learning process according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a self-test word test process according to an embodiment of the present invention. FIG. 6 is a diagram illustrating a feedback module according to an embodiment of the present invention. FIG. 7 is a diagram illustrating a speech evaluation module according to an embodiment of the present invention, and FIG. FIG. 9 is a flowchart illustrating a voice conversion test process according to an embodiment of the present invention. FIG. 10 is a flowchart illustrating a voice conversion test process according to an embodiment of the present invention. FIG. 11 is a flow chart illustrating a speech learning process according to an embodiment of the present invention. FIG. 12 is a diagram illustrating a speech learning module according to an embodiment of the present invention. FIG. Yes In accordance with an exemplary figure of speech learning the game screen.

In a method of evaluating and learning a speech in which a speech learning content app installed in a learner terminal performs speech training for a learner, the speech evaluation and learning process of the present invention includes a speech training content selection process (S100), a spontaneous speech test process An empirical test process S300, a voice conversion test process S400, and a speech learning process S500.

The speech training content selection process (S100) is a process of selecting a speech training content to be used for speech training from a learner, and selects a target level of speech to be used in speech training.

If you specify the selection process of the speech training content, the speech selection can be selected at the lexical topic level or at the pronunciation level. This is appropriate for learners who are lacking in infant or vocabulary skills when choosing them by vocabulary themes. In early language development, language comprehension must precede language expressiveness, so it is easy to teach a new language or pronunciation if the learner is familiar with or likes a vocabulary topic. Because the more vocabulary you had to hear and speak, the easier it is to learn.

If you choose from pronunciation level, it is suitable for the person whose pronunciation learning is important. For these learners, it is more effective to present learning materials among similar pronunciation types in order to enhance the pronunciation training effect. Therefore, it is divided into two levels so that learners can quickly search for the speech to be learned. And the learner 's search data is used to construct learner' s characteristics and customized learning materials together with the speech evaluation result. Then select the test speech level to determine the final speech. The test utterance level selection can be one of phoneme, syllable, word, sentence level according to the learner 's speech level. At the syllable level, you can select the beginning, neutral, and consonant positions of the consonants. At the word level, you can choose the number of syllables and the position of the target pronunciation on the word. For example, even if 'A' is the target pronunciation, 'A' may be a word that corresponds to the beginning of the first syllable, such as 'Pepper', or a word that corresponds to the beginning of the second syllable, such as 'baby'. At the sentence level, you can select the number of words in a sentence or choose the number of sentences. This is because there are differences in the accuracy and sound intensity of the pronunciation of the same speech depending on the position of the sound. Because the learner's pronunciation accuracy differs according to the position of the target stimulus and the number of the syllables of the target stimuli, the pronunciation is analyzed by the phonemes, syllables, words, and sentence units. For example, 'Candy' / 'Candy' / 'Candy' can be pronounced correctly according to the position of 'San', such as 'Candy' and 'Moving' I can not. After choosing the level of speech, the target stimulus is determined, and the final speech stimulus, including speech and speech levels, is determined for learner speech evaluation and learning.

Meanwhile, the spontaneous speech test process (S200) provides a spontaneous speech test screen of the selected speech speech training contents so that the spontaneous speech test is performed. Here, 'spontaneous language', as it is known, is a word that expresses itself as thoughts, not words learned or repeatedly learned. For example, when asked, "How old are you?", Your mother or neighbor should say "4 years old" or "4 years old" It expresses itself in words by oneself. This is how you can communicate yourself without urging or guidance.

As shown in FIG. 4, the spontaneous speech test is a spontaneous speech display process (S210) for displaying a picture or a character capable of generating phonemes, syllables, words, and sentences of a language meeting a learning target level, A spontaneous speech preprocessing step S230 of inputting voices of the learner to receive noise S220, a spontaneous speech response evaluation process S240 of evaluating the response of the noisy speech of the spontaneous speech, If the evaluation result meets the spoken language evaluation criteria (S240a), the pronunciation evaluation process is performed (S250), and the spontaneous pronunciation evaluation process and the spoken language reaction evaluation process (S240) (S240b), feedback of the spontaneous test failure (S261), and an empirical test moving process that proceeds to the empirical test (S300).

That is, in the spontaneous test, the learner's voluntary response is analyzed when a photograph or a letter including a test target pronunciation is presented. If the learner's response is correct, the pronunciation evaluation is performed. If the learner's pronunciation is correct, the test will proceed to the other test. If there is a pronunciation error, the learner's test will proceed to the pronunciation test. Also, if the learner 's speech is wrong, the test will be conducted.

In the above, the spontaneous speech preprocessing step (S230) is a preprocessing process of receiving a learner voice uttering a spontaneous speech and removing noise. As shown in FIG. 5, noise removal, feedback elimination, speech enhancement, Lips image, tongue motion image, and voice / character conversion. In other words, the preprocessing process extracts noises that reduce the ambient noise so that the learner's voice is suitable for evaluation, eliminates feedback that amplifies only the speech from input speech and removes other noise, enhances speech, and extracts a specific vector. From the video data of the input learners, only the mouth shape and the tongue movement are extracted and information is extracted for evaluation. Converts the learner's voice to text.

In addition, the feedback on the spontaneous response evaluation and the feedback on the spontaneous pronunciation evaluation can be performed in the feedback module shown in FIG.

The feedback module shown in Fig. 6 includes reaction evaluation feedback, pronunciation evaluation feedback, oral evaluation feedback, and compensation feedback.

The response evaluation feedback can present visual feedback, auditory feedback, 3D character feedback, and the like of the learner's response during evaluation and learning.

The pronunciation evaluation feedback is composed of pronunciation evaluation feedback and graphical feedback. The pronunciation evaluation feedback provides the visual feedback based on the pronunciation evaluation of the learner's voice. The graphical feedback provides the spectrum of the learner's voice, It is presented as graphical feedback, which can be presented in spontaneous pronunciation test, mimetic language test, voice conversion test for speech evaluation, and can be utilized as learning material in learning stage. Oral evaluation feedback is presented as a video of the learner's mouth shape and tongue motion image, and the points to be corrected are presented graphically. Compensation feedback provides feedback to learners in ways such as scoring. In addition, reaction evaluation feedback, pronunciation evaluation feedback, mouth evaluation feedback, and compensation feedback may utilize only necessary feedback depending on each test and learning step. For example, if the reaction and pronunciation are both correct in the spontaneous test, only reaction evaluation feedback, pronunciation evaluation feedback and compensation feedback can be presented.

In addition, the spontaneous speech response evaluation and the spontaneous pronunciation evaluation can be performed through the speech evaluation module shown in FIG. For reference, the reaction evaluation and the pronunciation evaluation in the speech evaluation module can be similarly applied to the reaction evaluation and pronunciation evaluation described later, the reaction evaluation to the speech conversion to be described later, and the pronunciation evaluation.

The speech evaluation module will be described with reference to FIG.

The speech evaluation module consists of a reaction evaluation module, a pronunciation evaluation module, and an oral evaluation module.

The response evaluation analyzes the verbal and nonverbal responses to the response to the target data.

Language response assessment evaluates correct responses and incorrect responses. In the correct response, the learner evaluates the pronunciation of the learner's pronunciation. For example, if there is a negative reaction corresponding to 'candy' photo stimulus / candy /, / sugar /, / sugar / etc. In response to wrong response, the learner responds to the stimulus that does not express the target stimulus and the stimulus does not have any relation to the target stimulus.

This is because the learner 's unresponsive or irrelevant response may be the result of the learner lacking the appropriate vocabulary for the target data or reading the text. Therefore, the evaluation of the pronunciation of wrong answers does not proceed. In the nonverbal reaction evaluation, the reaction rate, the screen switching speed, and the gaze are analyzed. Response rate analysis stores and analyzes the learner 's response rate.

The speed of screen switching is measured by the learner's individual learning speed by measuring the time until the next screen is switched through the touch as if the user is turning over the bookcase. Eye line analysis uses eye tracking technology to analyze the patterns and attention duration of learners' attention to learning tasks. Non-verbal response assessment analysis data is used to analyze learners' individual learning styles and to find out how to construct and present the most effective stimuli and learning materials for learners and to modify learning materials.

The pronunciation evaluation module analyzes characteristics of the difference between the language model and the pronunciation of the learner using the language therapy evaluation criteria. The evaluation was conducted in order to evaluate pronunciation accuracy according to the language level (phonemes, syllables, words, sentences), and to compare the pronouns, omissions, distortions, additions, repetitions, extensions, Includes analysis of the sound intensity for For example, there is an error characteristic that 'candy' is replaced with / tall / as / s / / t / / / There is also an error characteristic such as / repetition), / sachan / (speech added), / - tang / (speech omitted)

In addition, it is possible to pronounce 'ㅅ' of 'candy' as / a / and 'a' as / ah / (phoneme level) / Error of substitution, / mistake / addition error, '' I bought a candy at the mart yesterday '' At the level of the sentence / I bought the yesterday from the mart-tang ' . And, when a person speaks, the sound intensity varies depending on the spoken length level, depending on the position of each speech. This affects both the correct pronunciation of the speech and the pronunciation. Therefore, by analyzing the language model and learners 'pronunciation, it can be used as a factor to judge learner' s pronunciation error characteristics and linguistic listening characteristics. In addition, the pronunciation error characteristics of the learners vary according to the speech stimulus level, such as spontaneous speech test without speech stimulation, mimetic speech test with accurate speech stimulation of model language, and speech speech stimulation with transformed model language .

This provides information on the characteristics of the learner's pronunciation errors and the characteristics of the learner's stimulation level suitable for the learner's language learning.

The oral assessment module compares and analyzes the language model and the characteristics of the learner's mouth, tongue movement, and respiration when speaking. Depending on the speech, mouth shape and tongue movements are different, breathing, inhalation and exhalation are different, and respiratory control is also important to make the horse fluent. It is necessary to construct a learner 's customized learning material that analyzes the results and analyzes the results because the learner has different pronunciations and different causes of inaccurate pronunciation.

 The learner 's mouth and tongue movements are captured by the camera when the learner is speaking and analyzed using only mouth shape and tongue movement data through preprocessing. Breathing is measured graphically by UWB radar. A camera, a wireless earphone, a wireless microphone, and a breathing sensor can be used to analyze the learner's pronunciation status.

On the other hand, the mimetic language test process (S300) is a simulation language test screen for testing a mimetic language that follows a speech of a language model used in a speech training content when the self- So that an empirical test is performed. In other words, as shown in FIG. 8, a mimic presentation process (S310) for displaying a phoneme, a syllable, a word, and a sentence of a mimetic word corresponding to a learning target level, (S330) of inputting a voice of a learner who speaks a mimetic word (S320), and a mimic word response evaluation process (S3400) for evaluating a response to the voice of the noise-eliminated mimetic word A pronunciation evaluation process (S340a), a pronunciation evaluation process (S350) to provide a feedback result, and a pronunciation evaluation process (S340) as a result of the mimetic word reaction evaluation process, (Step S340b), feedback of the imitation test failure is provided (S360), and the process goes to the voice conversion test (S400).

That is, the empirical test starts when a target stimulus containing the learning target pronunciation is presented. Target stimuli include learning target pronunciation, content suitable for level (phonemes, syllables, words, sentences), composed of photographs and language models, or composed of photographs, language models, and letters. When the learner 's voice input to the target stimulus is obtained, the speech is improved through the noise elimination in the preprocessing step, and each data suitable for evaluation is extracted. The response evaluation analyzes the learner 's voice response mimicking the language model' s words. If the answer is correct, the learner evaluates the pronunciation and presents feedback. If the answer is incorrect, feedback is presented and voice conversion test is performed.

On the other hand, the speech conversion test process (S400) provides a speech conversion test screen for testing the speech conversion generated by reflecting the result of the speech recognition test, . That is, as shown in FIG. 9, the speech conversion test process (S400) reflects the imitation speech test result and performs speech conversion (S410) of the learning target level (S420) (S450), and the pronunciation evaluation process is performed on the learner's voice that has been removed (S450). If it is determined that the learner's voice is noisy (S450a) (S460), and provides a feedback result. If the evaluation result of the response to the learner's voice following the speech conversion is less than the evaluation criteria of the speech conversion (S450b) And provides feedback (S471).

The speech conversion test process (S400) may include a speech conversion process, a speech conversion preprocessing process, a speech conversion evaluation feedback providing process, and a speech conversion reaction evaluation feedback providing process until the learner's speech response meets a preset reference .

In detail, the speech conversion test amplifies and corrects the speech of the language model through voice conversion, reflecting the result of the analysis of the pronunciation test. Enter the learner's negative response by presenting the modified target stimulus including the modified language model.

In the speech conversion preprocessing process, the input learner 's speech is improved through noise elimination, etc., and each data suitable for evaluation is extracted. The response evaluation analyzes the learner 's voice response following the words of the modified language model. If the answer is correct, the learner evaluates the pronunciation and presents feedback, and then proceeds to the empirical test. In the case of an incorrect answer, the feedback is presented, and the analysis result is reflected in the voice conversion step of the language model.

The learner will automatically repeat the speech conversion process until the voice response is correct, so that the learner will analyze the speech characteristics of the speech and incorporate it into the treatment plan. For example, if you speak in the learner 's' candy' speech, the speech of the language model is amplified and the 'loud' sound is amplified. Candy ', the learner' s utterance error characteristic corresponds to the substitution type of the articulation error, and the amplification level of the learner can be determined as a solution for solving the learner 's error. Also, if the voice of the language model is amplified in two steps and the speed is presented at the slow speed of the first step, if the learner's utterance is correct, the learner's speech learning data is determined as the voice amplification level 2 and the speed level 1 level do.

Meanwhile, the speech learning process (S500) provides the learner-customized learning material reflecting the test results of the speech test, the mimetic word test, and the voice conversion test so that the learner can learn the speech. As shown in FIG. 11, the speech learning process (S500) includes a process (S520) of providing learner-customized learning data after selecting a speech to be learned (S510), a process of selecting a learner- (S530) of inputting learner information including a tongue shape and a breath (S530), a preprocessing process of removing noise from the learner information (S540), and a learning evaluation of whether or not the learner information (S560), the learning result is derived (S560), and the learner customized learning material is modified (S570) to repeat the learner information input and the learning evaluation (Step S580), a speech learning game process (step S590) in which the speech learning game is progressed and learned, the feedback is provided (step S580) Have.

In other words, in the speech learning process, the learner selects the speech to be learned. When a speech is selected, learning material containing a customized language model reflecting the learner's pronunciation analysis result is presented. The learning material may include a language model, mouth shape, oral structure data, and graphs, as shown in FIG. 12, and may be included as a learning aid. Learning can be done with only the voice of the customized language model according to the learner, and graphics can be presented together with learning. When the learner speaks after the lecture is performed, the learner's voice is recorded, the facial expression is photographed with the camera, the breath is measured wirelessly, and the learner's information is input. The learner's information input includes the learner's voice, mouth shape, tongue movement, and breathing. The pronunciation evaluation is performed through the preprocessing step. Modify the learning data by amplifying the voice of the language model according to the analysis result, converting the learning contents to be emphasized in the graphic point level and the mouth shape and tongue movement. Earphones and microphones can be used together when learning.

Speaking If you succeed at a certain level of accuracy in the speech evaluation stage, you will be presented with a speech learning game for repetition and fluency practice. The speech learning game provision (S590) is a learner-customized speech learning game in which the learner is composed of the speech already learned or the speech being currently being learned and the win or loss of the game is determined by the learner's speech response. It includes a variety of language game types that respond to learners' speech so that they can practice the speech fun. For example, as shown in FIG. 13, when a learning target word is candy, a learner quickly collects a lot of candy when a 'candy' picture stimulus appears among objects falling into a trash bin. At this time, the 3D character introduces the game method and provides feedback to improve learning achievement and mastery motivation. In addition, a language model or a voice-translated language model may be presented to improve pronunciation accuracy. The 3D character presents the feedback while the learner plays the game and plays the role of the teacher and the competitor of the learner to the learner. 3D characters presented in three dimensions provide learner with feedback such as praise and encouragement, and also enhance the fun and practice effect of game learning while playing a role of competitor stimulating learning motivation. The character response corresponding to the learner 's response result stimulates the learner to repeat the learner' s learning by providing the learning effect while interacting.

The embodiments of the present invention described above are selected and presented in order to facilitate the understanding of those skilled in the art from a variety of possible examples. The technical idea of the present invention is not necessarily limited to or limited to these embodiments Various changes, modifications, and other equivalent embodiments are possible without departing from the spirit of the present invention.

S100: Speaking Training Content Selection Process
S200: Self-assessment test
S300: Practice test process
S400: Transformed speech test process
S500: Speaking process

Claims (9)

A method of correcting a speech in which a speech learning content app installed in a learner terminal performs a speech training for a learner,
A speech training content selection process in which a speech training content to be used for speech training is selected from a learner;
A spontaneous speech test process for providing a spontaneous speech test screen of the selected speech training contents to allow a spontaneous speech test to be performed;
And an empirical test screen for performing a test on a mimetic word following the speech model of the language model used in the speech training content is provided so as to perform the mimetic word test if the self- An empirical test process;
And a speech conversion test screen for testing imitation of the speech-converted language model generated by reflecting the result of the imitation test is provided so as to perform the speech conversion test when the imitation test result is less than a preset imitation evaluation criterion A speech conversion test process; And
And a speech learning process for providing a learner customized learning material reflecting the test results of the spontaneous speech test, the mimetic word test, and the voice conversion test so that the learner's speech learning is performed,
The speech training content selection process includes:
We choose the level of the learning target to be used in the speech training,
In the spontaneous speech test,
A voluntary display process of displaying a photograph or character capable of generating phonemes, syllables, words, and sentences of a spontaneous word corresponding to the learning target level;
A spontaneous speech preprocessing process for eliminating noise by inputting speech of a learner who speaks spontaneous speech;
A spurious response evaluation process in which a response evaluation is performed on the noise of the noise canceled spontaneous speech;
A pronunciation evaluating step of providing a feedback result by proceeding pronunciation evaluation when the result of the self-verbal response evaluation meets the self-verbal evaluation criterion; And
And an empirical test moving step of providing feedback of a spontaneous test failure and proceeding to the empirical test when the spontaneous word response evaluation result is less than the spontaneous word evaluation standard,
The simulated speech test process includes:
Displaying the phonemes, syllables, words, and sentences of the mimetic word corresponding to the learning target level as photographs or characters and following the speech of the language model;
A preprocessing process of removing a noise by inputting a voice of a learner who speaks a mimetic word;
An empirical response evaluation process for evaluating the response to the voice of the noise canceled mimic word;
A pronunciation evaluation process of performing a pronunciation evaluation and providing a feedback result when the pronunciation evaluation meets the pronunciation evaluation result, And
And a voice conversion test movement step of providing feedback of a mimetic word test failure and proceeding to the voice conversion test when the mimic word evaluation criterion is not satisfied as a result of the mimic word response evaluation,
The voice conversion test process includes:
A voice conversion generating step of generating and presenting voice conversion of a learning target level by reflecting the result of the speaker test;
A speech conversion presentation process for presenting a speech of a speech-converted language model;
A speech conversion pre-processing step of inputting speech of a learner who speaks through speech conversion and removing noise;
A voice conversion pronunciation evaluation feedback providing step of performing a reaction evaluation on the learner's voice with noise removed and proceeding with pronunciation evaluation if the dictionary meets the predetermined voice conversion evaluation criteria to provide a feedback result; And
And a voice conversion reaction evaluation feedback providing step of providing a feedback of a voice conversion test failure when the voice conversion evaluation criterion is not satisfied as a result of the evaluation of the learner's voice following the voice conversion,
Wherein the voice conversion reaction evaluation feedback providing step comprises:
The learner's voice response characteristic of the voice-activated language model is analyzed. The learner's speech is greatly amplified and the syllable with no error is transformed into a small amount of speech and presented to the learner , And when the presented language model is pronounced correctly, the amplification level for voice conversion is determined as a solution for error correction.
delete delete delete delete The method according to claim 1,
The speech conversion process, the speech conversion preprocessing process, the speech conversion pronunciation evaluation feedback process, and the speech conversion reaction evaluation feedback process are repeated until the learner's speech response meets a preset reference. Way.
The method according to claim 1,
In the reaction evaluation, a language reaction evaluation and a non-verbal reaction evaluation are performed. In non-verbal reaction evaluation, a reaction rate, a screen transition, and a gaze treatment are evaluated.
Wherein the pronunciation evaluation is based on a language model based on a phoneme level, a syllable level, a word level, and a sentence level, and evaluates a learning target standard and a learning level of the learner.
The method according to claim 1,
Providing the learner customized learning material;
A step of receiving learner information including a voice, a mouth shape, a tongue shape, and breathing of a learner learning the learner customized learning material;
Removing noise from the learner information;
A step of learning evaluation of whether or not learner information whose noise has been removed meets a predetermined learning evaluation standard;
Repeating the learner information input and the learning evaluation by modifying the learner customized learning material when the learning evaluation result is less than the learning evaluation standard; And
A speech learning game progress process in which the speech learning game is progressed and learned when the learning evaluation result meets the learning evaluation standard;
/ RTI >
The method according to claim 8,
Wherein the learner is composed of already learned speech or speech being learned, and the win or loss of the game is determined by the learner's speech response.

Images