KR102414626B1 - Foreign language pronunciation training and evaluation system - Google Patents

Abstract

The present invention relates to a foreign language pronunciation training and evaluation system, comprising: a native speaker pronunciation reproducing unit for reproducing a native speaker pronunciation for a specific text; a learner pronunciation receiver for receiving the learner pronunciation for the specific text; a fluency measuring unit for measuring the fluency of vocalization by comparing the pronunciation graphs of the native speaker's pronunciation and the learner's pronunciation; an accuracy measuring unit for generating an evaluation text by converting the learner's pronunciation and measuring the accuracy of pronunciation by comparing the text with the specific text; and a pronunciation evaluation unit configured to generate an evaluation result by performing a pronunciation evaluation based on the fluency of the vocalization and the accuracy of the pronunciation.

Description

Foreign language pronunciation training and evaluation system

The present invention relates to a foreign language pronunciation training and evaluation system, and more particularly, to a foreign language pronunciation training and evaluation system capable of improving the efficiency of foreign language learning by accurately providing an evaluation result on voice pronunciation in a foreign language learning process.

As interest in foreign languages increases, the need for efficient and systematic foreign language learning methods is increasing. Recently, as practical communication skills are emphasized in language education, the importance of spoken language, especially speaking, as a means of communication is increasing, and the same trend appears in foreign language education.

In general, speaking evaluation is performed in a manual evaluation method in which a number of evaluation experts directly listen to learners' utterances and evaluate them. As a contrasting concept, an automatic evaluation method that uses a speech evaluation system to automatically evaluate a learner's utterance without an evaluator can be considered.

However, the automatic evaluation method up to now evaluates according to a uniform standard regardless of the foreign language learner's nationality or native language, and thus there may be a problem in accuracy. For example, there may be differences between the Korean pronunciation of a learner whose native language is Japanese and the Korean pronunciation of a learner whose native language is Chinese. can be dropped

Accordingly, there is a demand for the development of a technology and system that can relatively accurately evaluate a learner's utterance.

Korean Patent Registration No. 10-0733469 (2007.06.22)

An embodiment of the present invention is to provide a foreign language pronunciation training and evaluation system capable of improving the efficiency of foreign language learning by accurately providing an evaluation result on voice pronunciation in a foreign language learning process.

An embodiment of the present invention is to provide a foreign language pronunciation training and evaluation system capable of performing evaluation in terms of vocalization and pronunciation, respectively, through matching between the similarity of the speech graph and the words corresponding to the pronunciation.

An embodiment of the present invention is to provide a foreign language pronunciation training and evaluation system capable of performing pronunciation evaluation by comparing the area of each section of a speech graph and the length and position of the silent section.

In embodiments, the foreign language pronunciation training and evaluation system may include: a native speaker pronunciation reproducing unit for reproducing a native speaker pronunciation for a specific text; a learner pronunciation receiver for receiving the learner pronunciation for the specific text; a fluency measuring unit for measuring the fluency of vocalization by comparing the pronunciation graphs of the native speaker's pronunciation and the learner's pronunciation; an accuracy measuring unit for generating an evaluation text by converting the learner's pronunciation and measuring the accuracy of pronunciation by comparing the text with the specific text; and a pronunciation evaluation unit configured to generate an evaluation result by performing a pronunciation evaluation based on the fluency of the vocalization and the accuracy of the pronunciation.

The native speaker pronunciation reproducing unit provides a visualized highlighting of the specific text according to the progress of the native speaker pronunciation in the reproduction process, and when the peak value on the pronunciation graph of the native speaker exceeds a preset threshold, corresponding Words corresponding to peak values can be visually displayed.

obtaining, by the fluency measurement unit, a first speech graph related to the pronunciation of the native speaker; obtaining a second speech graph related to the learner's pronunciation; detecting and arranging an utterance timing of each of the first and second utterance graphs; and performing scaling on the second speech graph according to the gender of the learner's pronunciation based on the gender of the native speaker's pronunciation, thereby performing a preprocessing operation for comparing the speech graph.

The fluency measurement unit detects a mute section of each of the first and second vocalization graphs, identifies the position and length of the silent section based on the entire pronunciation section, and determines the first fluency based on the difference between the silent sections. performing an evaluation; For each of the first and second speech graphs, the entire pronunciation section is divided into a plurality of partial pronunciation sections, a graph area amount of each of the plurality of partial pronunciation sections is calculated, and the second speech section is formed based on the difference in the graph area amount. 2 performing a fluency assessment; and determining an evaluation score by integrating the first and second fluency evaluations to measure the fluency of the vocalization.

detecting, by the accuracy measuring unit, key words corresponding to a peak value on a corresponding speech graph in the evaluation text; detecting auxiliary words connected before and after each of the main words in the evaluation text; and determining an evaluation score according to whether each of the main words and the auxiliary words match each other based on the specific text, thereby measuring the accuracy of the pronunciation.

The system may further include a learning content recommendation unit that accumulates and manages training and evaluation histories for each difficulty, week, course, and topic of foreign language learning, and recommends learning content according to the learner's learning level.

The disclosed technology may have the following effects. However, this does not mean that a specific embodiment should include all of the following effects or only the following effects, so the scope of the disclosed technology should not be understood as being limited thereby.

The foreign language pronunciation training and evaluation system according to an embodiment of the present invention can improve the efficiency of foreign language learning by accurately providing an evaluation result on voice pronunciation in a foreign language learning process.

The foreign language pronunciation training and evaluation system according to an embodiment of the present invention may perform evaluation in terms of vocalization and pronunciation, respectively, through matching between the similarity of the vocalization graph and the words corresponding to the pronunciation.

The foreign language pronunciation training and evaluation system according to an embodiment of the present invention may perform pronunciation evaluation by comparing the area for each section of the speech graph and the length and position of the silent section.

1 is a view for explaining a foreign language pronunciation training and evaluation system according to the present invention.
FIG. 2 is a diagram for explaining a system configuration of the training and evaluation apparatus shown in FIG. 1 .
FIG. 3 is a view for explaining a functional configuration of the training and evaluation device in FIG. 1 .
4 is a flowchart illustrating a foreign language pronunciation training and evaluation process according to the present invention.
5 and 6 are diagrams for explaining an embodiment of an interface provided by the foreign language pronunciation training and evaluation system according to the present invention.

Since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiment described in the text. That is, since the embodiment may have various changes and may have various forms, it should be understood that the scope of the present invention includes equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only such effects, it should not be understood that the scope of the present invention is limited thereby.

On the other hand, the meaning of the terms described in the present application should be understood as follows.

Terms such as “first” and “second” are for distinguishing one component from another, and the scope of rights should not be limited by these terms. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component.

When a component is referred to as being “connected to” another component, it may be directly connected to the other component, but it should be understood that other components may exist in between. On the other hand, when it is mentioned that a certain element is "directly connected" to another element, it should be understood that the other element does not exist in the middle. Meanwhile, other expressions describing the relationship between elements, that is, "between" and "between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The singular expression is to be understood as including the plural expression unless the context clearly dictates otherwise, and terms such as "comprises" or "have" refer to the embodied feature, number, step, action, component, part or these It is intended to indicate that a combination exists, and it should be understood that it does not preclude the possibility of the existence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

In each step, identification numbers (eg, a, b, c, etc.) are used for convenience of description, and identification numbers do not describe the order of each step, and each step clearly indicates a specific order in context. Unless otherwise specified, it may occur in a different order from the specified order. That is, each step may occur in the same order as specified, may be performed substantially simultaneously, or may be performed in the reverse order.

The present invention can be embodied as computer-readable codes on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices in which data readable by a computer system is stored. . Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. In addition, the computer-readable recording medium is distributed in a computer system connected to a network, so that the computer-readable code can be stored and executed in a distributed manner.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless otherwise defined. Terms defined in general used in the dictionary should be interpreted as having the meaning consistent with the context of the related art, and cannot be interpreted as having an ideal or excessively formal meaning unless explicitly defined in the present application.

1 is a view for explaining a foreign language pronunciation training and evaluation system according to the present invention.

Referring to FIG. 1 , the foreign language pronunciation training and evaluation system 100 may include a user terminal 110 , a training and evaluation device 130 , and a database 150 .

The user terminal 110 may correspond to a computing device capable of checking learning contents for foreign language learning and inputting foreign language pronunciation. The user terminal 110 may be implemented as a smartphone, a notebook computer, or a computer, but is not limited thereto, and may be implemented in various devices such as a tablet PC. The user terminal 110 may be connected to the training and evaluation apparatus 130 through a network, and a plurality of user terminals 110 may be simultaneously connected to the training and evaluation apparatus 130 .

In addition, the user terminal 110 may install and execute a dedicated program or application for foreign language learning. The user terminal 110 may be implemented by including a microphone module capable of recording foreign language pronunciation and a speaker module capable of reproducing foreign language pronunciation, and further includes a camera module for video conversation between the user and a native speaker if necessary. can be

The training and evaluation device 130 may be implemented as a computer or a server corresponding to a program that provides various functions for supporting foreign language learning by evaluating a user's foreign language pronunciation in a foreign language learning process and providing feedback thereon. The training and evaluation device 130 may be connected to the user terminal 110 through a wired network or a wireless network such as Bluetooth or WiFi, and may communicate with the user terminal 110 through the network. In addition, the training and evaluation device 130 may operate in connection with an external system (not shown in FIG. 1 ), for example, may be connected to an external payment system, an online system, an authentication system, and the like.

The database 150 may correspond to a storage device for storing various types of information required in the operation process of the training and evaluation device 130 . The database 150 may store information on various learning contents for learning a foreign language, and may store information on pronunciation of a native speaker for learning a foreign language, but is not necessarily limited thereto, and the training and evaluation device 130 may be configured to learn a foreign language. Information collected or processed in various forms can be stored in the evaluation and training process for

FIG. 2 is a view for explaining the system configuration of the training and evaluation apparatus shown in FIG. 1 .

Referring to FIG. 2 , the training and evaluation apparatus 130 may be implemented including a processor 210 , a memory 230 , a user input/output unit 250 , and a network input/output unit 270 .

The processor 210 may execute a procedure for processing each step in the process in which the training and evaluation device 130 operates, and may manage the memory 230 that is read or written throughout the process, and the memory ( 230) may schedule a synchronization time between the volatile memory and the non-volatile memory. The processor 210 may control the overall operation of the training and evaluation device 130 , and is electrically connected to the memory 230 , the user input/output unit 250 , and the network input/output unit 270 to control the flow of data therebetween. can do. The processor 210 may be implemented as a central processing unit (CPU) of the training and evaluation device 130 .

The memory 230 is implemented as a non-volatile memory, such as a solid state drive (SSD) or a hard disk drive (HDD), and may include an auxiliary storage device used to store overall data required for the training and evaluation device 130 and , and may include a main memory implemented as a volatile memory such as random access memory (RAM).

The user input/output unit 250 may include an environment for receiving a user input and an environment for outputting specific information to the user. For example, the user input/output unit 250 may include an input device including an adapter such as a touch pad, a touch screen, an on-screen keyboard, or a pointing device, and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input/output unit 250 may correspond to a computing device accessed through a remote connection, and in such a case, the training and evaluation device 130 may be performed as an independent server.

The network input/output unit 270 includes an environment for connecting with an external device or system through a network, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), and a VAN (Wide Area Network) (VAN). It may include an adapter for communication such as Value Added Network).

FIG. 3 is a diagram for explaining a functional configuration of the training and evaluation device in FIG. 1 .

Referring to FIG. 3 , the training and evaluation device 130 includes a native speaker pronunciation reproducing unit 310 , a learner pronunciation receiving unit 320 , a fluency measurement unit 330 , an accuracy measurement unit 340 , and a pronunciation evaluation performing unit 350 . , a learning content recommendation unit 360 and a control unit 370 may be included.

The native speaker pronunciation reproducing unit 310 may reproduce the native speaker pronunciation of a specific text. The native speaker pronunciation may correspond to a recording of the voice contents of a native speaker regarding various texts included in the learning content, may be generated separately by topic, sentence, and word, and may be stored and managed in the database 150 have. The native speaker pronunciation reproducing unit 310 may read data stored in the database 150 at a user's request or in the course of learning to reproduce the native speaker's pronunciation, and may provide it through the user terminal 110 if necessary. In this case, the native speaker pronunciation reproducing unit 310 may operate in conjunction with a dedicated program or application of the user terminal 110 .

In an embodiment, the native speaker pronunciation reproducing unit 310 may provide a visualized highlighting of a specific text according to the progress of the native speaker pronunciation during the reproduction process. That is, the native speaker pronunciation reproducing unit 310 may display the corresponding text content through a related interface while playing the native speaker pronunciation, and by highlighting the text in synchronization with the progress of the native speaker pronunciation, the user can make the part corresponding to the native speaker pronunciation. You can instantly recognize where it is on this text.

In an embodiment, the native speaker pronunciation reproducing unit 310 may visually display a word corresponding to the peak value when the peak value exceeds a preset threshold value on the pronunciation graph of the native speaker pronunciation. Here, the speech graph may correspond to a waveform graph in which an audio signal related to the pronunciation of a native speaker is converted into a waveform. For example, the vocalization graph may correspond to a graph in which the energy level related to the pronunciation of a native speaker is converted into a waveform. The peak value may correspond to the maximum value of the peak portion on the curve of the vocalization graph.

That is, the native speaker pronunciation reproducing unit 310 can detect a peak value on the vocalization graph, compare the peak value with a preset threshold value, and when it is greater than the threshold value, determine and visualize the word corresponding to the peak value. can be displayed Through this, the learner can easily identify high-stressed words in the process of listening to the native speaker's pronunciation, and can effectively train an intonation close to the native speaker's pronunciation.

The learner pronunciation receiver 320 may receive a learner pronunciation for a specific text. After the native speaker's pronunciation is reproduced, the learner may listen to the pronunciation and train to pronounce the same text, and the learner pronunciation receiver 320 may record the learner's pronunciation as it is in the process and collect it in the form of a voice file. In addition, the learner pronunciation receiving unit 320 may receive an audio signal related to the learner's pronunciation instead of a voice file, and may receive a waveform graph corresponding to the learner's pronunciation as data about the learner's pronunciation through signal analysis. The learner pronunciation receiver 320 may store the collected data on the learner pronunciation in the database 150 .

The fluency measuring unit 330 may measure the vocal fluency by comparing the pronunciation graphs of the native speaker's pronunciation and the learner's pronunciation. The fluency measurement unit 330 may evaluate the fluency of the learner's vocalization among pronunciations through various methods based on the vocalization graph. Vocal fluency can correspond to the degree to which a learner's vocalization is close to that of a native speaker. Specifically, it corresponds to evaluating how close to a native speaker the expression of intonation or stress is in the process of pronouncing a specific word or specific text. can do. The fluency measurement unit 330 may measure the similarity of the speech graph based on the learner's pronunciation to evaluate the fluency of the speech, and may calculate and provide an evaluation score.

In an embodiment, the fluency measurement unit 330 may perform a pre-processing operation for comparing the speech graph, and the specific method may be as follows. The fluency measurement unit 330 obtains a first speech graph related to the pronunciation of a native speaker, obtains a second speech graph related to the learner's pronunciation, detects and arranges the speech timing of each of the first and second speech graphs, and arranges the speech by a native speaker. A preprocessing operation for comparing the speech graph may be performed by scaling the second speech graph according to the gender of the learner's pronunciation based on the gender of the pronunciation.

More specifically, the fluency measurement unit 330 may determine the beginning of the first word, that is, a time point, through speech-to-text (STT) speech recognition on each speech graph, and the speech points coincide with each other. You can arrange the vocalization graphs so that The utterance time may correspond to a point at which a change in the magnitude of the voice signal first occurs. In addition, the fluency measurement unit 330 may perform normalizing on the vocalization graph in order to correct the difference in tone between male and female, and through normalizing, it is evaluated by reducing the deviation of simple high and low values that occur in the difference in voice volume. It can improve the accuracy of the results.

In an embodiment, the fluency measurement unit 330 detects a mute section of each of the first and second speech graphs and identifies the position and length of the silent section based on the entire pronunciation section to determine the difference between the silent sections Based on the first fluency evaluation, the entire pronunciation section is divided into a plurality of partial pronunciation sections for each of the first and second speech graphs, and the graph area amount of each of the plurality of partial pronunciation sections is calculated. The second fluency evaluation may be performed based on the difference in the amount of graph area, and the evaluation score may be finally determined by integrating the first and second fluency evaluations.

More specifically, the fluency measurement unit 330 may perform each of the vocal fluency evaluations from different viewpoints, and may finally integrate the corresponding results to determine a final fluency score. The first fluency evaluation is a result of comparing positions and lengths for each silent section on the vocalization graph, and may correspond to an evaluation result regarding the vocal fluency. In this case, the mute section may occur in the process of the learner reading a space between sentences and words or reading a pause in the process of pronunciation. That is, the silent section is a section in which a voice signal of a specific intensity or less continues on the speech graph, and may correspond to a section in which the learner's speech does not exist. As a result, the fluency measurement unit 330 can effectively measure the expressive power or sense of rhythm of the learner by comparing the silent sections.

In addition, the second fluency evaluation may be performed as a result of measuring the amount of area between the vocalization graphs and comparing them. It may correspond to the evaluation result. On the other hand, unlike the first fluency evaluation, the second fluency evaluation differs in the method in that the related evaluation can be performed without separately measuring the maximum value (high), minimum value (low), or mute section on the vocalization graph. may exist. In addition, the second fluency evaluation may be performed based on the degree of similarity to the total area for the entire pronunciation section of the vocalization graph, and is performed by dividing the entire pronunciation section into partial pronunciation sections and comparing the area values for each section it might be

Also, the fluency measurement unit 330 may integrate the first and second fluency evaluations to finally determine an evaluation score for vocal fluency, and may apply various summing algorithms to integrate the first and second fluency evaluations. have. For example, the fluency measurement unit 330 may calculate a final evaluation score through a weighted sum by applying a weight to each of the first and second fluency evaluations. In this case, the weight for each evaluation result may be set in advance by the training and evaluation device 130 , and may be set differentially according to the content of the text used for learning, the difficulty of pronunciation, the composition of words, and the like.

The accuracy measuring unit 340 may convert the pronunciation of the learner to generate an evaluation text, and measure the accuracy of pronunciation by comparing the text with a specific text. The training and evaluation apparatus 130 may perform evaluation from various viewpoints regarding foreign language learning, and may independently evaluate not only vocal fluency but also pronunciation accuracy. The accuracy measurement unit 340 may be implemented and operated independently of the fluency measurement unit 330 , and the accuracy measurement may be performed in parallel with the fluency measurement. Accuracy measurement can be performed based on the text converted through the STT algorithm based on the speech signal pronounced by the learner, and a quantified result can be generated according to the degree of agreement by comparing it with the text about the pronunciation of a native speaker.

In an embodiment, the accuracy measurement unit 340 detects key words corresponding to a peak value on a corresponding speech graph in the evaluation text, detects auxiliary words connected before and after each of the main words in the evaluation text, and the specific text Accuracy of pronunciation may be evaluated through a process of determining an evaluation score according to whether each of the main words and the auxiliary words coincides with each other.

More specifically, the accuracy measuring unit 340 may detect key words based on a peak value on the vocalization graph. That is, the main words may correspond to the word with the greatest stress in the process of pronouncing a specific sentence, and may correspond to the word determining the intonation. The accuracy measurement unit 340 may perform accuracy evaluation based on a comparison result between key words.

Also, the accuracy measuring unit 340 may determine words connected before and after the main words as auxiliary words, and reflect the comparison result between the auxiliary words in the accuracy evaluation. That is, the accuracy measurement unit 340 may calculate a final evaluation score regarding pronunciation accuracy as a result of integrating the primary comparison result between the main words and the secondary comparison result between the auxiliary words. For example, the accuracy measuring unit 340 may calculate the first accuracy through the ratio of the number of word matches to the total number of main words, and the second accuracy through the ratio of the number of word matches to the total number of auxiliary words Accuracy may be calculated, and a final evaluation score regarding pronunciation accuracy may be determined through a weighted sum between the first and second accuracies.

The pronunciation evaluation unit 350 may generate an evaluation result by performing pronunciation evaluation based on the vocal fluency and pronunciation accuracy. That is, the pronunciation evaluation performing unit 350 may separate and independently measure the pronunciation of the learner in terms of vocalization and pronunciation, and may generate and provide a final evaluation result as a result of integrating them. In addition, the pronunciation evaluation performing unit 350 may provide visualization of real-time statistical indicators regarding the learner's learning and evaluation contents through various interfaces. Through this, learners can effectively grasp their foreign language learning content and inspire their motivation to learn.

The learning content recommendation unit 360 may accumulate and manage training and evaluation histories for each difficulty, week, course, and topic of foreign language learning, and recommend learning content according to the learner's learning level. The learning content recommendation unit 360 may manage the learning and evaluation contents received from the pronunciation evaluation execution unit 350 for each user and store it in the database 150 .

In addition, the learning content recommendation unit 360 may provide a history of foreign language learning for each user, and may provide visualization of detailed statistical indicators and the like through various interfaces. Through the visualized information, the user can effectively recognize his/her learning level and situation, and the need for additional learning.

Also, the learning content recommendation unit 360 may recommend learning content necessary for the user based on the user's evaluation content. To this end, the learning content recommendation unit 360 may refer to learning contents of other users, and may utilize a recommendation algorithm such as CF (Collaborative Filtering) and CBF (Contents Based Filtering).

Also, the learning content recommendation unit 360 may recommend learning content suitable for the learner's level. In this case, the learning content may be classified by learning type. For example, shadowing may correspond to content in which learning is performed by pronouncing according to a native speaker's pronunciation, and read aloud is learning by reading a learning text aloud without a native speaker's pronunciation. It may correspond to the content that The learning content recommendation unit 360 may recommend appropriate learning content by adjusting the learning difficulty according to the learner's level. For the learning content for each difficulty level, the number of learning times or the difficulty level of the learning text may be set in various ways along with additional functions such as hiding English.

In addition, the learning content recommendation unit 360 may provide learning content through which the learner can intensively learn the pronunciation based on the record of the pronunciation that the learner frequently makes wrong. For example, if a learner frequently misses the pronunciation of 'R' and 'L' during the learning process, learning content that can train the pronunciation can be intensively provided to support the learner's learning.

The control unit 370 controls the overall operation of the training and evaluation device 130 , the native speaker pronunciation reproducing unit 310 , the learner pronunciation receiving unit 320 , the fluency measurement unit 330 , the accuracy measurement unit 340 , and the pronunciation evaluation A control flow or data flow between the performing unit 350 and the learning content recommendation unit 360 may be managed.

4 is a flowchart illustrating a foreign language pronunciation training and evaluation process according to the present invention.

Referring to FIG. 4 , the training and evaluation apparatus 130 may reproduce the native speaker pronunciation of a specific text through the native speaker pronunciation reproducing unit 310 (step S410 ). The training and evaluation apparatus 130 may receive the learner's pronunciation of a specific text through the learner's pronunciation receiving unit 320 (step S430). The training and evaluation apparatus 130 may measure the fluency of vocalization by comparing the pronunciation graphs of the native speaker's pronunciation and the learner's pronunciation through the fluency measurement unit 330 (step S450).

In addition, the training and evaluation apparatus 130 may generate an evaluation text by converting the learner's pronunciation through the accuracy measuring unit 340 , and may measure the pronunciation accuracy by comparing the text with a specific text (step S470 ). The training and evaluation apparatus 130 may generate an evaluation result by performing pronunciation evaluation based on the vocal fluency and pronunciation accuracy through the pronunciation evaluation performing unit 350 (step S490).

In one embodiment, the training and evaluation device 130 accumulates and manages training and evaluation histories for each week, course, and topic of foreign language learning, and the learning content recommendation unit 360 recommends learning content for training and evaluation improvement. It may be implemented by further including.

5 and 6 are diagrams for explaining an embodiment of an interface provided by the foreign language pronunciation training and evaluation system according to the present invention.

5 and 6 , the training and evaluation apparatus 130 may provide various functions that can support training and evaluation of a learner's pronunciation in a foreign language learning process. The training and evaluation device 130 may sequentially provide learning topics designed for each week and each course according to a learning curriculum for systematic foreign language learning.

In the case of the interface shown in FIG. 5 , a task name 510 for each parking may be displayed in the upper center of the screen, and a name 520 for each course is displayed in the upper right portion of the screen to provide information about the content currently being studied. can do. In addition, at the upper part of the screen, the currently learned learning text 530 may be expressed in Korean and a foreign language, and may be displayed vertically and vertically. In this case, the training and evaluation apparatus 130 may provide external content as learning content in conjunction with an external system.

In addition, the training and evaluation apparatus 130 may provide a highlighting function for the training text 530 displayed on the screen, so that the training text 530 is highlighted in synchronization with the pronunciation of a native speaker currently being reproduced. Through this, the learner can intuitively understand which word part of the currently displayed learning text 530 is being pronounced by the native speaker.

In addition, the training and evaluation apparatus 130 may visualize and display the vocalization graph 550 for the pronunciation of a native speaker in the center of the screen. The vocalization graph 550 may be expressed in the form of a curved graph according to the strength (or the amount of energy) of the pronunciation, and the learner can intuitively understand the strength and intonation of each section of the pronunciation through sight while listening to the pronunciation of a native speaker. can The training and evaluation apparatus 130 may display the main word 540 corresponding to the peak value on the pronunciation graph 550 related to the pronunciation of the native speaker in parallel on the upper portion of the pronunciation graph 550 . Through this, the learner can intuitively understand the words that need attention to pronunciation in the learning text 530 .

In addition, the training and evaluation apparatus 130 may record the learner's foreign language pronunciation, receive the learner's foreign language pronunciation in real time, and display a speech graph 550 related thereto in the center of the screen. Through this, the learner can intuitively understand the difference in strength, intonation, and stress between the pronunciation of a native speaker and his or her actual pronunciation, and can check the part where learning should be concentrated through the section with a large difference.

At this time, the training and evaluation device 130 may detect the utterance timing of the learner's foreign language pronunciation, and align the learner's utterance graph to match the utterance timing of the native speaker's pronunciation based on the utterance timing, and then display it on the screen can do. That is, the learner can start his or her own pronunciation after hearing the native speaker's pronunciation, but it is not easy to accurately match the pronunciation start time. Therefore, pronunciation may not be started at a normal time point, and when the pronunciation graph is displayed as it is, the pronunciation graph of the native speaker may not be in sync with the pronunciation graph. In order to prevent this, the training and evaluation device 130 can automatically remove the silent section from the first starting section among the learner’s pronunciation, and automatically arrange and display the display point of the speech graph based on the first speech point of the learner’s pronunciation. have.

In addition, the training and evaluation device 130 may evaluate the pronunciation of the native speaker and the pronunciation of the learner in two aspects, and visually display the evaluation result in the center of the lower part of the screen. In the case of FIGS. 5 and 6 , the evaluation result may be displayed in the form of a speech bubble, and the entire evaluation result 570 for the learning text 530 may be displayed in the form of PASS or FAIL. In addition, as a more specific numerical value, the degree of correspondence 620 between the pronunciation of the native speaker and the pronunciation of the learner may be displayed.

In addition, the training and evaluation device 130 may collect the position and length of the mute section 630 on the vocalization graph 550 for pronunciation evaluation, and the silent section 630 for foreign language pronunciation and learner pronunciation. can be compared with each other to perform a pronunciation evaluation on the fluency of vocalization. On the other hand, the training and evaluation apparatus 130 may detect each of the speech time points 610 on the speech graph 550 for comparison between the speech graphs, and align the speech graphs 550 based on the speech time point 610 . can

In addition, the training and evaluation apparatus 130 may evaluate the accuracy of pronunciation by using the speech recognition STT. For example, if the learner pronounces the learning text 530 while babbling, the pronunciation graph 550 may be similar to each other, but the pronunciation accuracy may be relatively different. points may appear. If the pronunciation accuracy is less than 20%, deduction points may be applied, and if the pronunciation accuracy is 100%, additional points may be applied.

In addition, the training evaluation device 130 may provide a control bar 560 for listening to the pronunciation of a native speaker through an interface for learning a foreign language. In FIG. 5 , the control bar 560 for the learner may include a re-recording function, a volume control function, a play and stop function, a double speed function, a font size control function, and the like.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be done.

100: Foreign language pronunciation training and evaluation system
510: Assignment name by week 520: Name of each course
530: learning text 540: key words
550: vocalization graph 560: control bar
570: Overall evaluation result
610: ignition timing 620: coincidence
630: silent section

Claims (6)

a native speaker pronunciation reproducing unit for reproducing a native speaker pronunciation for a specific text;
a learner pronunciation receiver for receiving the learner pronunciation for the specific text;
a fluency measuring unit for measuring the fluency of vocalization by comparing the pronunciation graphs of the native speaker's pronunciation and the learner's pronunciation;
an accuracy measuring unit for generating an evaluation text by converting the learner's pronunciation and measuring the accuracy of pronunciation by comparing the text with the specific text; and
a pronunciation evaluation performing unit configured to generate an evaluation result by performing a pronunciation evaluation based on the fluency of the vocalization and the accuracy of the pronunciation,
obtaining, by the fluency measurement unit, a first speech graph related to the pronunciation of the native speaker; obtaining a second speech graph related to the learner's pronunciation; detecting and arranging an utterance timing of each of the first and second utterance graphs; and performing a preprocessing operation for comparing the speech graph through the step of scaling the second speech graph according to the gender of the learner's pronunciation based on the gender of the native speaker's pronunciation,
Detecting a mute section of each of the first and second speech graphs, identifying the position and length of the silent section based on the entire pronunciation section, and performing a first fluency evaluation based on the difference between the silent sections step; For each of the first and second speech graphs, the entire pronunciation section is divided into a plurality of partial pronunciation sections, and an amount of graph area of each of the plurality of partial pronunciation sections is calculated, and based on the difference in the graph area 2 performing a fluency assessment; and determining an evaluation score by integrating the first and second fluency evaluations to measure the fluency of the spoken language.
According to claim 1, wherein the native speaker pronunciation reproduction unit
Provide a visualized highlighting for the specific text according to the progress of the native speaker pronunciation in the playback process,
Foreign language pronunciation training and evaluation system, characterized in that when a peak value exceeds a preset threshold on the pronunciation graph of the native speaker, a word corresponding to the peak value is visually displayed.
delete delete According to claim 1, wherein the accuracy measuring unit
detecting key words corresponding to peak values on a corresponding speech graph in the evaluation text;
detecting auxiliary words connected before and after each of the main words in the evaluation text; and
Foreign language pronunciation training and evaluation system, characterized in that the measurement operation regarding the accuracy of the pronunciation is performed through the step of determining an evaluation score according to whether each of the main words and the auxiliary words coincide with the specific text .
The method of claim 1,
Foreign language pronunciation training and evaluation system, characterized in that it further comprises a learning content recommendation unit that accumulates and manages training and evaluation histories by difficulty, week, course, and topic of foreign language learning and recommends learning content according to the learner's learning level .

Images