KR102321141B1 - Apparatus and method for user interface for pronunciation assessment - Google Patents

Abstract

An apparatus and method for providing a user interface for pronunciation evaluation are provided according to an embodiment of the present invention. According to an embodiment of the present invention, there is provided an apparatus for providing a user interface for pronunciation evaluation, comprising: a communication unit configured to transmit and receive data; a display unit configured to display data; and a control unit configured to connect with the communication unit and the display unit, wherein the control unit obtains voice data of a user for a specific phoneme or a specific word, and uses the acquired voice data to evaluate the user's pronunciation and transmit to the providing server, receive pronunciation evaluation result data obtained by evaluating the user's pronunciation from the service providing server, and display an interface screen representing the received pronunciation evaluation result data through the display unit.

Description

Apparatus and method for providing a user interface for pronunciation evaluation

The present invention relates to an apparatus and method for providing a user interface for pronunciation evaluation.

According to the trend of industry specialization and internationalization, the importance of foreign languages is increasing, and accordingly, various services for foreign language learning are provided.

In general, foreign language learning is conducted under the guidance of a native speaker, but there are problems in that such learning consumes a lot of money and there are restrictions on place and time.

Accordingly, there is a need for a foreign language learning method capable of learning a foreign language at a low cost anytime, anywhere without restriction of place and time. To meet these needs, various language programs have been developed and provided.

However, such a language study program has a problem in that it is weak in analyzing which pronunciation among the user's foreign language pronunciations is weak and requires training or providing feedback on this.

Therefore, there is a need for a pronunciation evaluation method for analyzing a user's foreign language pronunciation without restrictions on place and time to provide feedback on which pronunciation is weak and needs training.

SUMMARY OF THE INVENTION An object of the present invention is to provide an apparatus and method for providing a user interface for pronunciation evaluation.

Specifically, an object of the present invention is to provide an apparatus and method for providing a user interface for evaluating a user's foreign language pronunciation for the user's foreign language learning without restriction of place and time.

Another object to be solved by the present invention is to provide an apparatus and method for providing a pronunciation evaluation user interface for analyzing a user's foreign language pronunciation and providing feedback on a weak part.

The problems of the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the problems described above, an apparatus and method for providing a user interface for pronunciation evaluation according to an embodiment of the present invention are provided. According to an embodiment of the present invention, there is provided an apparatus for providing a user interface for pronunciation evaluation, comprising: a communication unit configured to transmit and receive data; a display unit configured to display data; and a control unit configured to connect with the communication unit and the display unit, wherein the control unit obtains voice data of a user for a specific phoneme or a specific word, and uses the acquired voice data to evaluate the user's pronunciation configured to transmit to a providing server, receive pronunciation evaluation result data in which the user's pronunciation is evaluated for each pronunciation characteristic from the service providing server, and display an interface screen indicating the received pronunciation evaluation result data through the display unit and the pronunciation evaluation result data includes at least an evaluation score obtained by scoring the evaluation result of the user's pronunciation, pronunciation characteristics extracted from the user's voice data, and feedback data for guiding the user's pronunciation to the pronunciation of a native speaker. includes one

A method of providing a user interface for pronunciation evaluation performed by a control unit of an apparatus for providing a user interface for pronunciation evaluation according to an embodiment of the present invention includes: acquiring voice data of a user for a specific phoneme or a specific word; transmitting the acquired voice data to a service providing server for evaluating the user's pronunciation; receiving pronunciation evaluation result data obtained by evaluating the pronunciation of the user for each pronunciation characteristic from the service providing server; and displaying an interface screen indicating the received pronunciation evaluation result data, wherein the pronunciation evaluation result data includes an evaluation score obtained by scoring an evaluation result of the user's pronunciation, and a pronunciation extracted from the user's voice data. and at least one of a characteristic and feedback data for guiding the user's pronunciation to that of a native speaker.

The details of other embodiments are included in the detailed description and drawings.

The present invention may provide an apparatus and method for providing a user interface for evaluating a user's foreign language pronunciation without restrictions on place and time.

In addition, the present invention can analyze the user's foreign language pronunciation and provide feedback on the weak part of each pronunciation characteristic so that the user can perform training on the weak pronunciation.

In addition, the present invention guides the user to correct the user's foreign language pronunciation to the native speaker's pronunciation, so that the user's foreign language pronunciation can be improved to the level of a native speaker.

In addition, the present invention provides the pronunciation accuracy analysis result for each phoneme, so that the user can learn more strongly about the pronunciation of a foreign language.

The effect according to the present invention is not limited by the contents exemplified above, and more various effects are included in the present specification.

1 is a schematic diagram illustrating a system for providing a user interface for pronunciation evaluation according to an embodiment of the present invention.
2 is a schematic diagram of a user device according to an embodiment of the present invention;
3 is a schematic diagram of a service providing server according to an embodiment of the present invention.
4 is a schematic flowchart illustrating a method of providing a user interface for pronunciation evaluation in a user device according to an embodiment of the present invention.
5 is a schematic flowchart for explaining a method for pronunciation evaluation between a user device and a service providing server according to an embodiment of the present invention.
6A, 6B, 6C, 6D, 6E, 6F, and 6G are exemplary views of various interface screens related to pronunciation evaluation according to an embodiment of the present invention.
7 is an exemplary diagram of a mobile web screen for evaluating a user's pronunciation according to an embodiment of the present invention.

Advantages and features of the present invention and methods of achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only these embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims. In connection with the description of the drawings, like reference numerals may be used for like components.

In this document, expressions such as "has," "may have," "includes," or "may include" refer to the presence of a corresponding characteristic (eg, a numerical value, function, operation, or component such as a part). and does not exclude the presence of additional features.

In this document, expressions such as "A or B," "at least one of A and/and B," or "one or more of A or/and B" may include all possible combinations of the items listed together. . For example, "A or B," "at least one of A and B," or "at least one of A or B" means (1) includes at least one A, (2) includes at least one B; Or (3) it may refer to all cases including both at least one A and at least one B.

As used herein, expressions such as "first," "second," "first," or "second," may modify various elements, regardless of order and/or importance, and refer to one element. It is used only to distinguish it from other components, and does not limit the components. For example, the first user equipment and the second user equipment may represent different user equipment regardless of order or importance. For example, without departing from the scope of rights described in this document, the first component may be named as the second component, and similarly, the second component may also be renamed as the first component.

A component (eg, a first component) is "coupled with/to (operatively or communicatively)" to another component (eg, a second component) When referring to "connected to", it will be understood that the certain element may be directly connected to the other element or may be connected through another element (eg, a third element). On the other hand, when it is said that a component (eg, a first component) is "directly connected" or "directly connected" to another component (eg, a second component), the component and the It may be understood that other components (eg, a third component) do not exist between other components.

As used herein, the expression "configured to (or configured to)" depends on the context, for example, "suitable for," "having the capacity to ," "designed to," "adapted to," "made to," or "capable of." The term “configured (or configured to)” may not necessarily mean only “specifically designed to” in hardware. Instead, in some circumstances, the expression “a device configured to” may mean that the device is “capable of” with other devices or parts. For example, the phrase “a processor configured (or configured to perform) A, B, and C” refers to a dedicated processor (eg, an embedded processor) for performing the operations, or by executing one or more software programs stored in a memory device. , may mean a generic-purpose processor (eg, a CPU or an application processor) capable of performing corresponding operations.

Terms used in this document are only used to describe specific embodiments, and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly dictates otherwise. Terms used herein, including technical or scientific terms, may have the same meanings as commonly understood by one of ordinary skill in the art described in this document. Among the terms used in this document, terms defined in a general dictionary may be interpreted with the same or similar meaning as the meaning in the context of the related art, and unless explicitly defined in this document, ideal or excessively formal meanings is not interpreted as In some cases, even terms defined in this document cannot be construed to exclude embodiments of this document.

Each feature of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and technically various interlocking and driving are possible, as will be fully understood by those skilled in the art, and each embodiment may be independently implemented with respect to each other, It may be possible to implement together in a related relationship.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a schematic diagram illustrating a system for providing a user interface for pronunciation evaluation according to an embodiment of the present invention.

Referring to FIG. 1 , a system 100 for providing a user interface for pronunciation evaluation is a system that analyzes the user's voice data to evaluate the user's pronunciation, and provides a user interface for pronunciation evaluation based on the evaluation result. It may include a user device 110 that provides the user's voice data to request pronunciation evaluation and correction, and a service providing server 120 that provides a service for pronunciation evaluation.

First, the user device 110 is an electronic device that requests a pronunciation evaluation of the user's voice data and provides a user interface for displaying result data for the pronunciation evaluation, and includes a smartphone, a tablet PC (Personal Computer), and a notebook computer. and/or may include at least one of a PC and the like.

The user device 110 may acquire the user's voice data for pronunciation evaluation and pronunciation correction of the user, and transmit the acquired voice data to the service providing server 120 . For example, the user device 110 may include an input device such as a microphone for acquiring the user's voice data, and may provide a user interface for acquiring the voice data through the microphone. Such a user interface may include an area for displaying image data for learning a specific phoneme or a native speaker's pronunciation for a specific pronunciation.

When voice data is acquired through the user interface, the user device 110 may transmit the acquired voice data to the service providing server 120 .

The user device 110 may receive pronunciation evaluation result data representing an evaluation result of the user's pronunciation from the service providing server 120 , and display the received pronunciation evaluation result data through the display unit of the user device 110 . . Here, the pronunciation evaluation result data may include at least one of an evaluation score for the user's pronunciation, pronunciation characteristics for each phoneme extracted from the user's voice data, and feedback data. For example, the feedback data may be data for explaining a weak part in the user's pronunciation.

Next, the service providing server 120 analyzes the user's voice data provided from the user device 110 and provides a general-purpose computer, laptop, and/or data server that performs various calculations to provide a service for pronunciation evaluation. may include In various embodiments, the service providing server 120 may be a web server that provides a web page related to a service for pronunciation evaluation according to a request of a client or a mobile web server that provides a mobile web site. However, the present invention is not limited thereto.

Specifically, the service providing server 120 may receive voice data from the user device 110 , analyze the received voice data, and perform pronunciation evaluation for each pronunciation characteristic. For example, the service providing server 120 may sort the voice data for each phoneme, detect at least one pronunciation characteristic for each phoneme, and score the detected at least one pronunciation characteristic. For example, the at least one pronunciation characteristic detected for each phoneme may be a distinctive feature, but is not limited thereto. In this case, at least one of the phonetic characteristics is high tongue, low tongue, anterior tongue, posterior tongue, round lips, voiced, nasal, plosive, fricative, labial (or labial), interdental/alveolar, fluid, friste, R-voiced, Y It may include at least one of a semi-vowel, a W semi-vowel, a closed double, an open double, a central double, and a soft palate. As such, by using the discriminative quality as at least one pronunciation characteristic, the phonetic characteristics of segmental sounds can be clearly expressed, the interrelationship between phonemes can be clearly distinguished, many segmental sounds can be distinguished, and the phonological rules can be explicitly described. can do.

The service providing server 120 may provide pronunciation evaluation result data indicating the evaluation result of the user's pronunciation to the user device 110 .

As such, the data provided from the service providing server 120 may be provided as a web page through a web browser installed in the user device 120 , or may be provided in the form of an application or program. In various embodiments, such data may be provided in a form included in the platform in a client-server environment.

Through this, the present invention enables the user to learn by himself, increases the user's learning satisfaction, and provides a learning service for pronunciation evaluation and correction with improved efficiency.

Hereinafter, the user device 110 will be described in detail with reference to FIG. 2 .

2 is a schematic diagram of a user device according to an embodiment of the present invention;

Referring to FIG. 2 , the user device 200 includes a communication unit 210 , a display unit 220 , a storage unit 230 , and a control unit 240 . In the presented embodiment, the user device 200 may refer to the user device 110 of FIG. 1 .

The communication unit 210 connects the user device 200 to enable communication with an external device. The communication unit 210 may be connected to the service providing server 120 using wired/wireless communication to transmit/receive various data. Specifically, the communication unit 210 may transmit the user's voice data to the service providing server 120 and receive pronunciation evaluation result data from the service providing server 120 .

The display unit 220 may display various contents (eg, text, image, video, icon, banner or symbol, etc.) to the user. Specifically, the display unit 220 may request pronunciation evaluation of the user's voice data and display various interface screens for displaying result data for the pronunciation evaluation.

In various embodiments, the display unit 220 may include a touch screen, for example, a touch, a gesture, a proximity, a drag, and a swipe using an electronic pen or a part of the user's body. A swipe or hovering input may be received.

The storage 230 may store various data used to request a pronunciation evaluation of the user's voice data and provide a user interface for displaying result data for the pronunciation evaluation. In various embodiments, the storage unit 230 may include a flash memory type, a hard disk type, a multimedia card micro type, and a card type memory (eg, SD or XD). memory, etc.), Random Access Memory (RAM), Static Random Access Memory (SRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Programmable Read-Only Memory (PROM) , a magnetic memory, a magnetic disk, and an optical disk may include at least one type of storage medium. The service providing server 200 may operate in relation to a web storage that performs a storage function of the storage unit 230 on the Internet.

The control unit 240 is operatively connected to the communication unit 210 , the display unit 220 , and the storage unit 230 , requests pronunciation evaluation for the user's voice data, and results data for pronunciation evaluation for each pronunciation characteristic. Various commands may be performed to provide a user interface for displaying .

Specifically, the controller 240 may obtain the user's voice data for a specific phoneme or a specific word, and request the service providing server 120 to evaluate the pronunciation of the acquired voice data. Here, the specific word may be a word including the pronunciation of a specific phoneme, but is not limited thereto. For example, the control unit 240 may further include an input unit such as a microphone, and may acquire the user's voice data through the input unit, but is not limited thereto. In various embodiments, when the user device 200 does not include a microphone, it is connected to an external device such as an external microphone, and voice data may be acquired through this.

In order to obtain the user's voice data for a specific phoneme or specific word, the controller 240 may provide image data for learning the pronunciation of a native speaker for a specific phoneme or specific word. For example, the image data may be data obtained by imaging a native speaker's pronunciation of a specific phoneme or a specific word, and the shape of a native speaker's mouth. Through this, the user can learn pronunciation for a specific phoneme or a specific word.

Subsequently, the controller 240 may display an interface screen for requesting pronunciation evaluation for the acquired voice data through the display unit 220 .

The controller 240 may receive pronunciation evaluation result data from the service providing server 120 and display an interface screen indicating the received pronunciation evaluation result data. The interface screen may include a graphic object or a display area for displaying at least one of the determined evaluation score, pronunciation characteristics for each phoneme extracted from the voice data, and feedback data. In various embodiments, the interface screen may further include a display area indicating image data for learning pronunciation of a native speaker for a specific phoneme or specific word.

Through this, according to the present invention, the user can perform self-learning in order to correct the user's pronunciation to the native speaker's pronunciation.

Hereinafter, the service providing server 120 will be described in detail with reference to FIG. 3 .

3 is a schematic diagram of a service providing server according to an embodiment of the present invention.

Referring to FIG. 3 , the service providing server 300 includes a communication unit 310 , a storage unit 320 , and a control unit 330 . In the presented embodiment, the service providing server 300 may refer to the service providing server 120 of FIG. 1 .

The communication unit 310 connects the service providing server 300 to enable communication with an external device. The communication unit 310 may be connected to the user device 110 using wired/wireless communication to transmit/receive various data.

Specifically, the communication unit 310 may receive the user's voice data from the user device 110 , and transmit the pronunciation evaluation result data to the user device 110 .

The storage unit 320 may store various data for providing a service for pronunciation evaluation by analyzing the user's voice data. In various embodiments, the storage unit 330 is a flash memory type, hard disk type, multimedia card micro type, card type memory (eg, SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, magnetic memory. , a magnetic disk, and an optical disk may include at least one type of storage medium. The user device 300 may operate in relation to the web storage that performs the storage function of the storage unit 330 on the Internet.

The control unit 330 is operatively connected to the communication unit 310 and the storage unit 320 , and may perform various commands to analyze the user's voice data to provide a service for pronunciation evaluation.

Specifically, the controller 330 may receive the user's voice data from the user device 110 through the communication unit 310 and evaluate the user's pronunciation based on the received voice data.

To this end, the controller 330 may extract at least one pronunciation characteristic by analyzing the received user's voice data, and use a pre-learned pronunciation evaluation model to evaluate the user's pronunciation based on the extracted at least one pronunciation characteristic. have. For example, the pronunciation evaluation model is a model pre-trained based on native speaker pronunciation as the correct answer. have.

The controller 330 may extract at least one pronunciation characteristic from the voice data using the pronunciation evaluation model, and determine an evaluation score for the user's pronunciation based on the extracted at least one pronunciation characteristic. In this case, the evaluation score may be data obtained by quantifying the degree of similarity between the pronunciation characteristic of the user and the pronunciation characteristic of a native speaker.

For example, it is assumed that for the word “Boy”, the distinctive qualities of native speakers by phoneme are “B: fricative”, “o: round lips”, and “y: Y half vowel”. The controller 330 may sort the user's voice data by phoneme, and extract at least one distinctive feature from the voice data sorted by phoneme. In other words, the controller 330 may extract at least one distinctive feature from each of “B”, “o”, and “y”.

The controller 330 may determine that the similarity between the user's pronunciation characteristics and the native speaker's pronunciation characteristics is high when the distinctive qualities extracted from the user's voice data match "low tongue", "friction sound", and "half vowel Y". In various embodiments, the control unit 330 may include other distinctive features in addition to the "low tongue", "friction sound", and "Y half vowel", or the distinctive feature extracted from the user's voice data, or "friction sound", "round lips" If only one of ” and “Y half vowel” is extracted, it may be determined that the similarity between the user's pronunciation characteristics and the native speaker pronunciation characteristics is not high or low.

When the degree of similarity is determined in this way, the controller 330 may determine an evaluation score for the user's pronunciation based on the determined degree of similarity. The evaluation score determined in this way may be provided as a result of analyzing the pronunciation accuracy of each phoneme of the user.

In various embodiments, the control unit 330 may provide comparison data in which the voice features extracted for each phoneme from the user's voice data are compared with the pronunciation characteristics for each phoneme for a native speaker. For example, when the phoneme-specific distinctive qualities of a native speaker's voice extracted from each of “B”, “o”, and “y” are “low tongue,” “friction,” and “half vowel Y”, the controller 330 controls the user Provides data indicating whether the phoneme-specific distinctive qualities extracted from the voice data of “low tongue”, “frictional consonant”, and “Y half-vowel” correspond to If there is an extracted distinctive feature, data indicating the corresponding distinctive feature may be provided, but the present invention is not limited thereto.

In various embodiments, the controller 330 may provide predetermined feedback data according to the determined evaluation score and pronunciation characteristics for each phoneme. Here, the feedback data may be data for guiding the pronunciation of the user to the pronunciation of a native speaker. Specifically, the feedback data may be data for guiding that a pronunciation characteristic matching at least one pronunciation characteristic extracted from a native speaker's voice is extracted from the user's voice data for a specific phoneme or a specific word. For example, feedback data might say, “You shouldn't be making long sounds. Open your mouth wide and make a sound. It may be a text such as, but not limited to, “Do not touch the tip of your tongue to the letter of admission.”

The controller 330 may provide the pronunciation evaluation result data including at least one of the determined evaluation score, the pronunciation characteristics for each phoneme extracted from the voice data, and the feedback data to the user device 110 .

As described above, the present invention provides the result of analyzing pronunciation accuracy for each phoneme with respect to the user's foreign language pronunciation, so that the user can perform more intense pronunciation training for the foreign language pronunciation.

Hereinafter, a method of providing a user interface for pronunciation evaluation in the user device 110 will be described with reference to FIG. 4 .

4 is a schematic flowchart illustrating a method of providing a user interface for pronunciation evaluation in a user device according to an embodiment of the present invention. The operations described below may be performed by the controller 240 of the user device 200 .

Referring to FIG. 4 , the user device 200 acquires the user's voice data for a specific phoneme or a specific word (S400), and transmits the acquired voice data to the service providing server 120 (S410). For example, the user device 200 may receive voice data of a user who pronounces a specific phoneme or a specific word using a microphone. The user device 200 may transmit the voice data input through the microphone to the service providing server 120 to request pronunciation evaluation.

The user device 200 receives pronunciation evaluation result data obtained by evaluating the user's pronunciation of a specific phoneme or a specific word based on the user's voice data from the service providing server 120 (S420), and the received pronunciation evaluation result data and displays an interface screen indicating (S430). For example, the pronunciation evaluation result data is obtained from the evaluation score determined according to the similarity between the user's pronunciation and the native speaker's pronunciation with respect to a specific phoneme or a specific word using the pronunciation evaluation model in the service providing server 120, and the user's voice data. It may include at least one of the extracted pronunciation characteristics for each phoneme and feedback data for guiding the user's pronunciation to be corrected to the native speaker's pronunciation.

Accordingly, the interface screen displaying the pronunciation evaluation result data may include an evaluation score, a graphic object indicating pronunciation characteristics and feedback data for each phoneme, or a display area.

Hereinafter, a method for evaluating pronunciation between the user device 110 and the service providing server 120 will be described with reference to FIG. 5 .

5 is a schematic flowchart for explaining a method for pronunciation evaluation between a user device and a service providing server according to an embodiment of the present invention.

Referring to FIG. 5 , the user device 110 obtains the user's voice data for a specific phoneme or a specific word ( S500 ), and transmits the acquired voice data to the service providing server ( S510 ).

The service providing server 120 evaluates the user's pronunciation by using the pronunciation evaluation model learned to evaluate the user's pronunciation based on the voice data (S520), and transmits the pronunciation evaluation result data to the user device 110 ( S530).

Specifically, the service providing server 120 may extract pronunciation characteristics for each phoneme from the user's voice data using the pronunciation evaluation model, and determine the similarity between the extracted pronunciation characteristics for each phoneme and the pronunciation characteristics for each phoneme of a native speaker. The service providing server 120 may determine an evaluation score in response to the determined similarity, and transmit the pronunciation evaluation result data including at least one of the determined evaluation score, the extracted phoneme-specific pronunciation characteristics, and the feedback data to the user device 110 . have.

The user device 110 displays an interface screen indicating pronunciation evaluation result data (S540).

Through this, according to the present invention, a user can train foreign language pronunciation without restrictions on place and time, and it is possible to reduce the burden of an educator such as a foreign language instructor for pronunciation correction learning.

Hereinafter, various interface screens related to pronunciation evaluation will be described with reference to FIGS. 6A, 6B, 6C, 6D, 6E, 6F, and 6G.

6A, 6B, 6C, 6D, 6E, 6F, and 6G are exemplary views of various interface screens related to pronunciation evaluation according to an embodiment of the present invention. These interface screens may be displayed through the display unit 220 of the user device 200 . In the presented embodiment, a case in which the user device 200 is a PC and the display unit 220 is a monitor will be described.

Referring to FIG. 6A , the user device 200 may display an interface screen 600 for evaluating the user's pronunciation of a specific phoneme or a specific word. The interface screen 600 includes a first area 602 for selecting any one of phonemes or words to be obtained from the user for pronunciation evaluation, a second area 604 indicating the type of the selected phoneme or word, and the selected phoneme. Alternatively, it may include a third area 606 representing various data related to the word.

When a “phoneme” icon for pronunciation evaluation of a specific phoneme is selected through the first area 602 , the user device 200 displays at least one graphic object indicating the types of evaluable phonemes in the second area 604 . can do.

When the graphic object 608 corresponding to the pronunciation of “/a/” is selected from among the graphic objects displayed in the second area 604 , the user device 200 displays an image for learning the pronunciation of a native speaker for the pronunciation of “/a/” A fourth area 610 displaying "/a/" and a fifth area 612 for acquiring user's voice data related to pronunciation of "/a/" may be displayed in the third area 606 . Here, the image for learning the pronunciation of a native speaker for the pronunciation of “/a/” may be data obtained by imaging the pronunciation and mouth shape of a native speaker with respect to the pronunciation of “/a/”. Also, the fifth area 612 may include a recording icon 614 for acquiring (or recording) the user's voice data.

When the recording icon 614 is selected and the user's voice data is input through a microphone built into the user device 200 or connected as an external device, the user device 200 displays the voice being input through the microphone as shown in FIG. 6B . A graphic object 616 representing data may be displayed in the fifth area 612 .

When the voice input is completed, the user device 200 may transmit the voice data to the service providing server 120 to request evaluation of the pronunciation of “/a/”.

When the pronunciation evaluation result data is received from the service providing server 120 , the user device 200 displays graphic objects 618 , 620 , 622 , 624 , 626 representing the pronunciation evaluation result data in the third area 606 as shown in FIG. 6C . ) can be displayed.

Referring to FIG. 6C , the graphic objects 618 , 620 , 622 , 624 , and 626 representing the pronunciation evaluation result data are a first graphic object 618 indicating the overall pronunciation evaluation result of the user for the pronunciation of “/a/”. , a second graphic object 620 representing at least one pronunciation characteristic extracted from a native speaker's voice with respect to pronunciation of "/a/", a second graphic object 620 indicating at least one pronunciation characteristic extracted from a user's voice with respect to pronunciation of "/a/" 3 graphic object 622, a fourth graphic object 624 indicating a similarity between the pronunciation characteristic extracted from the native speaker's voice and the pronunciation characteristic extracted from the user's voice, and a fifth graphic object for guiding the user's pronunciation to be corrected to the native speaker's pronunciation object 626 .

Here, the first graphic object 618 is an image expressing the pronunciation evaluation score of the user determined by using the pronunciation evaluation model in the service providing server 120 as at least a part of five stars, and the pronunciation evaluation score as “bad, good, It can include text expressed with words such as “excellent”.

The second graphic object 620 may be an icon or an image representing at least one pronunciation characteristic extracted from voice data of a native speaker with respect to pronunciation of “/a/”.

The third graphic object 622 may be an icon or an image representing at least one pronunciation characteristic extracted from user voice data with respect to pronunciation of “/a/”.

The fourth graphic object 624 may be an icon or an image indicating whether at least one pronunciation characteristic extracted from the native speaker's voice data and at least one pronunciation characteristic extracted from the user's voice data match, as O and X.

The fifth graphic object 626 is feedback data for guiding to correct the user's pronunciation to the native speaker's pronunciation when at least one pronunciation characteristic extracted from the native speaker's voice data and at least one pronunciation characteristic extracted from the user's voice data do not match may be text representing For example, it may be text such as “Lower your tongue to make a sound”.

Referring to FIG. 6D , when a “word” icon 628 for pronunciation evaluation of a specific word is selected through the first area 602 , the user device 200 displays a word that can be evaluated in relation to the pronunciation of “/a/” At least one graphic object 630 indicating the type of may be displayed on the third area 606 .

For example, when the graphic object 632 for pronunciation evaluation of the word “fox” is selected among the at least one graphic object 630 , the user device 200 is an image for learning the pronunciation of the word “fox” by a native speaker A sixth area 634 displaying , and a seventh area 636 for acquiring user's voice data related to the word “fox” may be displayed in the third area 606 . Here, the seventh area 636 may include a recording icon 638 for recording the user's voice data.

When the recording icon 638 is selected and the user's voice data is input through the microphone, the user device 200 displays the graphic object 640 representing the voice data being input through the microphone as shown in FIG. 6E in the seventh area 636 ) can be displayed.

When the voice input is completed, the user device 200 may transmit the voice data to the service providing server 120 to request an evaluation of the user's pronunciation of the word “fox”.

When the pronunciation evaluation result data is received from the service providing server 120 , the user device 200 displays graphic objects 642 , 644 , 646 , 648 , 650 representing the pronunciation evaluation result data in the third area 606 as shown in FIG. 6F . ) can be displayed.

Referring to FIG. 6F , the graphic objects 642 , 644 , 646 , 648 , and 650 representing the pronunciation evaluation result data are for each of “f, a, k, s” corresponding to the phoneme-specific pronunciation of the word “fox”. The first graphic object 642 indicating the evaluation score, the second graphic object 644 indicating the overall pronunciation evaluation result of the user for the word “fox”, and the phoneme-specific pronunciation characteristics extracted from the native speaker’s voice for the word “fox” The third graphic object 646 representing the word “fox”, the fourth graphic object 648 representing the phoneme-specific pronunciation characteristics extracted from the user's voice, and the phoneme-specific pronunciation characteristics extracted from the native speaker's voice and the user's voice A fifth graphic object 650 indicating a similarity between pronunciation characteristics for each phoneme may be included.

According to various embodiments, the graphic objects representing the pronunciation evaluation result data may further include a graphic object for guiding the user's pronunciation to be corrected to the native speaker's pronunciation.

Here, the first graphic object 642 is a service providing server for each phoneme-specific pronunciation of “/faks/” corresponding to the pronunciation of the word “fox”, “f”, “a”, “k”, and “s” It may be a text indicating a pronunciation evaluation score determined by using the pronunciation evaluation model in step 120 . For example, it may be a text indicating an evaluation score for each phoneme determined as 80 points for “f”, 77 points for “a”, 44 points for “k”, and 96 points for “s”.

The second graphic object 644 is an image expressing the pronunciation evaluation score of the user determined by using the pronunciation evaluation model in the service providing server 120 as at least a part of five stars, and the pronunciation evaluation score of “bad, good, excellent” It may include text expressed in words such as

The third graphic object 646 may be an icon or an image representing pronunciation characteristics for each phoneme extracted from the voice data of a native speaker for the word “fox”.

The fourth graphic object 648 may be an icon or an image representing pronunciation characteristics for each phoneme extracted from user voice data for the word “fox”.

The fifth graphic object 650 may be an icon or an image indicating whether at least one pronunciation characteristic extracted from the native speaker's voice data and at least one pronunciation characteristic extracted from the user's voice data match, as O and X.

Referring to FIG. 6G , the user device 200 may display an interface screen 652 indicating the overall evaluation result of the user's pronunciation for each phoneme.

The interface screen 652 includes a first graphic object 654 indicating the number of phonemes with high similarity to a native speaker's pronunciation among phonemes requested by the user for pronunciation evaluation, a second graphic object 656 indicating a pronunciation requiring pronunciation practice, and It may include a third graphic object 658 representing the evaluation score for each pronunciation as a graph. Through this, the user can check the pronunciation that is insufficient for the user with respect to the plurality of phonemes.

Hereinafter, an embodiment in which an interface screen related to pronunciation evaluation is implemented as a mobile web screen will be described with reference to FIG. 7 .

7 is an exemplary diagram of a mobile web screen for evaluating a user's pronunciation according to an embodiment of the present invention. In the presented embodiment, a case in which the user device 200 is a mobile device will be described.

Referring to FIG. 7 , the user device 200 obtains voice data related to a specific phoneme or specific word to be evaluated, and an interface for evaluating the user's pronunciation of a specific phoneme or specific word based on the acquired voice data The screen 700 may be displayed.

The interface screen 700 includes a first area 710 for selecting a specific phoneme or a specific word to be evaluated, a second area 720 for displaying image data for learning a native speaker's pronunciation for a specific phoneme or specific word, and A third area 730 for acquiring the user's voice data may be included.

When the recording icon 732 for recording the user's voice is selected, the user device 220 may acquire the user's voice data through a built-in microphone. The voice data obtained in this way may be displayed in the third area 730 .

The user device 200 may transmit the acquired voice data to the service providing server 120 and receive pronunciation evaluation result data from the service providing server 120 . The received pronunciation evaluation result is a first graphic object 740 indicating an evaluation score obtained by quantifying the user's pronunciation evaluation with respect to a specific phoneme or specific word, and at least one pronunciation extracted from a native speaker's voice with respect to a specific phoneme or specific word. Interface screen 700 as a second graphic object 750 indicating characteristics and a third graphic object 760 indicating whether at least one pronunciation characteristic extracted from a user's voice and at least one pronunciation characteristic extracted from a native speaker's voice match can be displayed in

The configuration of the interface screens described in the presented embodiment is not limited to the above, and objects constituting each interface screen may be configured in various ways.

As described above, the present invention enables intensive training for the user's weak pronunciation, thereby improving the user's foreign language learning ability.

In addition, since the present invention enables learning for each phoneme, detailed or intensive training for phonemes requiring pronunciation correction is possible.

The apparatus and method according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination.

The program instructions recorded on the computer readable medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the computer software field. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - Includes magneto-optical media and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Although the embodiments of the present invention have been described in more detail with reference to the accompanying drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within the scope without departing from the technical spirit of the present invention. . Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: User interface providing system for pronunciation evaluation
110, 200: user device
120, 300: service providing server

Claims (13)

a communication unit configured to transmit and receive data;
a display unit configured to display data; and
a control unit configured to be connected to the communication unit and the display unit;
The control unit is
Acquire the user's voice data for a specific phoneme or specific word,
Transmitting the acquired voice data to a service providing server for evaluating the user's pronunciation,
receiving pronunciation evaluation result data for evaluating the pronunciation of the user from the service providing server for each pronunciation characteristic, wherein the pronunciation characteristic corresponds to at least one distinctive feature detected for each phoneme;
configured to display an interface screen indicating the received pronunciation evaluation result data through the display unit,
The distinctive feature is
High tongue, low tongue, anterior tongue, posterior tongue, round lips, voiced, nasal, plosive, fricative, labial, interdental, alveolar, fluid, cleavage, R-vowel, Y-half vowel, W-half vowel, closed double, open Containing at least one of middle, central double, and velar consonants,
The pronunciation evaluation result data is
It is determined by the service providing server using a pre-learned pronunciation evaluation model to evaluate the pronunciation of the user based on the user's voice data for each of the pronunciation characteristics corresponding to the distinctive qualities,
At least one of an evaluation score for scoring the evaluation result of the user's pronunciation, pronunciation characteristics extracted from the user's voice data, and feedback data for guiding the user's pronunciation to the native speaker's pronunciation, and the native speaker's pronunciation and mouth shape Including image data imaged,
The video data is
and a graphic object representing a combination of at least two or more distinguishing qualities of any one phoneme in the native speaker's pronunciation of the specific phoneme or the specific word. delete According to claim 1, wherein the pronunciation evaluation model,
A device for providing a user interface for pronunciation evaluation, which is a model pre-learned with a correct answer for the pronunciation of a native speaker for the specific phoneme or the specific word. According to claim 1, wherein the evaluation score,
Pronunciation evaluation, which is data obtained by quantifying the degree of similarity between the pronunciation characteristic of the user and the pronunciation characteristic of the native speaker based on at least one pronunciation characteristic extracted from the voice data using the pronunciation evaluation model by the service providing server A device that provides a user interface for The method of claim 1, wherein the degree of similarity between the pronunciation characteristics of the user and the pronunciation characteristics of the native speaker is:
The apparatus for providing a user interface for pronunciation evaluation, which is determined according to whether the pronunciation characteristic of the user matches the pronunciation characteristic of the native speaker. According to claim 1, wherein the feedback data,
When the pronunciation characteristics of the user and the pronunciation characteristics of the native speaker do not match each other, the apparatus for providing a user interface for pronunciation evaluation is provided to guide the extraction of pronunciation characteristics consistent with the pronunciation characteristics of the native speaker from the user's voice data . delete A method for providing a user interface for pronunciation evaluation performed by a control unit of a user interface providing device for pronunciation evaluation, the method comprising:
obtaining voice data of a user for a specific phoneme or a specific word;
transmitting the acquired voice data to a service providing server for evaluating the user's pronunciation;
receiving, from the service providing server, pronunciation evaluation result data for evaluating the pronunciation of the user's pronunciation for each pronunciation characteristic, wherein the pronunciation characteristic corresponds to at least one distinctive feature detected for each phoneme; and
Displaying an interface screen indicating the received pronunciation evaluation result data,
The distinctive feature is
High tongue, low tongue, anterior tongue, posterior tongue, round lips, voiced, nasal, plosive, fricative, labial, interdental, alveolar, fluid, cleavage, R-vowel, Y-half vowel, W-half vowel, closed double, open Containing at least one of middle, central double, and velar consonants,
The pronunciation evaluation result data is
It is determined using a pre-learned pronunciation evaluation model to evaluate the pronunciation of the user based on the user's voice data by the service providing server for each of the pronunciation characteristics corresponding to the distinctive qualities,
At least one of an evaluation score obtained by scoring the evaluation result of the user's pronunciation, a pronunciation characteristic extracted from the user's voice data, and feedback data for guiding the user's pronunciation to the native speaker's pronunciation, and the native speaker's pronunciation and mouth shape Including image data imaged,
The video data is
A method of providing a user interface for pronunciation evaluation, comprising a graphic object representing a combination of at least two or more distinguishing qualities of any one phoneme in the native speaker's pronunciation of the specific phoneme or the specific word. delete The method of claim 8, wherein the pronunciation evaluation model comprises:
A method for providing a user interface for pronunciation evaluation, which is a model pre-trained with a correct answer for the pronunciation of a native speaker for the specific phoneme or the specific word. According to claim 8, wherein the evaluation score,
Pronunciation evaluation, which is data obtained by quantifying the degree of similarity between the pronunciation characteristic of the user and the pronunciation characteristic of the native speaker based on at least one pronunciation characteristic extracted from the voice data using the pronunciation evaluation model by the service providing server How to provide a user interface for The method of claim 8, wherein the degree of similarity between the pronunciation characteristic of the user and the pronunciation characteristic of the native speaker is:
The method of providing a user interface for pronunciation evaluation, which is determined according to whether the pronunciation characteristics of the user and the pronunciation characteristics of the native speaker match. The method of claim 8, wherein the feedback data,
When the pronunciation characteristics of the user and the pronunciation characteristics of the native speaker do not match with each other, the method for providing a user interface for pronunciation evaluation is provided to guide the extraction of pronunciation characteristics matching the pronunciation characteristics of the native speaker from the user's voice data .

Images