KR101411039B1 - Method for evaluating pronunciation with speech recognition and electronic device using the same - Google Patents

Abstract

A pronunciation evaluation method using speech recognition is provided. The pronunciation evaluation method according to the embodiment of the present invention evaluates the user's pronunciation by using a plurality of candidate sentences extracted as a result of speech recognition of the user's speech. As a result, evaluation of pronunciation can be provided immediately and objectively.

Description

TECHNICAL FIELD [0001] The present invention relates to a pronunciation evaluation method using speech recognition,

The present invention relates to a pronunciation evaluation method, and more particularly, to a pronunciation evaluation method for acquiring a user's voice and evaluating the pronunciation accuracy through an application, and an electronic apparatus using the method.

The English learning application on the smartphone enabled easy learning of English regardless of time and place. Such an English learning application has provided various learning programs using various functions of the smartphone.

Typically, the recording function of the smartphone is used to record the user's English pronunciation, and the user is provided with the pronunciation again so that the user can directly hear and confirm his pronunciation. Furthermore, there is also a case where a recording file is registered on a bulletin board on the Internet using the wireless Internet function of a smartphone, and a pronunciation evaluation (a comment, a comment, etc.) is sent and received between users.

However, pronunciation evaluations of this type are based on subjective evaluation of oneself or others and are not objective. In addition, if there is no other person actively participating in the evaluation, there is a problem that it becomes useless because it is impossible to receive the evaluation of others.

In addition, even if a person evaluates his / her pronunciation, it takes a comparatively short time until another person confirms a recorded file through a bulletin board. Therefore, there is a frustration that an immediate evaluation is virtually impossible and a prompt evaluation is not received.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide a method of estimating pronunciation accuracy by using speech recognition as a method for more objective and immediate pronunciation evaluation, .

According to another aspect of the present invention, there is provided a pronunciation evaluation method including: obtaining a voice of a user; Acquiring a plurality of candidate sentences extracted as speech recognition results of the obtained voice of the user; And evaluating the pronunciation of the user using the candidate sentences.

Further, the pronunciation evaluation method according to the embodiment of the present invention may further include the step of providing a sentence, and the voice acquisition step may acquire a voice of a user reading the sentence provided in the providing step.

The voice recognition of the voice of the user may be voice recognition in units of sentences.

The evaluating step may include: dividing a plurality of candidate sentences into words; Generating a word group by merging and removing the separated words through the separation step; Recognizing words included in the word group among words constituting the provided sentence; And evaluating the pronunciation accuracy of the user based on the number of words recognized in the recognition step.

The evaluating step may further include removing the punctuation marks from the plurality of candidate sentences, and the separating step may separate the plurality of candidate sentences from which the punctuation marks have been removed into words.

In the pronunciation evaluation method according to the embodiment of the present invention, in the sentence provided in the providing step, the words identified in the recognition step may be changed to a color different from the unidentified word.

The pronunciation accuracy evaluation step may include calculating pronunciation accuracy based on the number of words recognized in the recognition step and the number of words constituting the provided sentence; And providing the pronunciation accuracy.

The candidate sentence obtaining step may include: requesting voice recognition while providing the voice of the user obtained in the voice obtaining step to the server; And receiving a plurality of candidate sentences extracted through speech recognition from the server in response to the speech recognition request.

According to another aspect of the present invention, there is provided an electronic device including: a microphone for acquiring a voice of a user; And a processor for evaluating the pronunciation of the user using a plurality of candidate sentences extracted as speech recognition results of the user's voice acquired through the microphone.

The control unit separates a plurality of candidate sentences into words, generates a word group by integrating the separated words while eliminating the redundant words, and then recognizes words included in the word group among the words constituting the sentence The pronunciation accuracy of the user can be evaluated.

According to another aspect of the present invention, there is provided a computer-readable recording medium including: Acquiring a plurality of candidate sentences extracted as speech recognition results of the obtained voice of the user; And evaluating the pronunciation of the user using the candidate sentences. The program for performing the pronunciation evaluation method is recorded.

INDUSTRIAL APPLICABILITY As described above, according to the present invention, the accuracy of pronunciation can be evaluated using speech recognition, and evaluation of pronunciation can be provided promptly and objectively.

Further, according to the present invention, since the pronunciation accuracy is calculated through the speech recognition on a sentence-by-word basis and the matching ratio on a word-by-word basis, the comparison performance of word units is further improved while being optimized for sentences, and reliable pronunciation accuracy can be calculated.

In addition, since the pronunciation accuracy evaluation result is provided in units of words constituting the sentence, it is possible to grasp the word that is inaccurately pronounced in the sentence, and intensive learning is possible for the inaccurate word as well.

In addition, the sentence including punctuation marks such as a period, a question mark, and an exclamation mark regardless of whether the character is a case or not can be calculated relatively accurately.

1 shows a main screen of an English pronunciation evaluation application using speech recognition,
2 shows an E-Study menu execution screen,
FIG. 3 shows a word list screen provided when Unit-1 is selected from the units shown in FIG. 2,
4 shows a word learning screen provided when "2. to death" is selected among the words shown in FIG. 3,
FIG. 5 shows a sentence learning screen,
FIG. 6 shows a sentence recording screen displayed when the Speak button shown in FIG. 4 is selected,
FIG. 7 shows a sentence learning screen in which the recorded user's accuracy of the pronunciation of the sentence is displayed,
8 is a flowchart provided in the explanation of the sentence pronunciation evaluation method according to the preferred embodiment of the present invention,
9 is a block diagram of a smartphone according to a preferred embodiment of the present invention.

Hereinafter, the present invention will be described in detail with reference to the drawings.

1 is a diagram showing a main screen of an English pronunciation evaluation application using speech recognition. The main screen shown in Fig. 1 corresponds to a user interface screen first displayed on the display when an English pronunciation evaluation application (abbreviated as "application" hereinafter) is executed.

As shown in FIG. 1, four main menus (E-Study, English, My Note, and Tutorial) are displayed on the main screen of the application.

1) "E-Study" menu is a menu for learning contents stored in DB (DataBase)

2) "Brutal English" is a menu for learning by inviting e-mail received for learning English.

3) The "My Note" menu is for re-learning only the words registered in the "E-Study" menu,

4) The "Tutorial" menu is a menu for receiving instructions on how to use the application.

2 is a view showing a screen on which the "E-Study" menu is executed. As shown in the upper part of Fig. 2, it can be seen that the "E-Study" has a format for selecting one of the three parts.

Parts are a bundle of conversations separated by themes. Specifically, Part-1 is conversations about behavior, Part-2 is conversations about emotions Part-3 is conversations about everyday life. In addition, each part consists of word learning, sentence learning and conversation learning.

FIG. 2 shows a state in which 12 selectable units for word learning of Part-1 (action) are displayed on the "E-Study" menu screen.

Fig. 3 shows a word list screen provided when "Unit-1" is selected among the units shown in Fig. FIG. 4 shows a word learning screen provided when "2. to death" of the words shown in FIG. 3 is selected.

As shown in Fig. 4, a word " Previous "button for calling a previous word is displayed on the left side and a" Next "button for calling the next word is displayed on the right side appear.

In the middle of the word learning screen, there are 1) a "Listen" button, 2) a "Speak" button, 3) a "Flash" 5) "Goal Setting" button and 6) "Interpretation" button.

1) The "Listen" button is a button for outputting the word pronunciation to hear the pronunciation of the word. 2) The "Speak" button is for the user to measure the accuracy of the pronunciation of the word It is a button to select when.

3) "Flash" button is a button for commanding to alternately display the word and word meaning; and 4) "My Note" button is a button for registering the current word separately for re-learning later , It is possible to re-learn only words registered through the above-described "My Note" menu.

5) The "Goal Setting" button is used to set the target pronunciation accuracy in "Speak". With this button, the target can be set according to the toggle method. Each time the button is selected, the target toggles to 'Not set → 30% → 50% → 70% → 90% → not set'.

6) The "Interpretation" button is a button for instructing not to display or display the word meaning together with the word.

FIG. 5 shows a sentence learning screen. 5, the sentence and sentence meanings appear at the center in the sentence learning screen, and a "Previous" button for calling the previous sentence and a "Next" button for calling the next sentence appear in the center lower part.

At the center bottom of the sentence learning screen, there are 1) "Listen" button, 2) "Speak" button, 3) "Record" button, 4) ) "Button, 5)" Goal Setting "button, and 6)" Interpretation "button.

1) The "Listen" button is a button for commanding the pronunciation of the sentence to hear the pronunciation of the sentence. 2) The "Speak" button is for the user to measure the accuracy of pronunciation of the sentence It is a button to select when.

3) The "Record" button is used to instruct the user to record the pronunciation of the sentence. 4) The "Play" button is used to play back the recorded content via "Record" It is a button for command. If nothing is recorded, the "Play" button is inactivated as shown in Fig.

5) The "Goal Setting" button is used to set the target pronunciation accuracy in "Speak". 6) The "Interpretation" button is used to display the sentence meaning with sentence or not Button.

6 shows a sentence recording screen that appears when the "Speak" button shown in Fig. 4 is selected. The sentence recording screen shown in FIG. 6 is a screen that appears while recording the pronunciation of the sentence of the user for measuring the accuracy of sentence pronunciation.

FIG. 7 is a diagram showing a sentence learning screen in which the accuracy of pronunciation of a sentence of a user is evaluated. As shown in FIG. 7, it can be confirmed that some of the words of the sentence are changed to red, and the words changed to red are words that are correctly pronounced by the user in the sentence. On the other hand, words kept black are words that the user did not pronounce correctly in the sentence.

On the other hand, as shown in FIG. 7, it can be seen that the "recognition rate: 58%" is displayed, which indicates the pronunciation accuracy. The recognition rate can be calculated as [(number of correctly pronounced words) / (total number of words in a sentence) * 100].

Although not shown in FIG. 7, when the recognition rate (58%) exceeds the target (30%), a circle is displayed in the sentence.

Hereinafter, the process of evaluating sentence pronunciation will be described in detail with reference to FIG. 8 is a flowchart provided in the explanation of a sentence pronunciation evaluation method according to a preferred embodiment of the present invention. The flowchart shown in Fig. 8 can be understood as an algorithm executed by a smart phone application.

As shown in FIG. 8, an application executed on a smartphone first displays a sentence learning screen on which a sentence to be learned is displayed on a touch screen, and provides the sentence learning screen to a user (S105).

When the "Speak" button is selected on the sentence learning screen (S110-Y), the application displays a sentence recording screen as shown in FIG. 6 on the touch screen, and the sentence And acquires the voice of the user who is reading (S115).

Then, the application requests the voice recognition while providing the voice of the user acquired in step S115 (S120).

In step S120, the speech recognition server extracts 10 candidate sentences from the user's voice received from the smart phone through speech recognition on a sentence-by-sentence basis. 10 candidate sentences can be extracted in descending order of similarity, and 10 extracted candidate sentences are transmitted to the smartphone in response to a voice recognition request.

Accordingly, the smartphone application receives 10 candidate sentences extracted by the speech recognition server (S125), and removes the sentence codes from the received 10 sentences (S130). Punctuation marks are exclamation points (!), Question marks (?), Commas (,), and periods (.

Then, the application separates the ten candidate sentences from which the punctuation marks have been removed in units of words (S135), and separates and removes the separated words to create a word group (S140).

For example, if the ten candidate sentences received from the speech recognition server are "It's my life.", "It is my knife.", ..., "It's my lie.", The group becomes [It's, my, life, It, is, knife, lie].

When the word group creation is completed, the application recognizes words included in the word group generated in step S140 among the words constituting the sentence provided in step S105 (S145).

Of the words constituting the sentence, the words recognized as being included in the word group in step S145 are classified as correctly pronounced words, while the other words are not correctly pronounced.

If the sentence provided in step S105 is "He is my knight." And the word group generated in step S140 is [It's, my, life, It, is, knife, lie] Since "is" and "my" are included in the word group, they are classified as correctly pronounced words, but "He" and "knight" are not included in the word group and are classified as not pronounced words.

On the other hand, when the word is recognized in step S145, uppercase and lowercase characters are ignored. For example, "It" and "it" are treated as the same word.

Thereafter, the application changes the words of sentences identified as being included in the word group in step S145 from red to a sentence (S150). Further, the application calculates the accuracy of the pronunciation of the sentence and displays it on the touch screen to inform the user (S155).

In step S155, the application calculates the ratio of the number of words constituting the sentence (the total number of words in the sentence) to the number of words recognized as being included in the word group in step S145 (the number of correctly pronounced words) (Recognition rate) can be calculated. The results of the operations in steps S150 and S155 are illustrated in FIG.

If the sentence provided in step S105 is "He is my knight." And the word group generated in step S140 is [It's, my, life, It, is, knife, lie] The number is "4", and the number of words recognized as being included in the word group is "2", so that the pronunciation accuracy is 50%.

FIG. 9 is a block diagram of a smartphone according to a preferred embodiment of the present invention, in which an application capable of performing the sentence pronunciation evaluation method shown in FIG. 8 is installed / executed. 9, the smartphone 200 according to the preferred embodiment of the present invention includes a communication unit 210, a touch screen 220, a processor 230, a speaker 240, a microphone 250, (260).

The communication unit 210 establishes and maintains a communication connection with an access point (AP) and peripheral devices as well as a base station through mobile communication and wireless networking. In particular, the communication unit 210 establishes a communication connection with the application server and the voice recognition server.

The touch screen 220 functions as a display for outputting an application execution screen, and functions as a user input means for receiving a user command and transmitting it to the processor 230.

The speaker 240 is an audio output means and outputs word pronunciation, sentence pronunciation, and pronunciation of the recorded user.

The microphone 250 converts the pronunciation of the user into an electrical signal from the voice signal, acquires the user voice, and transmits the acquired user voice to the processor 230.

The storage unit 260 is a storage medium that stores programs and data necessary for a smartphone, and the above-described applications are installed.

The processor 230 controls the overall operation of the smartphone according to a user command input through the touch screen 220. [ Particularly, the processor 230 executes the application installed in the storage unit 260 and performs the sentence pronunciation evaluation shown in Fig.

Up to now, a method of evaluating sentence pronunciation using speech recognition and a smartphone installed / executed by an application capable of performing the method have been described in detail.

In the above embodiment, English pronunciation evaluation is assumed, which is merely an example for convenience of explanation, and the present invention can also be applied to pronunciation evaluation for languages other than English.

In the above embodiment, the speech recognition server extracts 10 candidate sentences. However, this is also an example, and the number of candidate sentences can be changed according to the necessity and the DB / specification of the server.

In this case, the smart phone does not need to transmit the user voice to the voice recognition server, but the voice recognition server and the voice recognition server need to have an algorithm and DB for voice recognition.

On the other hand, in the above embodiment, a smartphone is referred to as an apparatus for installing / executing an application capable of performing sentence pronunciation evaluation using speech recognition. The smart phone is an example of an electronic device in which the application can be installed / executed . It goes without saying that the technical idea of the present invention is also applicable to electronic devices other than smart phones.

It goes without saying that the technical idea of the present invention can also be applied to a computer-readable recording medium having a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention may be embodied in computer-readable code form recorded on a computer-readable recording medium. The computer-readable recording medium is any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. In addition, the computer readable code or program stored in the computer readable recording medium may be transmitted through a network connected between the computers.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

200: smartphone 210: communication unit
220: Touch screen 230: Processor
240: speaker 250: microphone
260:

Claims (11)

Providing a sentence;
Obtaining a voice of a user reading a sentence provided in the providing step;
Extracting and acquiring a plurality of candidate sentences in descending order of similarity through speech recognition on the basis of sentences of the speech of the user;
Separating the words included in each of the plurality of candidate sentences into words;
Generating a word group by merging and removing the separated words through the separation step; And
Comparing the words constituting the sentence with words included in the word group and evaluating pronunciation of the user,
In the evaluation step,
Identifying words included in the word group among the words constituting the sentence individually irrespective of the order of inclusion; And
And evaluating the pronunciation accuracy of the user based on the number of words recognized in the recognition step,
The plurality of candidate sentences including a first candidate sentence and a second candidate sentence,
Wherein the separating step comprises:
A first separating step of separating the first candidate sentence into words; And
And a second separating step of separating the second candidate sentence into words,
Wherein the generating comprises:
Wherein a word group is generated by combining words separated from the first candidate sentence in the first separation step and words separated from the second candidate sentence in the second separation step, .
delete delete delete The method according to claim 1,
Removing punctuation marks from the plurality of candidate sentences,
Wherein the separating step comprises:
Wherein a plurality of candidate sentences from which the punctuation marks have been removed are divided into words.
The method according to claim 1,
And changing, in the sentence provided in the providing step, the words identified in the holding step to a color different from the unidentified word.
The method according to claim 1,
In the pronunciation accuracy evaluation step,
Calculating a pronunciation accuracy based on the number of words recognized in the recognition step and the number of words constituting the provided sentence; And
And providing the pronunciation accuracy. ≪ Desc / Clms Page number 19 >
The method according to claim 1,
The candidate sentence acquiring step includes:
Requesting voice recognition while providing the voice of the user acquired in the voice acquiring step to a server; And
And receiving a plurality of candidate sentences extracted through speech recognition from the server in response to the speech recognition request.
A microphone for acquiring a voice of a user who reads the provided sentence; And
And a processor for evaluating pronunciation of the user by using a plurality of candidate sentences extracted in descending order of similarity through speech recognition on a sentence unit basis of the user's voice acquired through the microphone,
The processor comprising:
The words included in each of the plurality of candidate sentences are divided into words, the separated words are removed, and words are grouped together, and then words constituting the sentence are compared with words included in the word group The pronunciation of the user is evaluated, and the words included in the word group among the words constituting the sentence are individually grasped irrespective of the order of inclusion, and the pronunciation accuracy of the user is evaluated based on the number of recognized words In addition,
The plurality of candidate sentences including a first candidate sentence and a second candidate sentence,
The processor comprising:
And separating the first candidate sentence by a word unit, separating the second candidate sentence by a word unit, integrating words separated from the first candidate sentence and words separated from the second candidate sentence, And generates a word group.
delete Providing a sentence;
Obtaining a voice of a user reading a sentence provided in the providing step;
Extracting and acquiring a plurality of candidate sentences in descending order of similarity through speech recognition on the basis of sentences of the speech of the user;
Separating the words included in each of the plurality of candidate sentences into words;
Generating a word group by merging and removing the separated words through the separation step; And
Comparing the words constituting the sentence with words included in the word group and evaluating pronunciation of the user,
In the evaluation step,
Identifying words included in the word group among the words constituting the sentence individually irrespective of the order of inclusion; And
And evaluating the pronunciation accuracy of the user based on the number of words recognized in the recognition step,
The plurality of candidate sentences including a first candidate sentence and a second candidate sentence,
Wherein the separating step comprises:
A first separating step of separating the first candidate sentence into words; And
And a second separating step of separating the second candidate sentence into words,
Wherein the generating comprises:
Wherein a word group is generated by combining words separated from the first candidate sentence in the first separation step and words separated from the second candidate sentence in the second separation step, A computer-readable recording medium having recorded thereon a program capable of performing the above-described operations.

Images