JP2005031207A - Pronunciation practice support system, pronunciation practice support method, pronunciation practice support program, and computer readable recording medium with the program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pronunciation practice support system and a pronunciation practice support method capable of providing content of pronunciation practice for English conversation or the like in a mobile environment such as a cellular phone, and to provide a pronunciation support program for realizing the pronunciation practice support system and a computer readable recording medium with the program recorded thereon. <P>SOLUTION: This pronunciation practice support system 1 is communicably connected to a learner's mobile apparatus 100 and is provided with a channel communication control device 10 for acquiring speech data inputted from the mobile apparatus 100 by the learner, an utterance evaluation engine 33 for evaluating the learner's pronunciation included in the speech data acquired by the channel communication control device 10, a content editing section 44 for editing the content according to the evaluation result by the utterance evaluation engine 33, and a Web Server 20 for displaying the contents edited by the contents editing section 44 on the mobile apparatus 100. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a system for supporting pronunciation practice such as English conversation, and more particularly, a pronunciation practice support system for assessing a learner's pronunciation, a pronunciation practice support method, a pronunciation practice support program, and a computer-readable record recording the same It relates to the medium.
[0002]
[Prior art]
It is estimated that there are always 13 million English learners in Japan. The number of English conversation schools has exceeded 10,000 classrooms. The number of examinees taking various English proficiency tests exceeds 5 million each year. In this way, Japanese people are highly motivated to learn English, especially English conversation.
[0003]
Also, while there is a desire to challenge English conversation communication, there are circumstances such as “I can not take a long time to study”, “English school is expensive and far away”, “I can not continue even if I buy teaching materials”, Many potential English learners are hesitant about learning because of anxiety such as “books, magazines, and television alone are too unilateral”.
[0004]
Under these circumstances, services that allow easy learning of English using a mobile phone are provided. For example, there is a service that allows a learner to practice “listening to English” by playing audio data in which a talent speaks a short phrase entangled in a column (Non-Patent Document 1). In addition, there is a service that allows a learner to practice “listening to English” by reproducing audio data of English sentences of a movie line for sales promotion of a movie DVD (Non-patent Document 2).
[0005]
In order to learn English conversation communication, it is said that it is effective to repeat the cycle of “learning basic elements”, “read”, “speak”, “listen” and “write”. In Japan, despite learning English for a long time at junior high school, high school, and university, many people say that they are not good at English conversation communication, but the reason is that they are not practicing speaking. It is considered. Therefore, it is effective for Japanese learners of English to use the “Communicative Approach” to train (experience) speaking from the viewpoint of communication, assuming actual situations based on basic learning elements. It is.
[0006]
In this regard, Non-Patent Document 3 describes software for a personal computer that artificially reproduces a free talk in English conversation using live action and voice recognition. In addition, this software can set the utterance level to 5 levels so that it can be recognized widely from fluent English to beginner katakana English.
[0007]
[Non-Patent Document 1]
“Sain Camus ABC”, [online],
[Search June 23, 2003] Internet
<URL: http: // i. thane. tc / no_member_general. php>
[0008]
[Non-Patent Document 2]
“Cinema English”, [online],
[Search June 23, 2003] Internet
<URL: http: // i_cinema. tsutaya. co. jp />
[0009]
[Non-Patent Document 3]
Learning Wear Co., Ltd., “Native World Functional Overview”, [online]
[Search June 23, 2003] Internet
<URL: http: // www. learningware. co. jp / product / nw / nw_toku. htm>
[0010]
[Non-Patent Document 4]
"Voice-speaking automatic rating system for language learners", Ritsumeikan University Faculty of Science and Technology, Department of Information Yoichi Yamashita, November 11, 2002, at the 9th Forum of the Intellectual Union Promotion Organization
[0011]
[Problems to be solved by the invention]
However, the conventional English learning service using a mobile phone is only “listening to English” practice, and there is no service that provides “speaking English” practice.
[0012]
Conventionally, there is software that provides “speaking English” practice, but because it is for a personal computer, there are significant restrictions on the place and time for the learner. That is, there were problems in portability and immediate availability. In addition, the learner is required to have skills for using a personal computer. Therefore, this software was not as easy to use as a mobile phone. Furthermore, since the change of the content of the teaching material involves an upgrade, it has been difficult to flexibly change the content of the teaching material.
[0013]
The present invention has been made to solve the above problems, and its purpose is to provide a pronunciation practice support system and pronunciation practice support capable of supplying pronunciation practice content such as English conversation in a mobile environment such as a mobile phone. It is to provide a method. Further, the object of the present invention includes providing a pronunciation practice support program that realizes the pronunciation practice support system and a computer-readable recording medium that records the program.
[0014]
[Means for Solving the Problems]
In order to solve the above problems, the pronunciation practice support system of the present invention is a pronunciation practice support system that is communicably connected to a learner's terminal device, and acquires voice data input by the learner from the terminal device Audio data acquisition means for performing, pronunciation evaluation means for evaluating the pronunciation of the learner included in the audio data acquired by the audio data acquisition means, and content editing for editing the content according to the evaluation result by the pronunciation evaluation means Means, and content presenting means for presenting the content edited by the content editing means to the terminal device.
[0015]
The pronunciation practice support method of the present invention is a pronunciation practice support method by a pronunciation practice support system that is communicably connected to a learner's terminal device, and is a voice that acquires voice data input from the terminal device by the learner. A data acquisition step, a pronunciation rating step for rating a learner's pronunciation included in the voice data acquired in the voice data acquisition step, and a content editing step for editing content according to the rating result in the pronunciation rating step; And a content presentation step of presenting the content edited in the content editing step on the terminal device.
[0016]
Using the above-described configuration and method, the voice data input by the learner from the terminal device (such as a mobile phone) is acquired, and the pronunciation of the learner included in the voice data is evaluated in the pronunciation practice support system, and the content according to the result Can be presented to the terminal device. In the present specification, “pronunciation” means the meaning of speech uttered by the learner, and is not limited to the utterance of words or sentences in a foreign language, and includes, for example, karaoke and musical instrument performance. That is, the pronunciation practice support system of the present invention is widely applicable as a system for remotely assessing various sounds produced by a learner by installing pronunciation rating means suitable for target speech data.
[0017]
Thus, pronunciation practice content such as English conversation can be supplied in a mobile environment such as a mobile phone. Therefore, it is possible to meet the demands of learners who want to use the “speaking” practice easily.
[0018]
In addition, since the content to be presented on the terminal device is transmitted from the pronunciation practice support system each time, it is easier to change the content as compared with a stand-alone device such as a personal computer.
[0019]
Moreover, since the content to be presented on the terminal device is edited according to the rating result by the pronunciation rating means, it is possible to present the content with appropriate contents according to the situation such as the learning progress and proficiency of the learner. Therefore, it is possible to provide a learning service that can dynamically reflect the situation of each learner on the learning content.
[0020]
From the above, if the pronunciation practice support system is used, for example, as an English conversation learning service, English learners will be trained in “speaking” from the viewpoint of communication, assuming actual situations based on basic learning elements (experience) Yes, it will be possible to provide an opportunity for a “communicadian approach”. Therefore, "I want to learn in a little free time", "Easy learning service that can be expected to have some effect", "Is there any way to enjoy and learn easily", "Content that you want to continue even if your intention is weak ”And“ Interactive content that gives priority to my convenience ”.
[0021]
Furthermore, the pronunciation practice support system of the present invention is such that the pronunciation rating means can change the strictness of the rating according to the accuracy parameter, and the accuracy of the accuracy according to the history of the rating results by the pronunciation rating means. It is characterized by comprising accuracy parameter changing means for changing parameters, and accuracy parameter holding means for holding the accuracy parameters set by the accuracy parameter changing means for each learner.
[0022]
With the above configuration, the accuracy of the rating by the pronunciation rating unit can be changed according to the history of the rating result of the pronunciation rating unit by using the accuracy parameter.
[0023]
Therefore, the strictness of the rating can be changed efficiently according to the situation such as the learning progress and proficiency of the learner. Therefore, the system can automatically reflect different situations for each learner on the learning content. Therefore, it becomes possible to provide a service that can flexibly customize the learning content according to the learner like an English conversation school.
[0024]
Furthermore, the pronunciation practice support system of the present invention is characterized by comprising learner authentication means for authenticating a learner based on the voice data acquired by the voice data acquisition means.
[0025]
With the configuration described above, further, authentication of a learner necessary for providing a service different for each learner can be performed based on voice data input from the terminal device by the learner. Note that the voice data for authentication may be a learner's name or a greeting. Moreover, you may authenticate in combination with ID and a password.
[0026]
Therefore, the service by the pronunciation practice support system can be used without a sense of resistance without making the learner aware of the authentication.
[0027]
The pronunciation practice support system of the present invention further includes learner data storage means for storing learner data for each learner, and the content editing means is stored in the data stored in the learner data storage means. It is characterized by including a classmate adding means for causing a character who simulates another learner to appear in the content.
[0028]
According to the above configuration, another learner can appear in the learner's content. Thereby, the atmosphere of a classroom can be given to content. Further, since other learners appearing in the content are based on the data of actual learners, the learners can feel reality.
[0029]
An avatar can be used as a presentation method for other learners. By expressing the virtual personality as the conversation partner with an avatar, smooth communication is possible, and it is easy to expect high learning effects and continuity.
[0030]
Further, the pronunciation practice support system of the present invention is characterized in that the pronunciation rating means performs association by labeling using both a learner's native language phoneme and a learned foreign language phoneme. Yes.
[0031]
With the above-described configuration, the pronunciation evaluation means further determines whether words and phrases are appropriate by labeling using both the phoneme of the learner's native language (for example, Japanese) and the phoneme of the foreign language to be learned (for example, English). Therefore, it is possible to accurately evaluate without judging a rating error or inappropriate utterance.
[0032]
Therefore, the pronunciation practice support system reduces learning efficiency and usability even when using a terminal device with a process overlap between a voice call function like a mobile phone and a data display / view function as a mobile terminal. It is possible to provide a service.
[0033]
Therefore, the pronunciation rating means is suitable as a pronunciation rating engine mounted on a pronunciation training support system that provides content for pronunciation training in a foreign language.
[0034]
The pronunciation training support program of the present invention is a computer program that causes a computer to function as each of the above-described means.
[0035]
With the above configuration, the pronunciation training support system can be realized by realizing each unit of the pronunciation training support system with a computer.
[0036]
A computer-readable recording medium on which the pronunciation practice support program of the present invention is recorded is a computer-readable recording medium on which a pronunciation practice support program for operating the pronunciation practice support system is realized by causing the above-described means to be realized by a computer. It is a recording medium.
[0037]
With the above configuration, the pronunciation practice support system can be realized on a computer by the pronunciation practice support program read from the recording medium.
[0038]
DETAILED DESCRIPTION OF THE INVENTION
One embodiment of the present invention will be described below with reference to FIGS.
[0039]
FIG. 1 is a functional block diagram showing an outline of the configuration of the pronunciation training support system 1 according to the present embodiment.
[0040]
The pronunciation practice support system 1 supplies content for pronunciation practice such as English conversation to a mobile device (terminal device) 100 such as a mobile phone via a communication network.
[0041]
In this embodiment, in order to provide a service for practicing English conversation, a configuration for evaluating the applicability of learners' pronunciation as English will be described, but the type of voice data to be evaluated is not limited to English. That is, the pronunciation practice support system 1 can employ an utterance rating engine 33 (described later) corresponding to the voice data to be rated. Thus, the pronunciation practice support system 1 can be configured as a system that supports utterances in languages other than English, karaoke, and practice of musical instrument performance.
[0042]
As shown in FIG. 1, the pronunciation practice support system 1 is connected to a mobile device 100 via a mobile phone communication network (not shown). Specifically, the voice call module 101 of the mobile device 100 and the voice call module 11 of the line call control device 10 are connected for a voice call. Further, the data communication module 102 of the mobile device 100 and the data communication module 21 of the Web server 20 are connected for data communication.
[0043]
The pronunciation practice support system 1 includes a line call control device (voice data acquisition means) 10, a Web server (content presentation means) 20, an utterance rating server 30, and a database server 40. Each device constituting the pronunciation practice support system 1 is connected via a LAN (local area network) by data communication modules 12, 21, 31, and 41 for data communication. The device configuration in the pronunciation practice support system 1 can be changed as appropriate. For example, a plurality of devices may be provided according to the processing speed and communication speed, or the four devices shown in FIG. 1 may be integrated into one device. Further, the device manager, the operating system, the native application interface, and the like are omitted from the blocks of the apparatuses shown in FIG.
[0044]
The line call control device 10 includes a voice call module 11, a data communication module 12, a voice call control unit 13, and a voice data conversion unit 14. In the line call control device 10, the voice call control unit 13 receives a voice call signal from the mobile device 100, and the voice data conversion unit 14 converts the voice call signal into a voice data signal that can be processed by the pronunciation practice support system 1. To do. That is, the line call control device 10 acquires the voice data of the learner who is the evaluation target from the mobile device 100. The line call control device 10 may include a terminal authentication unit that authenticates the mobile device 100.
[0045]
The web server 20 includes a data communication module 21 and a web server unit 22. The Web server 20 transmits the content of the English conversation learning service to the mobile device 100 and causes the mobile device 100 to present the content. Specifically, the Web server unit 22 communicates with the virtual machine 103 or the Web browser 105 of the mobile device 100, and transmits an HTML (hypertext markup language) file or the like for the mobile device 100 to present content. Obtained from the content editing unit 44 and transmitted to the mobile device 100.
[0046]
The utterance rating server 30 includes a data communication module 31, a speech recognition engine 32, an utterance rating engine (pronunciation rating means) 33, and an utterance data pattern storage unit 34. The utterance rating server 30 rates the learner's pronunciation included in the voice data acquired by the line call control device 10, and transmits the rating result to the database server 40.
[0047]
The speech recognition engine 32 cuts out phonemes from the learner's speech data.
[0048]
The utterance rating engine 33 performs association by labeling using both Japanese phonemes (the learner's native language) and English phonemes (the foreign language to be learned), so that the learner's utterance word or sentence and the data pattern Matching is performed to automatically evaluate the degree of similarity between the basic frequency patterns of the two utterances. The utterance rating engine 33 refers to the utterance data pattern stored in the utterance data pattern storage unit 34 as appropriate. Further, the utterance rating engine 33 can change the strictness of the rating in accordance with the accuracy parameter 57 (FIG. 2A). The accuracy parameter 57 is stored in the user database 45, and the utterance rating engine 33 acquires the learner accuracy parameter 57 via the user management unit 43.
[0049]
The utterance data pattern storage unit 34 stores utterance data patterns of native English speakers for assessing the suitability of the learner's pronunciation in the content. Note that the utterance data pattern of the utterance for authentication by the learner for authenticating the learner is stored in the user database 45 as an audio file 53 (FIG. 2A).
[0050]
The database server 40 includes a data communication module 41, a user authentication unit (learner authentication unit) 42, a user management unit 43, a content editing unit (content editing unit) 44, and a user database (learner data storage unit, accuracy parameter holding unit). 45, a content database 46 is provided.
[0051]
The user authentication unit 42 authenticates a learner who is a user of the mobile device 100. As the method, a password may be used, or the utterance rating engine 33 may perform comparison with the utterance data pattern (speech file 53) of the learner himself registered in advance as will be described later.
[0052]
The user management unit 43 manages the user database 45. FIG. 2A is an explanatory diagram illustrating an example of a data structure of data stored in the user database 45. In the user database 45, data for managing learners is registered for each learner. FIG. 2A is an example, and the user database 45 is not limited to this.
[0053]
Specifically, as shown in FIG. 2A, in the user database 45, the phone number 52, the audio file 53, the page number 54, the application parameter 55, the log 56, the learner's user ID 51 as a key, The accuracy parameter 57 is registered in association with it. In addition to these, the user database 45 can appropriately register a user's nickname, an e-mail address, and the like.
[0054]
The user ID 51 is a character string for identifying a learner. The telephone number 52 is a character string indicating the learner's telephone number. The audio file 53 is a 16-bit-16 kHz PCM (pulse-code modulation) file containing the learner's own utterance data pattern registered in advance for authentication of the learner. The page number 54 is a character string indicating an html page representing the most recent state of the service used by the learner. The application parameter (content ID) 55 is a character string for identifying the content page. The log 56 is a character string indicating the service use history of the learner. The accuracy parameter 57 is a numerical value (1 to 5 is assigned in the present embodiment) indicating the strictness of the rating by the utterance rating engine 33.
[0055]
Further, the user management unit 43 includes an accuracy parameter changing unit (accuracy parameter changing means) 43a. The accuracy parameter changing unit 43 a changes the accuracy parameter 57 according to the rating result history by the utterance rating engine 33 and stores it in the user database 45 in association with the user ID 51. In this way, by managing the accuracy parameter 57 in association with the user ID 51, the learning progress and skill level of each learner can be dynamically reflected in the learning content.
[0056]
The content editing unit 44 uses the data stored in the content database 46 to edit the content according to the rating result by the utterance rating engine 33.
In addition, the content editing unit 44 includes a classmate adding unit (classmate adding means) 44a. The classmate adding unit 44a makes a character who simulates another learner appear in the content based on the data stored in the user database 45.
[0057]
The content database 46 stores teaching material content data supplied from a content supplier. FIG. 2B is an explanatory diagram showing an example of the data structure of data stored in the content database 46. FIG. 2B is an example, and the content database 46 is not limited to this.
[0058]
Specifically, as shown in FIG. 2B, an utterance example sentence 62, a question 63, and an utterance sample 64 are associated and registered in the content database 46 using the content ID 61 as a key. The content ID 61 is a character string for identifying a content page. The utterance example sentence 62 is a character string displayed as an utterance example sentence. The problem 63 is a character string displayed as a problem. The utterance sample 64 is audio data that is an utterance sample, and the audio data having the same content is stored in different data formats (for example, 16-bit to 16-khz PCM) so that the audio data can be selected according to the playback function of the mobile device 100. Yes.
[0059]
As shown in FIG. 1, the mobile device 100 includes a voice call module 101, a data communication module 102, a virtual machine 103, and a web browser 105. Moreover, the mobile device 100 includes at least a display panel, operation keys, a microphone, and a speaker (not shown) as a mobile phone user interface. The virtual machine 103 presents content by executing the client program 104 received from the Web server 20. The web browser 105 displays content according to the HTML file received from the web server 20. Only one of the virtual machine 103 and the web browser 105 may be provided.
[0060]
In the present embodiment, mobile device 100 will be described as a mobile phone. However, mobile device 100 can use any terminal device as long as it has a voice call function and an application operating environment. The pronunciation practice support system 1 is suitable when a mobile device 100 is used as a terminal device in which processes such as a voice call function and a data display / browsing function, such as a mobile phone, overlap.
[0061]
Subsequently, the operation of the pronunciation practice support system 1 will be described.
[0062]
First, the attendance selection process will be described with reference to FIG. FIG. 3 is an explanatory diagram showing an example of a screen displayed in the attendance selection process.
[0063]
When the learner first accesses the pronunciation practice support system 1 from the mobile device 100, the mobile device 100 displays the screen W11 according to the HTML file received from the Web server unit 22. When the learner selects “English conversation classroom” on the screen W11, the screen W12 is displayed. Next, when the learner selects “business conversation course” on the screen W12, the screen W13 is displayed. Next, when the learner selects one of the lecturers on the screen W13, the screen W14 is displayed. On the screen W14, a hot spot “Let's listen to the live voice” for reproducing the voice of the instructor is displayed. When the learner selects a hot spot “select this course” on the screen W14, the attendance selection process ends.
[0064]
When the attendance selection process ends, the process proceeds to a learner data registration process. In the learner data registration process, a setting screen (not shown) is displayed on the mobile device 100 to allow the learner to input a telephone number 52, a nickname, an e-mail address, and the like. In the learner data registration process, the audio file 53 for authenticating the learner is registered.
[0065]
When the attendance selection process and the learner data registration process are completed, the set data is transmitted from the Web server unit 22 to the user management unit 43. At this time, the user management unit 43 issues a user ID 51 to the learner. The set data is stored in the user database 45 in association with the user ID 51.
[0066]
Next, a learner authentication process by utterance in the pronunciation practice support system 1 will be described with reference to FIGS. 4 to 6. FIG. 4 is a flowchart showing the flow of the learner authentication process by utterance. FIG. 5 is an explanatory diagram showing the registration process of the audio file 53 performed at the first access. FIG. 6 is an explanatory diagram showing a learner authentication process using the audio file 53 performed at the second and subsequent accesses.
[0067]
As shown in FIG. 4, the user authenticating unit 42 collects utterance data patterns by causing a learner to read a task sentence in English at the first access. Specifically, as shown in FIG. 5, first, a short task sentence including the learner's nickname “My name is Jimmy” is displayed (screen W21). After the learner presses the “speak” button on this screen W 21, the user management unit 43 stores the voice data of the learner who has read the task sentence as the voice file 53 in the user database 45. When saving of the audio file 53 is completed, learning content is started (screen W22).
[0068]
For example, a phrase composed of about 5 words of a language to be learned is appropriate as an assignment sentence used for authentication. The content of the phrase can be selected as appropriate. In addition, the user authentication unit 42 may evaluate, by the utterance rating engine 33, whether or not the voice data input by the learner is appropriate as the utterance data pattern for authentication before determining the voice file 53.
[0069]
Then, as shown in FIG. 4, at the second and subsequent accesses, the user authentication unit 42 first acquires the user ID 51 input by the learner to the mobile device 100 (S11). Next, an authentication task sentence is displayed on the mobile device 100 (screen W31), and voice data read by the learner is acquired (S12). Next, this voice data and the voice file 53 associated with the user ID 51 and stored in the user database 45 are collated by the utterance rating engine 33 by matching the data pattern (S13). If the utterance rating engine 33 succeeds in collation (YES in S14), the user authentication unit 42 authenticates the learner (S15) and starts the learning content (screen W32). On the other hand, if the utterance rating engine 33 fails to collate (NO in S14), the user authentication unit 42 does not authenticate the learner (S16) and displays the end screen W33.
[0070]
Thus, the pronunciation practice support system 1 can authenticate the learner by utterance. This allows the learner to utter and authenticate a task sentence in a foreign language to be learned, so that the learner feels it is part of the learning and there is no unnaturalness.
[0071]
Next, the flow of learning content will be described with reference to FIGS. FIG. 7 is an explanatory diagram showing a basic flow of learning content. FIG. 8 is an explanatory diagram showing the flow of conversation events with classmates that occur in learning content.
FIG. 9 is an explanatory diagram of an avatar (incarnation) displayed as learning content.
[0072]
In FIGS. 7, 8, and 10 (described later), the learner himself is “Jimmy (male)” and the classmate is “Rolly (female)”. Instructors, learners, and classmates are displayed with their respective avatars (FIG. 9).
[0073]
As shown in FIG. 7, in the learning content, (1) progress display (screen W41), (2) question sentence and option display (screen W42), (3) answer result display (screen W43), ( 4) The cycle of displaying the obtained number of points (screen W44) is repeated for each question. Therefore, if a lesson consists of 5 questions, this cycle is repeated 5 times.
[0074]
Each screen will be described in detail. On the screen W41, a course name, a lesson name that can be learned, a next step name, and the number of points obtained so far are displayed.
[0075]
On the screen W42, question sentences and answer options are displayed. After the learner presses the “answer” button and reads a sentence of an option to be selected, the voice data is transmitted from the mobile device 100 to the pronunciation practice support system 1. Instead of selection by voice input, selection by a key, a button, or the like may be possible.
[0076]
The screen W43 displays the result of the speech rating engine 33 rating the voice data input by the learner. On the screen W43, the fact that “Jimmy” indicating the learner is the correct answer is displayed together with the obtained number of points.
[0077]
On the screen W44, the points obtained in this step are displayed. When the learner presses the “confirm” button on this screen, the screen shifts to the first screen (corresponding to W42) of the next question.
[0078]
The contents of the screens W41 to W44 are determined by the content editing unit 44 according to the answer of the learner, whether the answer is correct, the time until the answer, and the like. Create based on. For example, an avatar with an expression corresponding to the current number of points is displayed. If the number of points obtained in one step is small, the same step is repeated without proceeding to the next step. At that time, “Avatar with troubled expression” is displayed.
[0079]
Here, as shown in the screen W43 in FIG. 7, the classmate answers the question given by the instructor together with the learner. And even if the learner answers correctly, if the answer is slower than the classmate, there are few points. As a result, the learner can compete with the classmate for faster answering, so that the learner can feel the atmosphere of the English conversation classroom.
[0080]
This classmate is created virtually by the classmate adding unit 44a based on the data of other learners selected from the user database 45 and added to the content. Specifically, the classmate adding unit 44a generates classmates using profile data such as the sex, nickname, and correct answer rate of other learners. Then, the correct answer rate and answer time of the classmate are adjusted in accordance with the correct answer rate and accuracy parameter 57 of the learner. That is, when the level of the learner is high, classmates showing correct answers are presented in a short time. Note that the number of classmates to appear may be one or more.
[0081]
Alternatively, the learner's answer and answer time are recorded in the user database 45, and based on the data, the class mate adding unit 44a is configured to reproduce the answer that the other learner has actually made in the past. You can also present your mate. In this case, the English conversation class can be more realistic.
[0082]
Next, a conversation event with a classmate that occurs in the learning content will be described with reference to FIG. A conversation event is an event in which a classmate suddenly speaks to a learner in the middle of learning content. Note that the conversation event may be incorporated in the learning content in advance, or may be randomly generated by the content editing unit 44. Further, other learners who are using the learning service at that time may be selected as classmates to appear.
[0083]
For example, when the learner has set the level of English conversation on the screen W51, the screen suddenly switches to the screen W52, and the classmate speaks to the learner. When the learner clicks the instructor avatar on the screen W52, advice from the instructor is displayed (screen W53). On the screen W53, the response text to the classmate is displayed, and if the learner presses the “Speak” button, the utterance of the response can be input. If the learner presses the “Listen” button, the instructor utters the text. Is played. The instructor's voice can be reproduced repeatedly.
[0084]
Then, as a result of rating the voice data input by the learner by the utterance rating engine 33, a screen W54 to which the classmate responds is displayed if appropriate. In addition, on the screen W54, a “Friendship Degree” point given according to the learner's response is displayed. If the intimacy becomes equal to or higher than a predetermined value, an additional service may be provided, such as enabling messages to be exchanged with other learners who have become classmate models. Then, after the conversation event ends, the original screen 51 is displayed.
[0085]
By generating such a conversation event, the atmosphere of the English conversation classroom can be made more realistic, and learning can be prevented from becoming monotonous.
[0086]
Next, a process for changing the strictness of the rating by the utterance rating engine 33 will be described with reference to FIG. FIG. 10 is an explanatory diagram showing a process of changing the strictness of the rating by the utterance rating engine 33.
[0087]
The utterance rating engine 33 can adjust the strictness of the rating based on the accuracy parameter 57 stored in the user database 45. The accuracy parameter 57 is automatically changed by the accuracy parameter changing unit 43a based on voice data input by the learner according to the learning content.
[0088]
Specifically, voice data input by the learner at the mobile device 100 in accordance with the learning content (screen W61) is input to the utterance rating engine 33 via the line call control device 10 ((1), (2) ( Audio data acquisition step)). At this time, the utterance rating engine 33 inquires the user management section 43 about the accuracy parameter 57 of the learner ((3)) and acquires it ((4)).
[0089]
Next, the utterance rating engine 33 evaluates the appropriateness of pronunciation included in the audio data with strictness according to the accuracy parameter 57 (pronunciation rating step), and the rating result is sent to the user management unit 43 and the content editing unit 44. Transmit (5). Thereafter, the content editing unit 44 generates content reflecting the evaluation result (content editing step), and transmits this content (screen W62) to the mobile device 100 via the Web server 20 ((6) (content presentation step). )).
[0090]
On the other hand, in the user management unit 43, the evaluation result is recorded in the log 56 (FIG. 2A), and the accuracy parameter changing unit 43a records the evaluation result (for example, the number of incorrect / correct answers has reached a predetermined number of times). Based on the above, the accuracy parameter 57 is changed. Note that the user management unit 43 stores the log 56 and the accuracy parameter 57 in the user database 45 in association with the user ID 51 of the learner. In FIG. 10, the accuracy parameter 57 is set in five stages, but the number of stages can be arbitrarily selected.
[0091]
Thereby, according to the utterance ability and proficiency level of the learner, the strictness of the rating can be automatically changed in the pronunciation practice support system 1 while the learning content is in progress. That is, it does not cause the learner to set the strictness of the rating, nor does it make the learner aware of the setting of the strictness of the rating. Therefore, as in the English conversation school, it is possible to provide an efficient learning service by changing the content of the teaching material flexibly and comfortably according to the state of the learner.
[0092]
Here, the details of the utterance rating engine 33 will be described. The technique implemented in the utterance rating engine 33 is proposed by the inventor of the present invention (Non-Patent Document 4).
[0093]
One factor that makes it difficult for Japanese to learn English is the difference in prosodic operation between Japanese and English. As one method for an English learner (hereinafter referred to as “learner”) to learn the prosodic operation of an English native speaker (hereinafter referred to as “native speaker”), there is a method of uttering by imitating the prosodic operation of the native speaker. In order to support such learning with a computer and automatically evaluate the degree of similarity, it is necessary to appropriately associate two utterances to be compared with each other.
[0094]
The utterance rating engine 33 performs automatic labeling using English phonemes and Japanese phonemes, including phonemes that can be considered as Japanese utterances, on the learner's voice and associates them. For example, automatic labeling is performed in consideration of utterance error patterns in Japanese utterances, such as permitting not only / th / in English but also Japanese / z / as phoneme notation of “th” in “the”. . For the Japanese phoneme model, 43 phonemes provided by the “Development of Japanese dictation basic software” project (http://winnie.kuis.kyoto-u.ac.jp/diction/) can be used. As the English phoneme model, for example, 46 phonemes created using HTK (The HTK Book (Version 2.1)) can be used.
[0095]
FIG. 11 is an explanatory diagram showing an example of association by automatic labeling between Japanese phonemes and English phonemes in the utterance rating engine 33. As shown in FIG. 11, based on the result of manual labeling, a section (frame) of learner speech to be dealt with is determined based on the speech of the native speaker, and the result and the speech rating engine 33 automatically When the ratio of the corresponding frames of learner's speech determined in a determined manner was displaced by 100 ms or more, it was 9.40 (%). This is extremely small as compared with the correspondence deviation by the conventional association method. That is, according to the automatic labeling method of English and Japanese phonemes, two utterances can be accurately associated.
[0096]
In the conventional speech rating engine, (1) fundamental frequency pattern, (2) spectrum information, and (3) automatic labeling of only English phonemes are used to correlate native speaker utterances with learner utterances at the phoneme level. I was doing a grade. However, in the case of these methods, a situation in which words in a phrase are not correctly associated and deviation occurs frequently occurs. Specifically, in the conventional utterance rating engine, when “an apple” is spoken for the expected utterance of “apple”, “an apple” is associated with “apple” and correct rating cannot be performed. Occurs at a considerable frequency. That is, if the user utters “an apple” in response to “apple”, it is handled as an unacceptable case (exception case) in the system design, or the user is notified as an incorrect utterance. It becomes.
[0097]
On the other hand, in the utterance rating engine 33, associating by labeling using both Japanese phonemes and English phonemes as described above, the shift in correspondence is remarkably reduced as compared with the conventional method. Yes. Specifically, even if the user speaks “an apple” to “apple”, “an” is correctly associated with “an” and “apple” is correctly associated with “apple”. It is done. As a result, the pronunciation training support system 1 using the utterance rating engine 33 can notify the learner that the “apple” utterance is correct.
[0098]
When using as a mobile device 100 a terminal device in which a voice call function such as a cellular phone and a data display / browsing function (Web browser, etc.) are separate processes, a series of services of voice utterance → result notification It is desirable that the flow be completed once. That is, when the conventional utterance rating engine as described above is used, when “an apple” is uttered with respect to the “apple” rating premise, it is determined that the rating error or inappropriate utterance, This forces the learner to retry the flow. This has a large adverse effect on learning efficiency and usability in learning services such as English conversation, and is unacceptable in terms of quality. This problem does not occur in a personal computer or the like in which the above two functions are integrated on the interface.
[0099]
As described above, the utterance rating engine 33 can appropriately associate a word and a phrase by labeling using both English phonemes and Japanese phonemes. Can be rated. Therefore, the pronunciation practice support system 1 reduces the learning efficiency and usability even when the mobile device 100 uses a terminal device in which processes such as a voice call function such as a mobile phone and a data display / view function overlap. It is possible to provide a service without causing it to occur.
[0100]
Further, the utterance rating engine 33 determines the accuracy of the learner's utterance based on the degree of pattern matching (degree of matching) between the utterance data of the learner and the data of the native language speaker for rating the utterance. Assess. Thereby, the degree of utterance can be classified using an expression such as a score distribution.
[0101]
For example, when the degree of matching is expressed by 0 to 100 points, the strictness of evaluation is naturally different between a case where 80 points or more are accepted as utterances and a case where 60 points or more are accepted as utterances. And the degree of utterance is very dependent on the individual ability of the learner, and statically prescribing the judgment of how much the utterance is accepted is a learning service that uses this judgment. It ignores the obvious individual differences that exist between the two.
[0102]
The pronunciation practice support system 1 dynamically collects the tendency of the score distribution obtained as a result of pattern matching from the learner's past history and the like, and judges the ability of the learner from the data every time it is evaluated. Then, the distribution points are moved up and down to the extent that the utterance is acceptable. As a result, it becomes possible to switch the strictness of the rating.
[0103]
Here, the accuracy parameter represents a set of utterances that can be accepted from what level of pattern matching rate. The accuracy parameter can be set, for example, in five stages from level 5 at which evaluation is performed most strictly to level 1 at which evaluation is performed most gently.
And even if the acceptance rating set in the default state at the time of starting the service is level 5 or higher, if the learner's subsequent rating result distribution is concentrated in the level of 3-4, the accuracy parameter changing unit 43a Change the acceptance grade set to level 3 or level 4 or higher. Thereby, the utterance rating according to the current utterance tendency of the learner becomes possible. The dense distribution of the learner's evaluation results is managed by a log 56 (FIG. 2A) unique to the learner.
[0104]
As described above, according to the pronunciation practice support system 1, learning is performed in a mobile phone environment that is highly portable and portable, is always connected to a network, and has a high degree of spread and user operation skills on average. Service can be provided. Moreover, since the learning can be customized for each learner, a learning service according to the ability of the learner can be efficiently provided. Therefore, it is possible to provide a learning service that is easy and can be expected to have a high learning effect and continuity.
[0105]
According to the pronunciation practice support system 1, the pronunciation and prosody are objectively assessed by the utterance assessment engine, and the assessment result is linked to the content, so that “speaking English” is practiced. In addition, since it is possible to extract conversations that are frequently used in actual practice, it becomes practice to “listen to English” and “read English”. By solving the problem, it becomes a practice of "writing English" in a pseudo manner. Moreover, a learning effect can be improved by using an avatar for a lecturer and a classmate.
[0106]
According to the pronunciation practice support system 1, the learner can listen to (1) native speech, understand the situation, and learn the utterance, and (2) imagine a real scene with a mobile phone. You can speak, (3) you can receive a rating of utterance, (4) you can immediately check the results of speaking on the mobile phone screen.
[0107]
According to the pronunciation practice support system 1, it is possible to realize a learning service in accordance with the following service concept. In other words, it is possible to realize a “communicative approach” with highly practical learning content that simulates English conversation communication realistically and always assumes real communication. In addition, you can realize an “edutainment approach” that allows you to experience useful English conversation while enjoying various communication events with instructors and classmates. A “task-based approach” that balances learning basic elements into conversational tasks can be realized.
[0108]
Here, each device (the line call control device 10, the Web server 20, the utterance rating server 30, and the database server 40) constituting the pronunciation practice support system 1 can be configured based on a general-purpose computer such as a workstation. In addition, the mobile device 100 can be configured based on a general-purpose computer including a mobile phone and a PDA (Personal Digital Assistant).
[0109]
That is, each device and mobile device 100 constituting the pronunciation practice support system 1 includes a CPU (central processing unit) that executes instructions of a program that realizes each function, a ROM (read only memory) that stores boot logic, and the above RAM (Random Access Memory) for developing programs, storage devices (recording media) such as hard disks for storing the programs and various databases, input devices such as keyboards and mice, output devices such as monitors, speakers, and printers, and external networks The network connection device connected to is connected by an internal bus.
[0110]
The mobile device 100 only needs to have a standard Internet browsing function and can be connected to the Web server 20 via the network in order to present the content acquired from the pronunciation training support system 1.
[0111]
FIG. 12 is an explanatory diagram illustrating a configuration of the mobile device 100 including the virtual machine 103. The virtual machine 103 is an application execution environment that executes a client program 104 (for example, a Java (registered trademark) program) acquired from the Web server 20. As described above, when the content is presented by executing the client program 104 in the virtual machine 103, the client program 104 is a program, and therefore, processing by an event, input, or the like can be executed in the mobile device 100. Therefore, it is possible to implement various behaviors according to the content.
[0112]
FIG. 13 is an explanatory diagram showing a configuration of the mobile device 100 including the web browser 105. The web browser 105 is a kind of native application, and displays on the screen of the mobile device 100 documents and data for a document structure described in a markup language such as HTML and HTML acquired from the web server 20. As described above, when the web browser 105 presents content according to an HTML file or the like, since the web browser 105 is installed in most mobile devices, the pronunciation training support system 1 can be used from many mobile devices.
[0113]
Finally, an object of the present invention is to provide a recording medium in which a program code (execution format program, intermediate code program, source program) of a pronunciation practice support program, which is software that realizes the above-described functions, is recorded in a computer-readable manner. Alternatively, this can also be achieved by supplying to the apparatus and reading and executing the program code recorded on the recording medium by the computer (or CPU, MPU, DSP) of the system or apparatus.
[0114]
Specifically, each function block included in the line call control device 10, the Web server 20, the utterance rating server 30, and the database server 40 is configured such that a predetermined program stored in a memory (not shown) is stored in the microprocessor in each device. It is realized by executing.
[0115]
The recording medium for supplying the program code can be configured to be separable from the system or apparatus. The recording medium may be a medium that is fixedly supported so that the program code can be supplied. Even if the recording medium is attached to the system or apparatus so that the recorded program code can be directly read by the computer, the recording medium can be connected via the program reading apparatus connected to the system or apparatus as an external storage device. It may be mounted so that it can be read.
[0116]
For example, as the recording medium, a disk including a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy (registered trademark) disk / hard disk, and an optical disk such as a CD-ROM / MO / MD / DVD / CD-R. Card system such as IC card, IC card (including memory card) / optical card, or semiconductor memory system such as mask ROM / EPROM / EEPROM / flash ROM.
[0117]
Further, the program code may be recorded so that the computer can read out from the recording medium and directly execute it, or after being transferred from the recording medium to the program storage area of the main memory, the computer can read out and execute it from the main memory. It may be recorded as follows.
[0118]
Furthermore, the system or apparatus may be configured to be connectable to a communication network, and the program code may be supplied via the communication network. The communication network is not particularly limited. Specifically, the Internet, intranet, extranet, LAN, ISDN, VAN, CATV communication network, virtual private network, telephone line network, mobile communication A network, a satellite communication network, etc. can be used. In addition, the transmission medium constituting the communication network is not particularly limited, and specifically, it is an infrared ray such as IrDA or a remote control even in a wired manner such as IEEE 1394, USB, power line carrier, cable TV line, telephone line, ADSL line or the like. , Bluetooth, 802.11 wireless, HDR, mobile phone network, satellite line, terrestrial digital network, and the like. The present invention can also be realized in the form of a carrier wave or a data signal sequence in which the program code is embodied by electronic transmission.
[0119]
The functions described above are not only realized by executing the program code read out by the computer, but based on an instruction of the program code, an OS or the like operating on the computer partially or entirely in actual processing. It is also realized by performing.
[0120]
Furthermore, the function described above is obtained by writing the program code read from the recording medium into a memory provided in a function expansion board attached to the computer or a function expansion unit connected to the computer, and then the program code. Based on the instruction, the CPU or the like provided in the function expansion board or function expansion unit also implements part or all of the actual processing.
[0121]
【The invention's effect】
As described above, the pronunciation practice support system according to the present invention is a pronunciation practice support system that is communicably connected to a learner's terminal device, and obtains voice data that is acquired by the learner from the terminal device. Means, a pronunciation rating means for rating the learner's pronunciation included in the voice data acquired by the voice data acquisition means, a content editing means for editing content according to the rating result by the pronunciation rating means, and the above Content presenting means for presenting the content edited by the content editing means to the terminal device.
[0122]
The pronunciation practice support method of the present invention is a pronunciation practice support method by a pronunciation practice support system that is communicably connected to a learner's terminal device, and is a voice that acquires voice data input from the terminal device by the learner. A data acquisition step, a pronunciation rating step for rating a learner's pronunciation included in the voice data acquired in the voice data acquisition step, and a content editing step for editing content according to the rating result in the pronunciation rating step; A content presentation step of presenting the content edited in the content editing step on the terminal device.
[0123]
Therefore, pronunciation practice content such as English conversation can be supplied in a mobile environment such as a mobile phone. Therefore, there is an effect that it is possible to meet the demand of the learner who wants to easily use the “speaking” practice.
[0124]
Further, since the content to be presented on the terminal device is transmitted from the pronunciation practice support system each time, the content can be easily changed as compared with a stand-alone device such as a personal computer.
[0125]
Moreover, since the content to be presented on the terminal device is edited according to the rating result by the pronunciation rating means, it is possible to present the content with appropriate contents according to the situation such as the learning progress and proficiency of the learner. Therefore, it is possible to provide a learning service that can dynamically reflect the situation of each learner on the learning content.
[0126]
Furthermore, the pronunciation practice support system of the present invention is such that the pronunciation rating means can change the strictness of the rating according to the accuracy parameter, and the accuracy of the accuracy according to the history of the rating results by the pronunciation rating means. An accuracy parameter changing unit that changes a parameter, and an accuracy parameter holding unit that holds the accuracy parameter set by the accuracy parameter changing unit for each learner.
[0127]
Therefore, the strictness of the rating can be changed efficiently according to the situation such as the learning progress and proficiency of the learner. Therefore, the system can automatically reflect different situations for each learner on the learning content. Therefore, it is possible to provide a service that can flexibly customize the learning content according to the learner like an English conversation school.
[0128]
Furthermore, the pronunciation practice support system of the present invention is configured to include learner authentication means for authenticating a learner based on the voice data acquired by the voice data acquisition means.
[0129]
Therefore, there is an effect that the service by the pronunciation practice support system can be used without a sense of resistance without making the learner conscious of being authenticated.
[0130]
The pronunciation practice support system of the present invention further includes learner data storage means for storing learner data for each learner, and the content editing means is stored in the data stored in the learner data storage means. Based on this, it is configured to include classmate adding means for causing a character imitating another learner to appear in the content.
[0131]
Therefore, other learners can appear in the learner's content. Thereby, there exists an effect that the atmosphere of a classroom can be provided to content. In addition, since other learners appearing in the content are based on the actual learner data, the learner can feel the reality.
[0132]
Furthermore, in the pronunciation practice support system of the present invention, the pronunciation rating means associates by labeling using both a learner's native language phoneme and a learned foreign language phoneme.
[0133]
Therefore, the pronunciation practice support system further improves the learning efficiency and usability even when using a terminal device with a process overlapping between a voice call function such as a mobile phone and a data display / view function as a mobile terminal. There is an effect that the service can be provided without lowering.
[0134]
The pronunciation training support program of the present invention is a computer program that causes a computer to function as each of the above-described means.
[0135]
Therefore, it is possible to realize the pronunciation practice support system by realizing each means of the pronunciation practice support system with a computer.
[0136]
A computer-readable recording medium on which the pronunciation practice support program of the present invention is recorded is a computer-readable recording medium on which a pronunciation practice support program for operating the pronunciation practice support system is realized by causing the above-described means to be realized by a computer. It is a recording medium.
[0137]
Therefore, the pronunciation practice support system can be realized on the computer by the pronunciation practice support program read from the recording medium.
[Brief description of the drawings]
FIG. 1 is a functional block diagram showing an outline of a configuration of a pronunciation training support system according to an embodiment of the present invention.
2 is an explanatory diagram showing a data structure of data used by the pronunciation practice support system shown in FIG. 1. FIG. 2 (a) is an example of a data structure of data stored in a user database, FIG. b) shows an example of the data structure of data stored in the content database.
FIG. 3 is an explanatory diagram showing an example of a screen displayed in the attendance selection process in the mobile device shown in FIG. 1;
4 is a flowchart showing a flow of user authentication processing by utterance in the pronunciation practice support system shown in FIG. 1; FIG.
FIG. 5 is an explanatory diagram showing audio file registration processing performed at the first access in the pronunciation practice support system shown in FIG. 1;
6 is an explanatory diagram showing a learner authentication process using an audio file that is performed at the second and subsequent accesses in the pronunciation practice support system shown in FIG. 1; FIG.
7 is an explanatory diagram showing a basic flow of learning content presented to the mobile device shown in FIG. 1; FIG.
8 is an explanatory diagram showing the flow of a conversation event with classmates that occurs in learning content presented to the mobile device shown in FIG. 1. FIG.
FIG. 9 is an explanatory diagram of an avatar displayed on the mobile device shown in FIG. 1;
FIG. 10 is an explanatory diagram showing a process of changing the strictness of the rating by the utterance rating engine in the pronunciation practice support system shown in FIG. 1;
FIG. 11 is an explanatory diagram showing an example of association by automatic labeling of Japanese phonemes and English phonemes in the utterance rating engine of the pronunciation practice support system shown in FIG. 1;
12 is an explanatory diagram showing a configuration including a virtual machine of the mobile device shown in FIG.
13 is an explanatory diagram showing a configuration including a Web browser of the mobile device shown in FIG. 1. FIG.
[Explanation of symbols]
1 Pronunciation practice support system
10 Line call control device (voice data acquisition means)
20 Web server (content presentation means)
33 Speech rating engine (pronouncement rating means)
42 User authentication unit (learner authentication means)
43a Accuracy parameter changing unit (Accuracy parameter changing means)
44 Content editing section (content editing means)
44a Classmate addition part (Classmate addition means)
45 User database (learner data storage means, accuracy parameter holding means)
57 Accuracy parameters
100 Mobile equipment (terminal equipment)

Claims (8)

A pronunciation practice support system communicably connected to a learner's terminal device,
Voice data acquisition means for acquiring voice data input by the learner from the terminal device; pronunciation rating means for rating the pronunciation of the learner included in the voice data acquired by the voice data acquisition means;
Content editing means for editing content according to the rating result by the pronunciation rating means;
A pronunciation practice support system comprising: content presenting means for presenting the content edited by the content editing means on the terminal device. The pronunciation rating means can change the strictness of the rating according to the accuracy parameter, and
Accuracy parameter changing means for changing the accuracy parameter according to the history of the rating results by the pronunciation rating means;
The pronunciation practice support system according to claim 1, further comprising accuracy parameter holding means for holding the accuracy parameter set by the accuracy parameter changing means for each learner. The pronunciation practice support system according to claim 1, further comprising learner authentication means for authenticating a learner based on the voice data acquired by the voice data acquisition means. Characters comprising learner data storage means for storing learner data for each learner, and the content editing means imitating other learners based on the data stored in the learner data storage means The pronunciation practice support system according to any one of claims 1 to 3, further comprising classmate adding means for causing the content to appear in the content. 5. The phonetic rating means associating by labeling using both a phoneme of a learner's native language and a phoneme of a foreign language to be learned. Pronunciation support system described in 1. A pronunciation practice support method by a pronunciation practice support system that is communicably connected to a learner's terminal device,
An audio data acquisition step of acquiring audio data input by the learner from the terminal device;
A pronunciation rating step for rating the pronunciation of the learner included in the voice data acquired in the voice data acquisition step;
A content editing step for editing the content according to the rating result in the pronunciation rating step;
A pronunciation presentation support method, comprising: a content presentation step of presenting the content edited in the content editing step on the terminal device. A pronunciation practice support program for operating the pronunciation practice support system according to any one of claims 1 to 5, wherein the pronunciation practice support program causes a computer to function as each of the above means. A computer-readable recording medium on which the pronunciation practice support program according to claim 7 is recorded.

Images