JP7195593B2 - Language learning devices and language learning programs - Google Patents

Description

The present invention relates to a language learning device and a language learning program for a user whose mother tongue is a first language (eg Japanese) to learn a second language (eg English).

In order to master foreign language conversation, a lot of dialogue practice is required. In recent years, language learning systems using computers have been developed as a means to replace dialogue practice through conversation with an English conversation teacher.

In such a language learning system, computers need to recognize non-native speech. For example, in a language learning system for Japanese people to learn English, a computer needs to recognize English with Japanese phonemes (so-called Japanese English). On the other hand, Patent Document 1 discloses a technology that can recognize pronunciations (combinations of phonemes) peculiar to Japanese people based on a "Japanese pronunciation model" recorded by 30 Japanese men and women for 8 hours each. ing.

JP 2012-215645 A

By repeating dialogue practice, the learner will gradually be able to make utterances containing the phonemes of the language to be learned. For example, in the English utterances of Japanese beginners, there are few phonemes (English phonemes) for those whose native language is English (English phonemes), and many Japanese phonemes are included. However, it is difficult for learners whose mother tongue is not English to speak completely with English phonemes. and Japanese phonemes are mixed. It is difficult for the technology described in Patent Document 1 to accurately recognize such a learner's voice by a computer.

SUMMARY OF THE INVENTION An object of the present invention is to provide a language learning device capable of recognizing the voice of a language learner who is still learning the language.

A language learning device according to the present invention is a language learning device for a user whose mother tongue is a first language to learn a second language, and which converts voice uttered by the user into phoneme data. and a character conversion unit that searches for dictionary data in which character data of the second language and phoneme data are associated with each other, and converts the phoneme data converted by the phoneme conversion unit into character data of the second language. wherein the phoneme data convertible by the phoneme conversion unit includes a first phoneme used by a native speaker of the first language and a second phoneme used by a native speaker of the second language, In the dictionary data, one character data includes phoneme data consisting only of the first phoneme, phoneme data consisting only of the second phoneme, and phoneme data including both the first phoneme and the second phoneme. It is characterized by being associated.

Preferably, the language learning device according to the present invention further comprises a search range limiting section for limiting a search range of the dictionary data by the character conversion section.

In the language learning device according to the present invention, the search range limiter may determine the search range according to the user's proficiency level of the first language.

In the language learning device according to the present invention, the search range limiter may determine the search range according to a scene in which the user converses.

In the language learning device according to the present invention, the search range limiter may determine the search range according to the user's utterance tendency.

A language learning program according to the present invention causes a computer to function as each part of any one of the language learning devices described above.

According to the present invention, it is possible to recognize the voice of a language learner who is still learning the language.

1 is a block diagram showing the configuration of a language learning device according to an embodiment of the present invention; FIG. It is a figure for demonstrating the preparation method of a phoneme conversion part. It is a figure which shows some dictionary data. It is a figure which shows some dictionary data. It is a figure which shows some dictionary data.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, this invention is not limited to the following embodiment.

(overall structure)
FIG. 1 is a block diagram showing the configuration of a language learning device 1 according to one embodiment of the present invention. The language learning device 1 can be configured by a smart phone or a general-purpose personal computer. In this embodiment, it is assumed that the language learning device 1 is configured by a smartphone.

The language learning device 1 is used by a user whose mother tongue is a first language to learn a second language. In this embodiment, the first language is Japanese and the second language is English, but the invention is not limited to this.

As shown in FIG. 1, the language learning device 1 mainly includes a storage 2, a control section 3, a display section 4, an input section 5, a microphone 6 and a speaker 7. As shown in FIG.

The storage 2 is a member for storing various programs and data used for arithmetic processing of the language learning device 1, and can be configured by, for example, a flash memory.

The control unit 3 is implemented by the CPU (not shown) of the language learning device 1 reading a language learning program (application) stored in the storage 2 into a main memory (not shown) and executing it. It is a functional block. The language learning program may be installed on the language learning device 1 via a network. Alternatively, the language learning program can be transferred to the language learning device 1 by causing the language learning device 1 to read a computer-readable non-temporary tangible recording medium such as an SD card in which the language learning program is recorded. may be installed.

The control unit 3 mainly includes a phoneme conversion unit 31 , a character conversion unit 32 , a determination unit 33 , a feedback unit 34 and a search range limiter 35 . Functions of these functional blocks will be described later.

The display unit 4 can be composed of, for example, a liquid crystal display. The input unit 5 is a device that receives an operation input from a user, and can be configured by, for example, a touch panel. The microphone 6 and the speaker 7 may be built in the language learning device 1 or may be externally attached.

(control part)
Next, functions of the control unit 3 will be described.

The phoneme conversion unit 31 is a functional block that converts voice uttered by the user into phoneme data. In this embodiment, the voice uttered by the user is converted into an analog voice signal by the microphone 6 , further converted into a digital voice signal by an AD converter (not shown), and input to the phoneme conversion section 31 . The phoneme conversion unit 31 is implemented by a machine-learned model, segments a digital speech signal, and converts each segmented speech signal into phoneme data.

There are some differences in phonemes depending on language variants and phonological theories, but in general, there are 24 phonemes in Japanese (5 vowels + 16 consonants + 3 special phonemes), and 44 phonemes in English (20 vowels + 24 phonemes). consonants). For machine learning, Japanese phonemes (first phoneme) were collected from multiple Japanese without an English accent, and English phonemes (second phoneme) were collected from multiple native English speakers. are collected, and as shown in FIG. 2, a learning data set is created in which each phoneme and a speech waveform are associated with each other. Based on this learning data set, the phoneme conversion unit 31 is created by performing machine learning such as deep learning.

に相当する英語音素（アとエの中間程度の音素）に変換され、日本語のネイティブスピーカーの発音に対しては、「ａ」（ア）に相当する日本語音素に変換される。 As a result, the phoneme conversion unit 31 can convert the voice uttered by the user into phoneme data including Japanese phonemes and English phonemes. That is, the phoneme data that can be converted by the phoneme conversion unit 31 includes Japanese phonemes and English phonemes. Convert to For example, for the "a" part of the word "apple", the phonetic symbol for the pronunciation of a native English speaker is

is converted into an English phoneme (a phoneme intermediate between a and e) corresponding to .

In the following description, for convenience, English phonemes are written in capital letters and Japanese phonemes are written in small letters. For example, for the phoneme "a" in "apple", the English phoneme corresponding to the phoneme between "a" and "e" is written as "A", and the Japanese phoneme is written as "a".

The character conversion unit 32 is a functional block that searches dictionary data D in which English character data and phoneme data are associated with each other, and converts the phoneme data converted by the phoneme conversion unit 31 into English character data. As shown in FIG. 1, the dictionary data D is stored in the storage 2, but may be stored in another device (such as a server) communicatively connected to the language learning device 1. FIG. Unlike normal dictionary data, in the dictionary data D, one character data includes phoneme data consisting only of Japanese phonemes (first phoneme), phoneme data consisting only of English phonemes (second phoneme), and Japanese phonemes. Phoneme data containing both of the phonemes and English phonemes (hereinafter referred to as mixed phoneme data) are associated.

For example, as shown in FIG. 3, the word "cake" includes phoneme data (KEYK) consisting only of English phonemes, phoneme data (ke:ki) consisting only of Japanese phonemes, and mixed phoneme data (KEYku). are associated. The word "rice" is associated with phoneme data (RAIS) consisting only of English phonemes, phoneme data (raisu) consisting only of Japanese phonemes, and mixed phoneme data (raiS).

In addition, in the example shown in FIG. 3, there is only one mixed phoneme data, but it may be plural. In general, the greater the number of letters in a word, the more combinations of Japanese and English phonemes. , the rate of occurrence of English phonemes changes according to the speaker's state of consciousness. In terms of word units, Japanese people's proficiency in speaking English is affected, but patterns in which Japanese phonemes and English phonemes are mixed for each word are limited. Therefore, although it depends on the contents of the included phonemes, the mixed phoneme data does not need to prepare all the theoretical patterns, and it is sufficient to prepare one to several patterns that are easy to pronounce. As a result, the amount of dictionary data D can be reduced without being affected by the number of characters.

By configuring the dictionary data D in this way, the character conversion unit 32 can convert both Japanese phonemes and English phonemes into the phoneme data input from the phoneme conversion unit 31. By selecting appropriate mixed phoneme data, the phoneme data can be converted into character data. Therefore, it is possible to accurately recognize not only the pronunciation of a native English speaker and the pronunciation of a beginner in learning English, but also the speech of a language learner in the process of mastering a mixture of English and Japanese phonemes.

The determination unit 33 is a functional block that determines whether or not the user's utterance content and pronunciation are appropriate. Specifically, the determination unit 33 constructs an English sentence from the character data converted by the character conversion unit 32, and determines whether the content of the user's utterance is appropriate from the constructed sentence. Further, the determination unit 33 determines the proficiency level of pronunciation of the user based on the search processing of the dictionary data D by the character conversion unit 32 . More specifically, for each word uttered by the user, the determination unit 33 selects any of the phoneme data consisting only of Japanese phonemes, the phoneme data consisting only of English phonemes, and the mixed phoneme data. It is determined whether the user's pronunciation is closer to English (native speaker), closer to Japanese (beginner), or intermediate between them (intermediate/advanced) based on whether the character data has been converted into character data.

The feedback unit 34 is a functional block that feeds back to the user when it is determined that the user's utterance is inappropriate. For example, when the user is practicing a dialogue with a virtual character in a language learning application, when the determination unit 33 determines that the user's utterance contains incorrect words or sentences, the feedback unit 34 The erroneous utterance is displayed on the display unit 4 so that the dialogue character behaves as if it does not understand the meaning or recognizes it as a different meaning. Further, when the user's utterance is determined to be inappropriate, the feedback unit 34 deducts the proficiency score indicating the proficiency of the user.

In addition, the determination unit 33 determines whether or not the user's wording is appropriate according to the situation and the conversational partner. You may make it judge. In that case, the feedback unit 34 displays the character to behave as if he/she received a negative impression so that the user can understand that the dialogue is inappropriate.

For example, the following a) and b) are conceivable as utterances when the user wants to confirm with the other party what happened.
a) What's your problem?
b) What's wrong?/What happened?/What's the problem?
Both a) and b) make sense, but a) poses a problem for the other person's mental state, and the rough expression gives the other person a strong sense of displeasure. On the other hand, b) is a normal interaction confirming only the facts. Therefore, when the user utters a), the feedback unit 34 causes the character to behave as if it had a negative impression, and deducts the proficiency score of the user.

In addition, the following c) and d) are conceivable as utterances for conveying that you know Prime Minister Abe.
c) I know Prime Minister Abe.
d) I know of Prime Minister Abe.
Both c) and d) have the same meaning, but c) has a meaning known personally, and d) has a meaning known as information.

Whether or not such dialogue is appropriate depends on the relationship with the other party. Since this is misleading, the feedback unit 34 displays the character so that it behaves as if it received an unexpected impression, and deducts the user's proficiency score.

Further, in the above example, when the user utters a) or c), the feedback unit 34 may cause the character or the like to explain why the user's utterance is inappropriate.

By providing the determination unit 33 and the feedback unit 34 with such functions, it is possible to correct utterances that are not grammatically incorrect but have different nuances or make the other party uncomfortable depending on the situation.

The search range limiter 35 is a functional block that limits the search range of the dictionary data D by the character converter 32 . By limiting the search range of the dictionary data D, the amount of search processing by the character conversion unit 32 can be reduced, so the speed of character conversion can be improved.

In this embodiment, the search range limiter 35 determines the search range of the dictionary data D according to at least one of the user's proficiency in English, the scene in which the user converses, and the user's utterance tendency. In order to realize this function, the search range limiting section 35 includes a proficiency level grasping section 351 , a scene identifying section 352 , and an utterance tendency grasping section 353 .

The proficiency level grasping unit 351 is a functional block that grasps the user's proficiency level of English. The proficiency level grasping unit 351 updates the proficiency level score according to the determination result of the judging unit 33, and grasps the user's English proficiency level based on the proficiency level score.

The scene identification unit 352 is a functional block that identifies a scene that serves as a background for dialogue between the user and the virtual character. In this embodiment, the user can select a desired scene to be the background of the dialogue practice prior to the dialogue practice, and the scene identifying unit 352 selects the background of the dialogue based on the user's selection operation. identify the scene.

Alternatively, the scene identification unit 352 may identify the scene along the flow of dialogue. For example, a scene may be identified based on whether or not the terms used by the user and the character include keywords related to the topic.

The utterance tendency comprehension unit 353 is a functional block that comprehends frequently used terms that are frequently used by the user. In this embodiment, the utterance tendency grasping unit 353 grasps frequently appearing terms based on the record of the user's dialogue practice so far.

The dictionary data D is composed of a plurality of small dictionaries classified based on conversation scenes (conversation scene, situation). 3 to 5 each show an example of a small dictionary. In the small dictionary D1 shown in FIG. 3, terms mainly used in meal scenes are associated with their phonemes. In the small dictionary D2 shown in FIG. 4, terms mainly used in conversations about animals and plants are associated with their phonemes. In the small dictionary D3 shown in FIG. 5, terms used in all kinds of conversations, not limited to specific scenes, are associated with their phonemes (hereinafter also referred to as general-purpose small dictionary D3).

Note that the classification method of the small dictionary in this embodiment is an example, and is not particularly limited. For example, small dictionaries may be classified according to the user's skill level or the user's frequency of use.

The search range limiting unit 35 selects a small dictionary to be used for searching from a plurality of small dictionaries in the dictionary data D, thereby determining the search range of the dictionary data D by the character conversion unit 32 . For example, if the user's proficiency level is low and the scene of conversation is a meal scene, the search range limiting unit 35 selects the small dictionary D1 and the general-purpose small dictionary D3 corresponding to the meal scene, and selects these small dictionaries. Only the dictionaries D1 and D3 are determined as the search range.

As a result, for example, "lice" (plural form of louse (maggot)) in the small dictionary D2 shown in FIG. Even if the phoneme of the user's utterance is closer to "rice" than "rice", the character conversion unit 32 will convert it to "rice". It is rarely used and is rarely used by beginners as it is an advanced word. In general, beginners are not good at pronouncing "r" rather than pronouncing "l". is likely to be converted to "lice", the present embodiment can prevent such unintended conversion.

On the other hand, if the user's proficiency level is high, the search range limiter 35 selects more small dictionaries from the dictionary data D and determines the selected small dictionaries as the search range.

Moreover, when the scene identification unit 352 identifies a scene along the flow of dialogue, the search range restriction unit 35 may select a small dictionary as a search range for each utterance unit. As a result, for example, it is possible to prevent erroneous recognition of utterances that can occur in the same situation but cannot occur in response to a question, thereby improving the accuracy of conversion into intended character data.

Further, when the search range is limited according to the user's utterance tendency, the small dictionaries may be classified according to the frequency of use by the user, and the small dictionaries containing frequently used terms may be preferentially searched.

Also, the dictionary data D may include phoneme data that is not to be converted into character data. For example, by including phoneme data (eetto, nanndakke) corresponding to soliloquy in the user's native language and fillers ("Uh," "What is it?" etc.) in the dictionary data D, the character conversion unit 32 can Even if such phonemes are input, they are not converted into character data. As a result, only the utterance contents that are highly likely to be uttered can be recognized.

(Additional notes)
The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims, and forms obtained by appropriately combining the technical means disclosed in the embodiments are also included in the present invention. Included in the technical scope.

For example, in the above embodiment, all the functions of the control unit 3 are implemented by the CPU of the language learning device 1, but some of the functions of the control unit 3 may be implemented by another device such as a server. good. In this case, the language learning device according to the present invention is provided as a system cooperating with the server.

Further, in the above embodiment, "one character data" associated with phoneme data in the dictionary data is in principle one word, but the present invention is not limited to this. For example, a phrase consisting of a plurality of words such as "have you" in the small dictionary D3 shown in FIG. included in the category.

1 Language learning device 2 Storage 3 Control unit 4 Display unit 5 Input unit 6 Microphone 7 Speaker 31 Phoneme conversion unit 32 Character conversion unit 33 Judging unit 34 Feedback unit 35 Search range limiting unit 351 Proficiency level grasping unit 352 Scene specifying unit 353 Speech Trend grasping unit D Dictionary data D1 Small dictionary D2 Small dictionary D3 Small dictionary

Claims (6)

A language learning device for learning a second language by a user whose native language is a first language, comprising:
a phoneme conversion unit that converts the voice uttered by the user into phoneme data;
a character conversion unit that searches for dictionary data in which character data of the second language and phoneme data are associated with each other, and converts the phoneme data converted by the phoneme conversion unit into character data of the second language. ,
The phoneme data convertible by the phoneme conversion unit includes a first phoneme used by a native speaker of the first language and a second phoneme used by a native speaker of the second language,
In the dictionary data, one character data includes phoneme data consisting only of the first phoneme, phoneme data consisting only of the second phoneme, and phoneme data including both the first phoneme and the second phoneme. A language learning device characterized by being associated. 2. The language learning device according to claim 1, further comprising a search range limiting section for limiting a search range of said dictionary data by said character conversion section. 3. The language learning device according to claim 2, wherein said search range limiting unit determines said search range according to said user's proficiency level of said first language. 4. The language learning device according to claim 2, wherein said search range limiting unit determines said search range according to a scene in which said user converses. 5. The language learning device according to any one of claims 2 to 4, wherein said search range limiting unit determines said search range according to said user's utterance tendency. A language learning program that causes a computer to function as each part of the language learning device according to any one of claims 1 to 5.

Images