JP2023076106A - Foreign language learning support device, foreign language learning support method, and computer program - Google Patents

Abstract

To provide a foreign language learning support device, a foreign language learning support method, and a computer program that allow learners and instructors to multilaterally grasp the learners' ability to speak a foreign language.SOLUTION: A foreign language learning support device includes: a conversion unit that converts audio data into text; an analysis unit that analyzes the text regarding a plurality of evaluation indexes; a data generation unit that generates, for each evaluation index, display data representing the text and analysis results; and a communication unit that transmits the display data generated for each evaluation index to a terminal that displays an image according to the display data. The data generation unit may generate, for each evaluation index, display data indicating places related to the analysis results of the text in the text.SELECTED DRAWING: Figure 1

Description

The present invention relates to a foreign language learning support device, a foreign language learning support method, and a computer program.

The Ministry of Education, Culture, Sports, Science and Technology introduced the evaluation of the four English skills (listening, speaking, reading, and writing) into the course of study. In addition, the overseas expansion of Japanese companies is progressing further. In response to this situation, the demand for speaking English is increasing more and more. In order to meet such demand, the number of businesses providing English conversation services (foreign language learning support services) is increasing. Each business that provides English conversation services provides matching between learners and instructors, online English conversation services, and online proficiency tests (see Non-Patent Document 1) in order to differentiate from other businesses. We are making various attempts such as

“Versant Test”, [online], October 28, 2021, Pearson Japan K.K., [searched on October 28, 2021], Internet <URL: https://www.pearson.com/ en/en/assessment/versant-test.html>

In an online language school, a large amount of voice data is generated according to the utterances of learners (students). However, in online language schools, multifaceted analysis results and quantitative analysis results for speech data are not provided (feedback) to learners.

In order to improve English proficiency, it is essential for learners to reflect on what they have said. However, even if there are services that simply provide learners with the evaluation results of proficiency tests, there are no services that provide learners with detailed reviews that contribute to their English learning.

There are business operators that provide separate services of an English pronunciation analysis service, a morphological analysis and lexical frequency classification service, and an artificial intelligence grammar check service. However, in order for the learner to obtain multifaceted analysis results from the operator, it is necessary for the learner to read the same sentence aloud and input the voice data of the same sentence to the foreign language learning support device for each service. be. In addition, text analysis (sentence conversion) is performed on the input speech data, but processing for converting the speech into text (Speech To Text) is required, which is not very convenient for learners. As described above, multifaceted analysis of the speech data of the same sentence is not performed because it is not convenient for the learner.

Multifaceted analysis used in speaking tests and the like is used for the purpose of measuring a learner's speaking ability. However, weakness analysis and presentation of improvement points for the purpose of improving a learner's ability have not been performed so far. When an instructor (human) performs such multifaceted analysis, it is difficult for the instructor to give negative feedback to the learner for emotional reasons. In addition, since the analysis result by the instructor is qualitative, the learner's satisfaction with the analysis result provided by the instructor is not high. Thus, there is a problem that the learner and the instructor cannot grasp the foreign language speaking ability of the learner from various aspects.

In view of the above circumstances, the present invention provides a foreign language learning support device, a foreign language learning support method, and a computer program that enable learners and instructors to grasp the learner's ability to speak a foreign language from multiple perspectives. It is intended to

One aspect of the present invention is a conversion unit that converts voice data into text, an analysis unit that analyzes the text with respect to a plurality of evaluation indices, and generates display data representing the text and analysis results for each of the evaluation indices. The foreign language learning support device includes a data generation unit and a communication unit that transmits the display data generated for each evaluation index to a terminal that displays an image according to the display data.

One aspect of the present invention is the above-described foreign language learning support device, wherein the data generation unit converts the display data indicating a part related to the analysis result of the text in the text to the evaluation index. generated for each

One aspect of the present invention is the foreign language learning support device described above, wherein the communication unit transmits the display data including a radar chart representing analysis results of the plurality of evaluation indices to the terminal.

One aspect of the present invention is the foreign language learning support device described above, wherein the communication unit transmits the display data including the text displayed in fluorescent color according to the analysis results of the plurality of evaluation indices, Send to the terminal.

One aspect of the present invention is a foreign language learning support method executed by the above foreign language learning support device, comprising: a conversion step of converting voice data into text; and an analysis step of analyzing the text with respect to a plurality of evaluation indices. a data generating step of generating display data representing the text and analysis results for each evaluation index; and a terminal displaying an image based on the display data generated for each evaluation index. and a communication step of transmitting to the foreign language learning support method.

One aspect of the present invention is a computer program for causing a computer to function as the foreign language learning support device.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible for a learner and an instructor to grasp a learner's speaking ability in a foreign language from various aspects.

1 is a diagram showing a configuration example of a foreign language learning support system in an embodiment; FIG. 1 is a diagram showing an overview of a configuration example of a foreign language learning support system in an embodiment; FIG. 1 is a diagram showing an overview of a functional example of a foreign language learning support system in an embodiment; FIG. It is a figure which shows the example of a data table in embodiment. 4 is a sequence diagram showing an operation example of the foreign language learning support system in the embodiment; FIG. 4 is a flowchart showing an operation example of a terminal in the embodiment; 4 is a flow chart showing an operation example of the foreign language learning support device in the embodiment. FIG. 10 is a diagram showing an example of an upload operation screen in the embodiment; FIG. It is a figure which shows the example of the recording operation screen of an utterance in embodiment. It is a figure which shows the example of the recording operation screen of a reading aloud test in embodiment. It is a figure which shows the example of the analysis result screen in embodiment. FIG. 10 is a diagram showing an example of a display screen for syntactic complexity in an embodiment; FIG. 4 is a diagram showing a first example of a display screen regarding lexical complexity in the embodiment; FIG. 10 is a diagram showing a second example of a display screen regarding lexical complexity in the embodiment; FIG. 10 is a diagram showing an example of a display screen for synonyms or synonym suggestions in the embodiment; FIG. 10 is a diagram showing a first example of a display screen regarding accuracy in the embodiment; FIG. 10 is a diagram showing a second example of a display screen regarding accuracy in the embodiment; FIG. 10 is a diagram showing a third example of a display screen regarding accuracy in the embodiment; FIG. 4 is a diagram showing a first example of a display screen regarding fluency in the embodiment; FIG. 10 is a diagram showing a second example of a display screen regarding fluency in the embodiment;

(overview)
A foreign language learning support system acquires voice data of a learner's foreign language in a foreign language lesson or the like using a communication line such as the Internet. In the following, the foreign language is English as an example. A foreign language learning support device (for example, a server device) of a foreign language learning support system stores voice data of English sentences uttered (voice input) by a learner.

A foreign language learning support system uses text conversion, morphological analysis, and the like on speech data to multifacetedly analyze various evaluation indexes (elements) of pronunciation, phonemes, and utterance content. For example, the foreign language learning support system generates a text (sentence) representing the content of the sentence uttered by the learner (speech content). For example, foreign language learning support systems generate phoneme data for spoken sentences by analyzing (eg, grammatical diagnosis) the text. The foreign language learning support system may use a model (learned model) trained using a machine learning technique to perform analysis processing on speech data.

The foreign language learning support device transmits display data (text, graph, etc.) including multifaceted analysis results to the terminal of the learner. The results of multifaceted analysis are, for example, the syntactic complexity of the speech-input sentence (utterance content), the lexical complexity of the speech-input sentence, and the speech-input sentence. and the fluency of sentences input by voice.

Syntactic complexity is, for example, at least one of the number of words per sentence and the number of types of grammatical expressions used in the sentence. The lexical complexity is, for example, at least one of the difficulty of vocabulary included in the voice-inputted sentence and the frequency of use of specific words. The accuracy of a sentence is, for example, the number of grammatically incorrect parts in a sentence input by voice, and the degree of similarity between the pronunciation of an English-speaking speaker (native speaker) and the learner's pronunciation. Fluency is at least one of the number of words contained in a sentence per predetermined time (for example, 1 minute) and the type and frequency (number) of filler words (filler words). is.

The foreign language learning support device transmits multifaceted analysis results such as syntactic complexity, lexical complexity, accuracy and fluency to the terminal of the learner. As a result, multifaceted analysis results are fed back to the learner. The foreign language learning support device may transmit a final proficiency score related to foreign language learning to the terminal of the learner based on each score of the multifaceted analysis results. The foreign language learning support device may present the learner with the skills the learner lacks based on the final proficiency score. The foreign language learning support device may present to the learner a list of lesson history information and changes in chronological evaluation indices (analysis results).

The terminal of the learner displays the contents of the feedback on the display unit of the own terminal based on the display data. The UI/UX (user interface/user experience) of the display data images containing the analysis results is well thought out, so that the learner can easily obtain the multifaceted analysis results for the sentences that the learner himself/herself input by voice. can grasp. The display unit of the terminal may display a radar chart representing the score of each evaluation index. The display of the terminal may display the final proficiency score. The display unit of the terminal may display a list of lesson history information and changes in the evaluation index. The learner and the instructor can grasp the learner's speaking ability multilaterally and quantitatively. It becomes easy for learners and instructors to grasp specific improvement methods and quantitative standards.

Embodiments of the present invention will be described in detail with reference to the drawings.
FIG. 1 is a diagram showing a configuration example of a foreign language learning support system 1 in an embodiment. The foreign language learning support system 1 is a system that supports learning of a foreign language. A foreign language learning support system 1 includes one or more terminals 2 , a communication line 3 , and a foreign language learning support device 4 .

FIG. 2 is a diagram showing an overview of a configuration example of the foreign language learning support system 1 in the embodiment. The foreign language learning support system 1 is realized using a predetermined API. A given API is an application programming interface that follows the constraints of a predetermined architectural style.

FIG. 3 is a diagram showing an overview of a functional example of the foreign language learning support system 1 in the embodiment. The foreign language learning support system 1 is realized, for example, as a cloud system in a public cloud. Also, the foreign language learning support device 4 is implemented as, for example, a virtual machine.

Returning to FIG. 1, the terminal 2 will be described.
The terminal 2 is an information processing terminal such as a smartphone terminal, a tablet terminal, and a notebook computer. The terminal 2 includes a communication section 21 , a storage section 22 , a terminal control section 23 , an operation section 24 , a display section 25 and a microphone 26 . The communication line 3 is, for example, the Internet.

Communication unit 21 executes communication with foreign language learning support device 4 . The communication unit 21 performs real-time communication with the foreign language learning support device 4 based on, for example, the WebRTC (Web Real-Time Communication) standard.

Storage unit 22 stores data received from foreign language learning support device 4 by communication unit 21 . The received data includes, for example, display data representing analysis results. Analysis results are expressed using text, graphs, and the like. Text, graphs, and the like may be embedded in display data in HTML (Hypertext Markup Language) format, for example.

The terminal control unit 23 controls each functional unit of the terminal 2 . The terminal control unit 23 generates an image representing the analysis result based on the display data. The operation unit 24 is an operation device such as a mouse, keyboard, and touch panel. The operation unit 24 receives an operation such as a pressing operation. The operation unit 24 is operated by, for example, a learner (user of the terminal 2) who is taking a foreign language lesson. Note that the operation unit 24 may be operated, for example, by an instructor who gives a foreign language lesson to a learner.

The display unit 25 is a display device such as a liquid crystal display. The display unit 25 displays images based on the image data output from the terminal control unit 23 . The display unit 25 displays, for example, an image representing the analysis result of the sentence input by voice by the learner. Microphone 26 collects sounds around terminal 2 . The collected sounds include, for example, the learner's voice.

Next, the foreign language learning support device 4 will be described.
The foreign language learning support device 4 is an information processing device. The foreign language learning support device 4 includes a server device 41 , an interface 42 and an analysis section 43 . The server device 41 is, for example, a cloud server. The server device 41 includes a communication unit 411 , a storage unit 412 , an authentication processing unit 413 , a conversion unit 414 and a data generation unit 415 . The analysis unit 43 includes a morphological analysis unit 431, a first complexity analysis unit 432, a second complexity analysis unit 433, an accuracy analysis unit 434, and a fluency analysis unit 435 as various plug-ins.

Some or all of the functional units of the foreign language learning support device 4 are stored in a storage unit having a non-volatile recording medium (non-temporary recording medium), such as a CPU (Central Processing Unit). It is implemented as software by executing a program (computer program). The program may be recorded on a computer-readable recording medium. Computer-readable recording media include portable media such as flexible discs, magneto-optical discs, ROM (Read Only Memory), CD-ROM (Compact Disc Read Only Memory), and storage such as hard disks built into computer systems. It is a non-temporary recording medium such as a device. The program may be acquired via a communication line.

Some or all of the functional units of the foreign language learning support device 4 are, for example, LSI (Large Scale Integrated circuit), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), or FPGA (Field Programmable Gate Array). ), etc., may be implemented using hardware including electronic circuitry or circuitry.

The communication unit 411 executes communication with the terminal 2 . The communication unit 21 performs real-time communication with the terminal 2 based on the WebRTC standard, for example.

The storage unit 412 stores computer programs executed by the server device 41 in advance. Storage unit 412 temporarily stores data transmitted from communication unit 411 before transmission of the data. The storage unit 412 stores data received from the terminal 2 by the communication unit 411 . The data received from the terminal 2 is, for example, voice data input to the terminal 2 by the learner.

FIG. 4 is a diagram showing an example of a data table in the embodiment. The storage unit 412 stores, for example, a Json (JavaScript Object Notation) format database. The storage unit 412 stores a user data table 51 as a data table regarding learners. The storage unit 412 stores a lesson data table 52 as a data table relating to lessons. In the lesson data table 52, for example, text (English) converted from voice data input by the learner is registered.

The authentication processing unit 413 executes communication authentication processing for the terminal 2 via the communication unit 411 . The authentication processing unit 413 may perform charging processing for the authenticated terminal 2 via the communication unit 411 .

The conversion unit 414 converts the voice data received from the terminal 2 into text using a known text conversion API (Application Programming Interface).

The conversion unit 414 generates sentences (text) input by the learner into the terminal 2 by voice, using an API for converting voice data to text. This speech-input sentence is a sentence commonly used for a plurality of evaluation indices. As a result, the learner can easily obtain analysis results (multifaceted analysis results) for a plurality of evaluation indices, for example, for sentences input by voice in one recording.

Note that the conversion unit 414 may perform pronunciation diagnosis on the voice data. The conversion unit 414 may acquire pronunciation data of an English-speaking speaker from the storage unit 412 . The conversion unit 414 may perform pronunciation diagnosis based on the degree of similarity between the pronunciation of an English-speaking speaker (native speaker) and the learner's pronunciation. That is, the conversion unit 414 may convert voice data into phonetic symbols.

The interface 42 outputs the text to be analyzed to the analysis unit 43 . The interface 42 acquires the analysis result of the text to be analyzed from the analysis unit 43 . The interface 42 outputs the analysis result of the text to be analyzed to the server device 41 .

The morphological analysis unit 431 executes morphological analysis processing on the text. The morphological analysis unit 431 uses morphological analysis to decompose the text (sentence input by voice) into parts of speech.

The first complexity analysis unit 432 analyzes the sentence input by the learner to the terminal 2 for syntactic complexity among the plurality of evaluation indices. The first complexity analysis unit 432 analyzes (counts) the number of words per sentence, for example, as the analysis of the syntactic complexity of the sentence. The first complexity analysis unit 432 counts the number of periods after counting the number of words. At that time, the first complexity analysis unit 432 counts excluding periods "." in excessively short sentences (eg, Yeah.). The first complexity analysis unit 432 may omit and count sentences of a predetermined length or less (for example, sentences of three words or less).

As the syntactic complexity analysis of the sentence, the first complexity analysis unit 432 may, for example, analyze how many kinds of grammatical expressions are used among a predetermined variety of grammatical expressions. The first complexity analysis unit 432 extracts feature points in the result of the morphological analysis on a word-by-word basis from the text input (uttered) by voice. The first complexity analysis unit 432 converts the speech-input sentence into a part-of-speech arrangement. The first complexity analysis unit 432 acquires the characteristic part-of-speech arrangement data for each grammatical expression from the storage unit 412 . The first complexity analysis unit 432 extracts a grammatical expression having the same pattern as the part-of-speech arrangement in the arrangement data from the sentence input by voice. The first complexity analyzer 432 counts the extracted grammatical expressions.

The second complexity analysis unit 433 analyzes sentences voice-inputted by the learner to the terminal 2 for lexical complexity among the plurality of evaluation indices using a known API for analyzing lexical complexity. .

The second complexity analysis unit 433 analyzes the lexical complexity of the text, for example, the difficulty level of the vocabulary included in the voice input text. A plurality of corpora of, for example, one billion words or more are prepared in advance for various usage scenes in the English-speaking world. While referring to these corpora, the second complexity analysis unit 433 determines how frequently all the words in the text corresponding to the voice data input by the learner are used in the corpus. detected for each range. The second complexity analysis unit 433 sets the usage frequency in the learner's (speaker's) utterance as a ratio for each range, and writes the ratio in the data table. The second complexity analysis unit 433 fluorescently colors (highlights), according to the frequency of use, the indicated portion (for example, the portion of the word whose frequency of use is equal to or greater than a threshold) in the English text converted to text. . This enhances the listability of pointed out points.

The second complexity analysis unit 433 analyzes the lexical complexity of the sentence, for example, the frequency of use of specific words in the text. The second complexity analysis unit 433 classifies the result of the morphological analysis by the morphological analysis unit 431 into content words and non-content words, and extracts only the content words from the result of the morphological analysis. Content words are, for example, adjectives, adverbs, verbs and nouns (other than pronouns).

The accuracy analysis unit 434 analyzes the text input by the learner to the terminal 2 by voice using a known API for analyzing accuracy with respect to the accuracy of the plurality of evaluation indices.

The accuracy analysis unit 434 analyzes whether or not there is a grammatical error in the text (sentence) as an analysis of the accuracy of the sentence. The accuracy analysis unit 434 detects points pointed out regarding various improvements in the text as examples of points related to the analysis result. The accuracy analysis unit 434 excludes improvement points specific to the written language from candidates for pointed out points. For example, the correctness analysis unit 434 eliminates capital letters of the first character and spelling mistakes from the candidate points. The accuracy analysis unit 434 uses the data generation unit 415 and the communication unit 411 to transmit to the terminal 2 grammatical expressions to be improved and proposals for improvement regarding bias in the vocabulary used.

The accuracy analysis unit 434 uses the data generation unit 415 and the communication unit 411 to display the indicated portion in the text on the display unit 25 of the terminal 2 in fluorescent color. The accuracy analysis unit 434 may cause the display unit 25 of the terminal 2 to display a list of indicated points (pointed content). As a result, it is possible to improve the visibility of the indicated locations. The learner can easily grasp the points pointed out in the text.

Accuracy analysis unit 434 may perform pronunciation diagnosis as to whether the pronunciation of the read aloud sentence is correct. The accuracy analysis unit 434 derives the degree of similarity between the pronunciation of the English-speaking speaker (native speaker) and the learner's pronunciation for the same sentence. Pronunciation data of English-speaking speakers are collected in advance.

The fluency analysis unit 435 analyzes sentences voice-inputted by the learner to the terminal 2 for fluency among the plurality of evaluation indices. Fluency analysis unit 435 derives the number of words per predetermined time as an analysis of sentence fluency. In addition, in order not to easily give a high evaluation when the utterance time is short (when the sentence is short), the number of words in the sentence uttered in one minute, for example, is derived. Fluency analysis unit 435 analyzes the type and frequency of use of filler words in the text. The filler is predefined. The fluency analysis unit 435 counts the number of occurrences (number of times of use) of the filler word in the text. The fluency analysis unit 435 uses the data generation unit 415 and the communication unit 411 to display the number of appearances of the filler words on the display unit 25 of the terminal 2 .

The data generation unit 415 (data processing unit) generates text and display data representing analysis results for each evaluation index. The display data is, for example, data in HTML format. The data generation unit 415 causes the display unit 25 of the terminal 2 to display an image based on the display data using the communication unit 411 .

Next, an operation example of the foreign language learning support system 1 will be described.
FIG. 5 is a sequence diagram showing an operation example of the foreign language learning support system 1 in the embodiment. Terminal 2 transmits voice data to foreign language learning support device 4 (step S101). A conversion unit 414 converts the voice data into text. The conversion unit 414 may diagnose the pronunciation of audio data. The conversion unit 414 transmits the text to the terminal 2 (step S102).

Terminal 2 associates the converted text with the accompanying information, and transmits them to foreign language learning support device 4 . This accompanying information is information associated with the text, such as the identification number of the test conducted in the lesson (step S103). The data generation unit 415 associates the converted text with the accompanying information and registers them in the database stored in the storage unit 412 . The data generator 415 requests the interface 42 to analyze the text (step S104).

Interface 42 retrieves analytical information. That is, the interface 42 searches for various plug-ins of the analysis unit 43 . The interface 42 requests the analysis unit 43 to analyze the text (step S105). The analysis unit 43 executes API linkage between software of external databases of various APIs and plug-ins of the analysis unit 43 . Thereby, the analysis unit 43 analyzes the text (step S106).

The external databases of various APIs return text analysis results to the analysis unit 43 (step S107). The analysis unit 43 returns the text analysis result to the interface 42 (step S108). The interface 42 returns the text analysis result to the data generator 415 (step S109).

The data generation unit 415 updates the database by registering the text analysis results in the database in which the text and accompanying information are registered. The data generation unit 415 integrates the analysis results of all evaluation indexes for which analysis has been performed. The data generation unit 415 returns the text analysis result (integrated analysis result) to the terminal 2 using the communication unit 411 (step S110).

FIG. 6 is a flow chart showing an operation example of the terminal 2 in the embodiment. The communication unit 21 executes authentication processing (step S201). The terminal control unit 23 uses the operation signal acquired from the operation unit 24 to indicate whether the operation mode selected by the learner is the test mode (recording mode for reading aloud test) or the recording mode (recording mode for speech). (step S202).

When it is determined that the selected operation mode is the test mode (step S202: test), the terminal control unit 23 acquires an operation signal representing the difficulty level of the test from the operation unit 24 (step S203). The display unit 25 displays the text for reading aloud (step S204). The communication unit 21 transmits the voice data input by the learner who is reading aloud to the foreign language learning support device 4 (step S205). Then, the display unit 25 displays the text of the question in the test (step S206). The communication unit 21 transmits voice data input as an answer to the question to the foreign language learning support device 4 (step S207). Note that the process of step S205 may be included in the process of step S207.

As a result, multifaceted analysis results and quantitative analysis results of the speech data are registered in the database in the foreign language learning support device 4 . Additionally, the text and accompanying information may be registered in the database.

If it is determined that the selected operation mode is the recording mode (step S202: recording), the communication unit 21 transmits the voice data input by the learner to the foreign language learning support device 4 (step S208). ).

The communication unit 21 acquires display data representing the multifaceted analysis result and the quantitative analysis result as a review of learning (step S209). The display unit 25 displays images representing multifaceted analysis results and quantitative analysis results based on the display data (step S210). The terminal control unit 23 determines whether the display form selected by the learner is the summary form or the detailed form based on the operation signal acquired from the operation unit 24 (step S211).

If it is determined that the selected display form is the summary form (step S211: summary), the terminal control unit 23 displays an image based on the display data representing the summary of the history information and the summary of the analysis results on the display unit. 25 (step S212).

When it is determined that the selected display form is the detailed form (Step S211: Details), the terminal control unit 23 controls the foreign language learning support device according to the evaluation index represented by the operation signal acquired from the operation unit 24. Change the display form of the text obtained from 4. For example, when the selected evaluation index is the accuracy of sentences, the terminal control unit 23 causes the display unit 25 to display text in which grammatical errors are indicated in fluorescent color (step S213). The terminal control unit 23 causes the display unit 25 to display the multifaceted analysis result and the quantitative analysis result according to the evaluation index indicated by the operation signal (step S214).

The terminal control unit 23 determines whether or not to end the process based on the operation signal acquired from the operation unit 24 (step S215). If the acquired operation signal indicates continuation (step S215: No), the terminal control unit 23 returns the process to step S211. If the acquired operation signal indicates the end (step S215: Yes), the terminal control unit 23 ends the process.

FIG. 7 is a flow chart showing an operation example of the foreign language learning support device 4 in the embodiment. The authentication processing unit 413 acquires authentication information from the terminal 2 (step S301). The authentication processing unit 413 determines whether or not the authentication is successful based on the authentication information (step S302). If it is determined that the authentication has failed (step S302: No), the authentication processing unit 413 returns the process to step S301.

When it is determined that the authentication has succeeded (step S302: Yes), the communication unit 411 transmits display data representing a predetermined initial screen to the terminal 2 (step S303). The communication unit 411 transmits history information (information on past lessons) stored in the storage unit 412 to the terminal 2 (step S304).

The communication unit 411 acquires the voice data input by voice from the terminal 2 (step S305). The conversion unit 414 converts the voice data into text (step S306). The communication unit 411 transmits the text to the terminal 2 (step S307). The data generator 415 registers the text in the user data table 51 (step S308). The analysis unit 43 analyzes the text commonly used for multiple evaluation indices (step S309). The data generator 415 registers the analysis result in the database (step S310). The communication unit 411 transmits analysis results of a plurality of evaluation indices to the terminal 2 (step S311).

Next, an example of an image based on display data will be described.
FIG. 8 is a diagram showing an example of an upload operation screen in the embodiment. In step S207 illustrated in FIG. 6, the display unit 25 displays the upload operation screen based on the display data. The learner presses the record button image 10 by operating the operation unit 24 . The terminal control unit 23 converts the voice collected from the microphone 26 into voice data. The terminal control unit 23 includes the audio data in the audio file. The learner presses the upload button image 11 by operating the operation unit 24 . The communication unit 21 uploads a voice file including voice data to be analyzed to the foreign language learning support device 4 under the control of the terminal control unit 23 .

FIG. 9 is a diagram showing an example of an utterance recording operation screen in the embodiment. In step S208 illustrated in FIG. 6, when the recording button image 10 illustrated in FIG. . The communication unit 21 uploads the voice data to the foreign language learning support device 4 during the recording process. The display unit 25 displays the text acquired from the foreign language learning support device 4 on the recording operation screen in real time. When the completion button image 12 is pressed on the recording operation screen, the terminal control unit 23 ends the recording process.

FIG. 10 is a diagram showing an example of a recording operation screen for a reading test in the embodiment. In step S<b>204 illustrated in FIG. 6 , when a predetermined record button image is pressed, the display unit 25 displays a recording operation screen for the reading aloud test according to control by the terminal control unit 23 . The communication unit 21 uploads the voice data to the foreign language learning support device 4 during the recording process. The display unit 25 displays the text acquired from the foreign language learning support device 4 on the recording operation screen in real time. When the completion button image 13 is pressed on the recording operation screen of the reading test, the terminal control unit 23 ends the recording process.

FIG. 11 is a diagram showing an example of an analysis result screen (summary screen) in the embodiment. In step S212 illustrated in FIG. 6, an analysis result screen (summary screen) is displayed on the display unit 25. FIG. The analysis result screen illustrated in FIG. 11 includes a radar chart display area 14, a simple result display area 15, a first switching button image 16, a second switching button image 17, a third switching button image 18, and a third switching button image 18. 4 switching button images 19 are included. Radar charts of four types (four items) of evaluation indices (syntactic complexity, lexical complexity, accuracy, and fluency) are displayed in the radar chart display area 14 . Each evaluation index may be divided into two types (two items) of evaluation indexes.

For example, one of the evaluation indices, syntactic complexity, is divided into the number of words per sentence and the number of types of grammatical expressions used in the sentence. For example, lexical complexity, which is one of the evaluation indices, is divided into evaluation indices of difficulty of vocabulary included in a sentence and frequency of use of specific words. For example, accuracy, which is one of the evaluation indices, can be divided into the number of grammatical errors in a sentence and the degree of similarity between the pronunciation of an English-speaking speaker and that of a learner. . For example, accuracy, which is one of the evaluation indexes, is divided into evaluation indexes of the number of words included in a sentence per predetermined time, and the type and frequency of filler words (word whiskers). As a result, in FIG. 11 , radar charts of a total of 8 types (=4×2 types) of evaluation indices are displayed in the radar chart display area 14 as an example.

In the simple result display area 15, a summary of the analysis results is displayed for each evaluation index. When the first switch button image 16 is operated, a display screen regarding syntactic complexity is displayed. When the second switching button image 17 is operated, a display screen regarding lexical complexity is displayed. When the third switching button image 18 is operated, a display screen regarding accuracy is displayed. When the fourth switching button image 19 is operated, a display screen regarding fluency is displayed.

FIG. 12 is a diagram showing an example of a display screen regarding syntactic complexity in the embodiment. The analysis result screen illustrated in FIG. 11 includes a sentence display area 100, an operation button display area 110, and an analysis result display area 120. FIG. In the text display area 100, text common to each analysis result screen is displayed. Here, the portion of the fluorescent color (indicated portion) in the text may differ for each analysis result screen (for each evaluation index). In FIG. 12, as an example, the word "which" is displayed in the text display area 100 in fluorescent color.

In the operation button display area 110, a first operation button image 111, a second operation button image 112, a third operation button image 113, and a fourth operation button image 114 are displayed. When the first operation button image 111 “Complexity” is operated, a display screen regarding syntactic complexity is displayed. When the second operation button image 112 “Lecxial Complexity” is operated, a display screen regarding lexical complexity is displayed. When the third operation button image 113 “Accuracy” is operated, a display screen regarding accuracy is displayed. When the fourth operation button image 114 “Fluency” is operated, a display screen regarding fluency is displayed.

The analysis result display area 120 displays the number of words per sentence and the analysis result of how many types of grammatical expressions are used.

FIG. 13 is a diagram showing a first example of a display screen regarding lexical complexity in the embodiment. In FIG. 13, as an example, "Teacher" and "Schedule" are displayed in fluorescent colors different from other fluorescent colors in the sentence display area 100 according to the difficulty level of these vocabularies. In the analysis result display area 120, the difficulty level of the vocabulary included in the sentence and the type/token rate button image 121 are displayed.

FIG. 14 is a diagram showing a second example of a display screen regarding lexical complexity in the embodiment. When the type-token ratio button image 121 “Type-Token ratio” is operated, the analysis result display area 120 displays the type-token ratio (frequency of use of specific words). The type token rate may be expressed graphically as well as numerically. The analysis result display area 120 may further display a display screen regarding synonyms or synonym suggestions.

FIG. 15 is a diagram showing an example of a display screen regarding synonyms or synonym suggestions in the embodiment. A synonym or synonym suggestion display screen displays synonyms or quasi-synonyms of words in the spoken text as synonyms or synonym suggestions for paraphrasing. The display screen for synonyms or synonym suggestions may also display a definition of the suggested word, "definition." This allows the learner to improve the lexical complexity of the utterance.

FIG. 16 is a diagram showing a first example of a display screen regarding accuracy in the embodiment. In FIG. 16, as an example, "need" and "worse" are displayed in a fluorescent color different from other fluorescent colors in the sentence display area 100 as grammatically incorrect portions. In the analysis result display area 120, an errata list of grammatical errors and pronunciation button images 122 are displayed.

FIG. 17 is a diagram showing a second example of a display screen regarding accuracy in the embodiment. When the pronunciation button image 122 “Pronunciation” is operated, an errata for pronunciation is displayed in the analysis result display area 120 .

FIG. 18 is a diagram showing a third example of a display screen regarding accuracy in the embodiment. When the pronunciation button image 122 “Pronunciation” is operated, the display screen regarding accuracy illustrated in FIG. 18 may be displayed on the display unit 25 . In addition, text registered in the history information (user database) and phonetic symbols of the text may be displayed on the display unit 25 based on the display data in response to a predetermined operation on the operation unit 24. . The display screen for accuracy illustrated in FIG. 18 includes a voice reproduction operation area 200 and a phonetic symbol display area 201 . An operation image for reproducing the voice of the text registered in the history information is displayed in the voice reproduction operation area 200 . In the phonetic symbol display area 201, text and phonetic symbols are displayed so that their positions correspond to each other.

FIG. 19 is a diagram showing a first example of a fluency-related display screen in the embodiment. In the analysis result display area 120, the number of words in sentences uttered in one minute “WPM (Words Per Minute)” and a linking word button image 123 are displayed. Word counts may be displayed in chronological order. The number of words may be expressed using graphs as well as numerical values.

FIG. 20 is a diagram showing a second example of a display screen regarding fluency in the embodiment. When the filler word button image 123 “Filler Word” is operated, the number of filler words is displayed for each filler word in the analysis result display area 120 . The number of filler words may be expressed for each filler word using a graph as well as a numerical value.

As described above, the conversion unit 414 converts voice data into text. The analysis unit 43 analyzes text commonly used for multiple evaluation indices. The data generation unit 415 generates text and display data representing analysis results for each evaluation index. The communication unit 411 transmits the display data generated for each evaluation index to the terminal 2 that displays the image according to the display data.

As a result, when sentences commonly used for a plurality of evaluation indices are voice-inputted, the learner and the instructor can grasp the learner's ability to speak a foreign language from multiple perspectives. Also, it is possible for the learner and the instructor to quantitatively grasp the foreign language speaking ability of the learner.

The data generation unit 415 may generate, for each evaluation index, display data indicating a portion of the text related to the analysis result of the text. Locations related to analysis results are indicated, for example, using colors distinguishable from other colors. The communication unit 411 may transmit display data including radar charts representing analysis results of a plurality of evaluation indices to the terminal. The communication unit 411 may transmit display data including text displayed in fluorescent colors according to the analysis results of the plurality of evaluation indices to the terminal.

Furthermore, it is possible to improve the learning efficiency of a foreign language. It is possible for the learner to obtain the analysis result of the foreign language speaking ability in real time.

The multiple evaluation indices used in the foreign language learning support system 1 are, for example, at least two of the syntactic complexity of a sentence, the lexical complexity of a sentence, the accuracy of a sentence, and the fluency of a sentence. is. Syntactic complexity is, for example, the number of words per sentence and the number of types of grammatical expressions used in the sentence. Lexical complexity is, for example, the difficulty of the vocabulary contained in the sentence and the frequency of use of particular words in the sentence.

The UI/UX (user interface/user experience) is also devised, so that the learner can appropriately browse the indices of interest to the learner. It is possible to provide learners with multifaceted and quantitative analysis results with high convenience.

For learners, there are at least the following merits (1A) to (3A).
(1A) Since the learner's own English utterances can be accumulated and analyzed as materials for reflection, not only can the efficiency of learning English be improved, but also the willingness to learn English can be enhanced.
(2A) Speech ability originally requires a comprehensive level of pronunciation, grammar, vocabulary, fluency, and the like. Pronunciation, grammar, vocabulary, fluency, etc. are comprehensively (integratedly) evaluated by the foreign language learning support system. Since the analysis results are presented and accumulated using a user interface that is highly convenient and easy to view, the learner can accurately grasp the comprehensive level of the learner's own English speaking ability.
(3A) The foreign language learning support system unifies the evaluation criteria and goals for comprehensive evaluation to a certain extent. Learners can get quantitative feedback on their overall assessment. A learner can determine a specific study plan and study content. Learners can improve their learning efficiency.

For instructors (lecturers), there are at least the following merits (1B) to (4B).
(1B) It is possible to enhance the effect of learning guidance by reducing the teaching load of the instructor.
(2B) Since the foreign language learning support system records and summarizes the learner's mistakes in English conversation, the instructor can concentrate on his role as a conversation partner in English conversation.
(3B) Since the teacher himself/herself does not have to point out mistakes to the learner, it is possible to avoid offending the learner.
(4B) Since the foreign language learning support system provides learners with quantitative feedback, instructors can provide more advanced instruction.

INDUSTRIAL APPLICABILITY The present invention is applicable to a system for supporting learning of a foreign language.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and design and the like are included within the scope of the gist of the present invention.

1 Foreign language learning support system 2 Terminal 3 Communication line 4 Foreign language learning support device 10 Record button image 11 Upload button image 12 Complete button image 13 Complete button image 14 Radar chart display area 15 Simple result display area 16 First switching button image 17 Second switching button image 18 Third switching button image 19 Fourth switching button image 21 Communication section 22 storage unit 23 terminal control unit 24 operation unit 25 display unit 26 microphone 41 server device 42 interface 43 analysis unit 51 user data table 52 lesson data table , 100... sentence display area, 110... operation button display area, 111... first operation button image, 112... second operation button image, 113... third operation button image, 114... fourth operation button image, 120... analysis result Display area 121 Type/token rate button image 122 Pronunciation button image 123 Linking word button image 200 Audio reproduction operation area 201 Phonetic symbol display area 411 Communication unit 412 Storage unit 413 Authentication processing unit 414 Conversion unit 415 Data generation unit 431 Morphological analysis unit 432 First complexity analysis unit 433 Second complexity analysis unit 434 Accuracy analysis unit 435 Fluency sex analysis department

Claims (6)

a conversion unit that converts voice data into text;
an analysis unit that analyzes the text for a plurality of evaluation indicators;
a data generation unit that generates display data representing the text and analysis results for each evaluation index;
A foreign language learning support device comprising: a communication unit configured to transmit the display data generated for each evaluation index to a terminal that displays an image according to the display data. The data generation unit generates the display data indicating a portion related to the analysis result of the text in the text for each evaluation index.
The foreign language learning support device according to claim 1. The communication unit transmits the display data including a radar chart representing analysis results of the plurality of evaluation indices to the terminal.
3. The foreign language learning support device according to claim 1 or 2. wherein the communication unit transmits to the terminal the display data including the text displayed in fluorescent color according to the analysis results of the plurality of evaluation indices;
The foreign language learning support device according to any one of claims 1 to 3. A foreign language learning support method executed by a foreign language learning support device,
a conversion step of converting the audio data to text;
an analysis step of analyzing the text for multiple metrics;
a data generation step of generating display data representing the text and analysis results for each evaluation index;
and a communication step of transmitting the display data generated for each evaluation index to a terminal that displays an image according to the display data. A computer program for causing a computer to function as the foreign language learning support device according to any one of claims 1 to 4.

Images