WO2019146753A1 - Language proficiency assessment device using brain activity, and language proficiency assessment system - Google Patents

Language proficiency assessment device using brain activity, and language proficiency assessment system Download PDF

Info

Publication number
WO2019146753A1
WO2019146753A1 PCT/JP2019/002452 JP2019002452W WO2019146753A1 WO 2019146753 A1 WO2019146753 A1 WO 2019146753A1 JP 2019002452 W JP2019002452 W JP 2019002452W WO 2019146753 A1 WO2019146753 A1 WO 2019146753A1
Authority
WO
WIPO (PCT)
Prior art keywords
learning
language
subject
data
brain activity
Prior art date
Application number
PCT/JP2019/002452
Other languages
French (fr)
Japanese (ja)
Inventor
松本　敦
綾 井原
康 成瀬
司郎 尾島
順一 片山
Original Assignee
国立研究開発法人情報通信研究機構
国立大学法人滋賀大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 国立研究開発法人情報通信研究機構, 国立大学法人滋賀大学 filed Critical 国立研究開発法人情報通信研究機構
Publication of WO2019146753A1 publication Critical patent/WO2019146753A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316—Modalities, i.e. specific diagnostic methods
    • A61B5/369—Electroencephalography [EEG]
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B19/00—Teaching not covered by other main groups of this subclass
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/04—Electrically-operated educational appliances with audible presentation of the material to be studied

Definitions

  • the present invention relates to a language ability evaluation device and a language ability evaluation system.
  • Priority is claimed on Japanese Patent Application No. 2018-011431, filed January 26, 2018, the content of which is incorporated herein by reference.
  • a listening test is performed as language ability evaluation of a language such as a foreign language, and for example, there is a method in which a candidate listens to an English voice and answers with options. In such a method, it is only possible to evaluate the answer with binary values of correct answer and incorrect answer. For example, even a correct answer does not necessarily catch all of the English speech, and it is important for effective learning of a specific language to recognize where listening is inadequate. . Further, there is known a technique of extracting an unintended signal from an electroencephalogram when correctness / incorrectness of the user's answer is presented, and determining the degree of comprehension of the user based on the presence or absence of the unintended signal (see Patent Document 1). ).
  • Patent Document 1 it is necessary for the user to be presented with an answer correctness, which is insufficient as a language listening ability evaluation.
  • the present invention has been made to solve the above problems, and an object thereof is to provide a language ability evaluation device and a language ability evaluation system capable of appropriately evaluating the ability of a specific language.
  • one aspect of the present invention is a data acquisition unit that acquires measurement data of brain activity of the evaluation subject measured when the evaluation subject listens to a voice of a specific language;
  • a teacher who measures measurement data of the brain activity of the learning subject corresponding to an element related to understanding of the language, measured when the subject listens to the voice of the specific language, and the learning level of the particular language of the learning subject
  • the learning result obtained by machine learning based on the added learning data, and the element related to the understanding of the language
  • the elements relating to the understanding of the language include phoneme, part of speech, word difficulty, sentence length, sentence structure, and speed of the speech. At least one of them, wherein the determination unit extracts element component data of brain activity corresponding to an element relating to understanding of the language from measurement data of brain activity of the subject to be evaluated; The learning level of the subject to be evaluated may be determined for each element related to the understanding of the language based on the learning result.
  • the element relating to the understanding of the language is classified into a plurality of classification items, and the determination unit determines the evaluation subject corresponding to the classification items
  • the learning level of each classification item may be determined based on measurement data of brain activity and the learning result.
  • the language ability evaluation apparatus described above may further include a learning unit that executes machine learning based on the supervised learning data and generates the learning result.
  • the measurement data of the brain activity may be brain wave data.
  • a language ability evaluation device described above, a voice output device for outputting the voice of the specific language to the evaluation subject, and a brain activity measuring device for measuring brain activity of the evaluation subject. And the brain activity measuring device measures brain activity when the subject to be evaluated listens to the voice from the voice output device, and the measured data is input to the language ability evaluation device to specify the specified It is a language ability evaluation system characterized by determining a language acquisition level.
  • FIG. 1 is a block diagram showing an example of the English language ability evaluation system 1 according to the present embodiment.
  • the English language ability evaluation system 1 includes an English language ability evaluation device 10, an audio output unit 20, an electroencephalograph 30, and a head gear 31.
  • the English language ability evaluation system 1 is an example of a language ability evaluation system.
  • the voice output unit 20 (an example of a voice output device) is, for example, a speaker, a headphone, etc., converts an English voice signal output from the English ability evaluation device 10 into English voice and evaluates an evaluation target person U1 (evaluation target Output to the subject).
  • the electroencephalograph 30 measures the electroencephalogram when the evaluation subject U1 listens to the English voice output from the voice output unit 20 using the headgear 31 mounted on the head of the evaluation subject U1.
  • the electroencephalograph 30 outputs the measured electroencephalogram to the English ability evaluation device 10 as an electroencephalogram signal.
  • the electroencephalogram signal is an example of a measurement result of brain activity.
  • the electroencephalograph 30 is an example of a brain activity measurement device that measures the brain activity of the evaluation target person U1.
  • the head gear 31 is attachable to the head of the evaluation subject U1 and has an electrode for measuring an electroencephalogram.
  • the English ability evaluation device 10 (an example of a language ability evaluation device) is a device that evaluates the listening ability of English, and based on measurement data of brain activity of the evaluation subject U1 measured by the electroencephalograph 30, listening of English The acquisition level of evaluation object person U1 in is determined.
  • the English language ability evaluation apparatus 10 includes a storage unit 11 and a control unit 12.
  • the storage unit 11 stores various information used by the English language ability evaluation apparatus 10.
  • the storage unit 11 includes an audio data storage unit 111, a learning data storage unit 112, a learning result storage unit 113, an electroencephalogram data storage unit 114, and an evaluation result storage unit 115.
  • the voice data storage unit 111 stores English voice data that the evaluation subject U1 listens to.
  • the voice data storage unit 111 stores, for example, English voice data representing a voice signal of English, types such as phonemes and words included in the English voice data, and their start times.
  • the audio data storage unit 111 stores, for example, audio data as shown in FIG.
  • FIG. 2 is a diagram for explaining an example of audio data in the present embodiment.
  • the voice data includes English voice data representing an English voice signal such as "I fell off my bike", phoneme information such as "ay” and “f” included in the voice, content words, and so forth.
  • English voice data representing an English voice signal such as "I fell off my bike”
  • phoneme information such as "ay” and "f” included in the voice
  • content words and so forth.
  • Different functional words, each phoneme, timing information of each word, and the like are associated with each other and stored in the learning result storage unit 113.
  • the learning data storage unit 112 stores learning data in which the learning subject's acquired level of English and the brain wave data of the learning subject measured when the learning subject listens to the English voice are associated.
  • learning data for example, about 100 learning subjects with high mastery of English (group with high mastery level) and about 100 with unskilled students with low mastery level of English (group with low mastery level) And contains.
  • the learning data storage unit 112 stores, for example, supervised learning data in which element component data in which an electroencephalogram signal of a learning subject is modeled for each element concerning understanding of English is associated with an acquisition level of English.
  • the element component data is obtained by extracting an element component corresponding to an element from an electroencephalogram signal, and, for example, based on the type of an electroencephalogram signal, a phoneme or a word contained in English speech, and its start time. It is a TRF (Time Response Function) calculated by a general linear model or inverse correlation method.
  • TRF Time Response Function
  • each element is classified into a plurality of classification items.
  • the “classification item” of the element “phoneme” includes, for example, “short vowel”, “long vowel”, “double vowel”, “plosive sound”, “frictional sound”, “breaking sound”, etc. .
  • the “classification item” of the element “part of speech” includes, for example, “noun”, “verb”, “adjective”, “adverb”, “interrogative word”, “preposition” and the like.
  • the “classification item” of the element “word level” (word difficulty level) includes, for example, “CEFR A1”, “CEFR A2”, “CEFR B1”, “CEFR B2” and the like.
  • CEFR Common European Framework of Reference
  • the word level is divided into six levels A1 to C2 Do.
  • the “classification item” of the element “sentence length” includes, for example, “5 words or less”, “5 to 10 words or less”, “10 to 20 words or less”, and “20 words or more”.
  • the “classification item” of the element “statement structure” includes, for example, “affirmative sentence”, “negative sentence”, “question sentence”, “instruction sentence” and the like.
  • the “classification item” of the element “speed” includes, for example, “110 wpm (SLOW)”, “160 wpm (NATURAL)”, “210 wpm (FAST)” and the like.
  • wpm (word per minute) indicates the number of words per minute and indicates the speed of reading English sentences.
  • the learning data storage unit 112 stores supervised learning data in which element component data corresponding to each classification item is associated with an acquisition level.
  • the learning result storage unit 113 measures measurement data (brain wave signal) of brain activity of the learning subject corresponding to elements related to comprehension of English, which are measured when the learning subject listens to English speech, and learning level of the learning subject's English Storing the machine-learned learning result based on the supervised learning data in which
  • the learning result is measurement data (brain wave signal) of brain activity measured when the evaluation subject U1 listens to an English voice based on the supervised learning data stored in the learning data storage unit 112 described above.
  • the result is machine learning to estimate the level of English acquisition.
  • the learning result is generated by a learning processing unit 122 described later.
  • the learning result storage unit 113 stores the learning result corresponding to the classification item of each element.
  • the electroencephalogram data storage unit 114 stores the electroencephalogram data measured by the electroencephalograph 30 when the evaluation target person U1 listens to English speech.
  • the evaluation result storage unit 115 stores the determination result of the learning level of English of the evaluation target person U1 determined by the determination processing unit 123 described later.
  • the evaluation result storage unit 115 stores the determination result for each classification item of each element.
  • the control unit 12 is a processor including, for example, a CPU (Central Processing Unit) and the like, and controls the English ability evaluation apparatus 10 in a centralized manner.
  • the control unit 12 includes an electroencephalogram signal acquisition unit 121, a learning processing unit 122, and a determination processing unit 123.
  • the electroencephalogram signal acquisition unit 121 acquires measurement data of brain activity of the evaluation subject U1 measured when the evaluation subject U1 listens to an English voice. That is, the brain wave signal acquisition unit 121 outputs the English voice data stored in the voice data storage unit 111 as a voice signal to the voice output unit 20, and the voice output unit 20 outputs the English voice to the evaluation target person U1.
  • An electroencephalogram signal measured by the electroencephalograph 30 when being heard is acquired from the electroencephalograph 30.
  • the electroencephalogram signal acquisition unit 121 stores the acquired electroencephalogram signal as electroencephalogram data in the electroencephalogram data storage unit 114.
  • the learning processing unit 122 executes machine learning based on supervised learning data stored in the learning data storage unit 112, and generates a learning result.
  • the learning processing unit 122 outputs the English speech data stored in the speech data storage unit 111 as a speech signal to the speech output unit 20, and the speech output unit 20 causes the learning subject to listen to the speech of the English language Supervised by associating element component data (TRF) corresponding to each classification item of each element extracted from the electroencephalogram signal measured by the electroencephalograph 30 using the general linear model or the inverse correlation method with the learning level Machine learning is performed to estimate the acquisition level from new element component data based on the learning data.
  • TRF element component data
  • the learning processing unit 122 executes machine learning using, for example, a support vector machine or ridge regression.
  • the learning processing unit 122 generates a learning result obtained by executing machine learning, and stores the learning result in the learning result storage unit 113.
  • the learning processing unit 122 causes the learning result storage unit 113 to store the learning result corresponding to each classification item of each element.
  • the determination processing unit 123 acquires the learning result stored in the learning result storage unit 113, and based on the brain wave data of the evaluation subject U1 acquired by the brain wave signal acquiring unit 121 and the learning result. , Determine the acquisition level of the evaluation target person U1 corresponding to each element.
  • the elements include, for example, at least one of a phoneme, a part of speech, a word difficulty level, a sentence length, a sentence structure, and an audio speed.
  • the determination processing unit 123 extracts element component data of brain activity corresponding to each classification item of each element from the electroencephalogram data of the evaluation subject U1, and classifies each element based on the element component data and the learning result. The acquisition level of the evaluation target person U1 for each item is determined.
  • the determination processing unit 123 stores the determined determination result (learning level) in the evaluation result storage unit 115 for each classification item of each element. In addition, the determination processing unit 123 outputs the determined determination result (acquisition level) to the outside of the English language ability evaluation apparatus 10.
  • FIG. 4 is a flowchart showing an example of learning processing of the English language ability evaluation apparatus 10 according to the present embodiment.
  • the electroencephalograph 30 measures electroencephalogram data when the learning subject listens to English voice (step S101).
  • the learning processing unit 122 of the control unit 12 outputs the English voice data stored in the voice data storage unit 111 to the voice output unit 20 as a voice signal, and the voice output unit 20 sends the English voice to the learning subject Make it output.
  • the learning processing unit 122 acquires, from the electroencephalograph 30, an electroencephalogram signal measured by the electroencephalograph 30 when the learning subject makes the voice of the English sound listen to from the voice output unit 20.
  • the learning processing unit 122 extracts element component data (TRF) for each element of the acquired electroencephalogram signal using, for example, a general linear model or an inverse correlation method.
  • the learning processing unit 122 causes the learning data storage unit 112 to store supervised learning data in which element component data extracted for each element is associated with the learning level of English.
  • the learning processing unit 122 performs machine learning based on supervised learning data in which the brain wave data corresponding to each element and the learning level are associated (step S102).
  • the learning processing unit 122 obtains, for example, learning data stored in the learning data storage unit 112, and uses, for example, a standard for estimating (determining) an acquisition level using a learning method such as a support vector machine or ridge regression. learn.
  • the learning processing unit 122 machine-learns, for example, a criterion for estimating (determining) an acquisition level for each classification item of each element.
  • the learning processing unit 122 stores the learning result in the learning result storage unit 113 (step S103).
  • the learning processing unit 122 causes the learning result storage unit 113 to store the learning result obtained by machine learning for each classification item of each element.
  • the learning processing unit 122 ends the learning process.
  • FIG. 5 is a flowchart showing an example of the determination process of the English language ability evaluation apparatus 10 according to the present embodiment.
  • the electroencephalograph 30 measures electroencephalogram data when the evaluation subject U1 listens to English voice (step S201).
  • the brain wave signal acquisition unit 121 of the control unit 12 outputs the English voice data stored in the voice data storage unit 111 as a voice signal to the voice output unit 20, and the voice output unit 20 evaluates the English voice Output to the user U1.
  • the electroencephalograph 30 measures the brain wave of the evaluation subject U1 using the headgear 31 mounted on the head of the evaluation subject U1, and outputs it as an electroencephalogram signal.
  • the electroencephalogram signal acquisition unit 121 acquires brain wave data of the evaluation subject U1 (step S202).
  • the electroencephalogram signal acquisition unit 121 acquires, from the electroencephalograph 30, an electroencephalogram signal measured by the electroencephalograph 30 when the evaluation target person U1 causes the voice output unit 20 to listen to the English voice.
  • the brain wave signal acquisition unit 121 stores the acquired brain wave signal as brain wave data in the brain wave data storage unit 114.
  • the determination processing unit 123 of the control unit 12 extracts element component data corresponding to each element from the electroencephalogram data (step S203).
  • the determination processing unit 123 uses, for example, a general linear model or an inverse correlation method based on brain wave data stored in the brain wave data storage unit 114 and English speech data and phoneme / word information stored in the speech data storage unit 111.
  • element component data (TRF) corresponding to the classification item of each element is extracted.
  • the determination processing unit 123 estimates the learning level of each element based on the element component data corresponding to each element and the learning result (step S204).
  • the determination processing unit 123 uses, for example, the learning result corresponding to the classification item of each element stored in the learning result storage unit 113 from the element component data corresponding to the classification item of each element.
  • the acquisition level of the evaluation subject person U1 is estimated.
  • the determination processing unit 123 causes the evaluation result storage unit 115 to store the acquired level of the evaluation target person U1 of the estimated classification item of each element.
  • the determination processing unit 123 outputs the acquired learning level of each element as an evaluation determination result (step S205). For example, as shown in FIG. 6, the determination processing unit 123 outputs, as an evaluation determination result, the learning level of the evaluation target person U1 that is the classification item of each element stored by the evaluation result storage unit 115. After the process of step S205, the determination processing unit 123 ends the determination process.
  • FIG. 6 is a diagram showing an example of the determination result of the English language ability evaluation apparatus 10 according to the present embodiment.
  • the English language ability evaluation apparatus 10 determines the learning level of the evaluation subject U1 based on the brain wave data of the evaluation subject U1 and the learning result stored in the learning result storage unit 113. It is output as an evaluation judgment result of the classification item.
  • the example shown in FIG. 6 shows that the learning level of the evaluation subject person U1 of the "short vowel” of the element “phoneme” is "H” (high acquisition degree), and the evaluation subject person U1 of the “long vowel”
  • the master's acquisition level of "M” indicates that the acquisition level is medium.
  • the learning level may be three or more levels instead of two levels of “H” and “L” (low learning level).
  • the English language ability evaluation apparatus 10 An example is shown in the case where the judgment is made to three levels of H ",” M “and” L ".
  • the English language ability evaluation apparatus 10 (language ability evaluation system) according to the present embodiment includes the electroencephalogram signal acquisition unit 121 (data acquisition unit) and the determination processing unit 123 (determination unit).
  • the electroencephalogram signal acquisition unit 121 acquires measurement data (for example, electroencephalogram data) of the brain activity of the evaluation subject U1 measured when the evaluation subject U1 (the evaluation target subject) listens to an English voice.
  • the determination processing unit 123 acquires machine-learned learning results, and responds to elements related to language understanding based on the measurement data of brain activity of the evaluation subject U1 acquired by the electroencephalogram signal acquisition unit 121 and the learning results.
  • the acquisition level of evaluation object person U1 to judge is determined.
  • the learning result is measured when the learning subject listens to English speech (speech), and elements related to understanding of the language (eg, phoneme, part of speech, word difficulty, sentence length, sentence structure, And the speed of voice, etc.) is machine-learned based on supervised learning data in which measurement data of brain activity of the learning subject corresponding to the learning speed of the learning subject and the learning level of the learning subject are associated.
  • speech speech
  • elements related to understanding of the language eg, phoneme, part of speech, word difficulty, sentence length, sentence structure, And the speed of voice, etc.
  • the English language ability evaluation apparatus 10 is quantitative and objective in order to determine the acquisition level of the evaluation subject person U1 corresponding to the element related to understanding of English from measurement data of brain activity (for example, electroencephalogram data). It is possible to assess English listening skills. That is, the English language ability evaluation apparatus 10 according to the present embodiment can, for example, evaluate the English listening ability without having to present the user with the correctness of the answer. Therefore, the English language ability evaluation apparatus 10 according to the present embodiment can appropriately evaluate English listening ability. Furthermore, the English language ability evaluation apparatus 10 according to the present embodiment recognizes, for example, even if it is a correct answer, how much and how well it can hear English speech (speech), which place is insufficient to listen to English, etc. To effectively learn English.
  • speech English speech
  • the elements relating to the understanding of the language include at least one of a phoneme, a part of speech, a word difficulty level, a sentence length, a sentence structure, and an audio speed.
  • the determination processing unit 123 extracts element component data of brain activity corresponding to an element related to understanding of a language from measurement data of brain activity of the evaluation target person U1, and based on the element component data and a learning result by machine learning. Then, the learning level of the evaluation subject U1 is determined for each element related to the understanding of the language.
  • the English language ability evaluation apparatus 10 according to the present embodiment enables more detailed evaluation of each element.
  • the determination processing unit 123 determines the learning level for each classification item based on the measurement data and the learning result of the brain activity of the evaluation target person U1 corresponding to the classification item.
  • the English language ability evaluation apparatus 10 can determine the acquisition level for each classification item, and clearly identify the classification items that can be heard and the classification items that are insufficient for listening. Can. From this, the English language ability evaluation apparatus 10 according to the present embodiment can learn English language learning more effectively.
  • the English language ability evaluation apparatus 10 includes a learning processing unit 122 (learning unit) that executes machine learning based on the above-described supervised learning data and generates a learning result.
  • a learning processing unit 122 (learning unit) that executes machine learning based on the above-described supervised learning data and generates a learning result.
  • the English language ability evaluation apparatus 10 according to the present embodiment can newly execute machine learning, generate a learning result, and can update the learning result by relearning.
  • the English language ability evaluation method includes a data acquisition step and a determination step.
  • the data acquisition step the electroencephalogram signal acquisition unit 121 acquires measurement data of brain activity of the evaluation subject U1, which is measured when the evaluation subject U1 listens to an English voice.
  • the determination step the determination processing unit 123 acquires the machine-learned learning result described above, and based on the measurement data of the brain activity of the evaluation target person U1 acquired in the data acquisition step and the learning result, The acquisition level of the person U1 to be evaluated corresponding to the element concerning understanding is determined.
  • the English language ability evaluation method can achieve the same effect as the above-described English language ability evaluation apparatus 10, and can appropriately evaluate English listening ability.
  • the English language ability evaluation system 1 (language ability evaluation system) according to the present embodiment includes the above-described English language ability evaluation device 10 (language ability evaluation device), a voice output unit 20 (voice output device), and an electroencephalograph 30 (brain Activity measuring device).
  • the voice output unit 20 outputs English voice to the evaluation subject U1.
  • the electroencephalograph 30 measures brain activity of the evaluation subject U1.
  • the English ability evaluation system 1 measures brain activity when the evaluation target person U1 listens to the voice from the voice output unit 20 with the electroencephalograph 30, and inputs the measured data to the English ability evaluation device 10 for English Determine your mastery level.
  • the English language ability evaluation system 1 according to the present embodiment can achieve the same effect as the above-described English language ability evaluation apparatus 10, and can appropriately evaluate English listening ability.
  • the English language ability evaluation apparatus 10 may be configured to include part or all of the audio output unit 20 or the electroencephalograph 30. Further, the audio output unit 20, the electroencephalograph 30, and the headgear 31 may be placed at a different place from the English language ability evaluation apparatus 10, and in that case, the form of cloud computing can also be used. In addition, part or all of the storage unit 11 may be provided outside the English language ability evaluation apparatus 10. In addition, the English language ability evaluation apparatus 10 may not include the learning processing unit 122, or may have an external learning device that executes the same processing as the learning processing unit 122.
  • the electroencephalograph 30 may have a form in which the head gear 31 is not used. Further, the sound to be heard by the evaluation target person U1 may be different from the sound to be heard by the learning subject. In addition, after determining the learning level of evaluation target person U1 corresponding to an element relating to the understanding of language, for example, a voice including a large number of insufficient listening elements is selected or the speed of voice is changed and reproduced. When the evaluation target person U1 is made to listen and the learning level is determined again, the learning level of the individual evaluation target person U1 can be finely determined.
  • the English language ability evaluation system 1 and the English language ability evaluation device 10 have been described as an example of the language ability evaluation system and the language ability evaluation device, but the invention is not limited thereto. It may be applied to specific languages such as German, French, Italian, Spanish, Russian, Hindi, Chinese, Korean and so on.
  • an example of measuring electroencephalogram data as an example of measurement data of brain activity has been described.
  • the present invention is not limited to this. For example, brain activity by fMRI (functional Magnetic Resonance Imaging) or the like Or other brain activity measurement data.
  • the elements related to understanding of the language are phoneme, part of speech, word difficulty, sentence length, sentence structure, and speech speed
  • the present invention is limited thereto.
  • it may include dullness, linking (connection of the last sound of the word and the first sound of the next word), an accent, a sentence complexity, and the like.
  • the general linear model / inverse correlation method is used to extract component data of brain activity
  • the present invention is not limited thereto.
  • Other methods may be used as long as component data of brain activity can be extracted.
  • examples of machine learning using support vector machines and ridge regression have been described as an example of machine learning, the present invention is not limited to this, and machine learning using other methods such as using neural networks is described. It may apply.
  • each structure with which the English language ability evaluation system 1 mentioned above and the English language ability evaluation apparatus 10 are equipped has a computer system inside. Then, a program for realizing the function of each component included in the above-described English language ability evaluation system 1 and the English language ability evaluation device 10 is recorded in a computer readable recording medium, and the program recorded in the recording medium is a computer system
  • the processing in each configuration provided in the above-described English language ability evaluation system 1 and the English language ability evaluation device 10 may be performed by reading and executing the above.
  • "to read and execute the program recorded on the recording medium into the computer system” includes installing the program on the computer system.
  • the “computer system” mentioned here includes an OS and hardware such as peripheral devices.
  • the “computer system” may include a plurality of computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines.
  • the “computer-readable recording medium” means a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in a computer system.
  • the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Biophysics (AREA)
  • Medical Informatics (AREA)
  • Psychiatry (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Psychology (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

This language proficiency assessment device comprises: a data acquisition unit that acquires measurement data for a to-be-assessed subject's brain activity that was measured when the to-be-assessed subject listened to speech in a specific language; and a determination unit that acquires machine learning results based on supervised learning data, in which measurement data for the learning subject's brain activity corresponding to an element of language comprehension measured when the learning subject listened to speech in the specific language is associated with the learning level of the learning subject with respect to the specific language, and that determines the learning level of the to-be-assessed subject corresponding to the element of language comprehension on the basis of the learning results and the measurement data for the to-be-assessed subject's brain activity acquired by the data acquisition unit.

Description

脳活動を利用した語学能力評価装置、及び語学能力評価システムLanguage ability evaluation system and language ability evaluation system using brain activity
　本発明は、語学能力評価装置、及び語学能力評価システムに関する。
　本願は、２０１８年１月２６日に、日本に出願された特願２０１８－０１１４３１号に基づき優先権を主張し、その内容をここに援用する。 The present invention relates to a language ability evaluation device and a language ability evaluation system.
Priority is claimed on Japanese Patent Application No. 2018-011431, filed January 26, 2018, the content of which is incorporated herein by reference.
　従来、外国語などの言語の語学能力評価として、リスニングのテストが行われており、受験者は、例えば、英語の音声を聴いて選択肢で回答する手法が取られている。このような手法では、解答に対して正答か不正答かの二値でしか評価することができない。例えば、正答であっても、英語の音声の全てを聴き取れているとは限らず、どこが聴き取り不十分なのかを認識することは、効果的に特定の言語を学習する上で重要である。
　また、ユーザの解答の正誤が提示されたときの脳波から期待外れ信号を抽出し、期待外れ信号の有無に基づいて、ユーザの理解度を判定する技術が知られている（特許文献１を参照）。 Conventionally, a listening test is performed as language ability evaluation of a language such as a foreign language, and for example, there is a method in which a candidate listens to an English voice and answers with options. In such a method, it is only possible to evaluate the answer with binary values of correct answer and incorrect answer. For example, even a correct answer does not necessarily catch all of the English speech, and it is important for effective learning of a specific language to recognize where listening is inadequate. .
Further, there is known a technique of extracting an unintended signal from an electroencephalogram when correctness / incorrectness of the user's answer is presented, and determining the degree of comprehension of the user based on the presence or absence of the unintended signal (see Patent Document 1). ).
特許第４４４１３４５号公報Patent No. 4441345
　しかしながら、上述した特許文献１に記載の技術では、例えば、ユーザに解答の正誤が提示される必要があり、言語のリスニング能力評価としては不十分であった。このように、従来技術では、言語のリスニング能力を適切に評価することは困難であった。 However, in the technology described in Patent Document 1 described above, for example, it is necessary for the user to be presented with an answer correctness, which is insufficient as a language listening ability evaluation. Thus, in the prior art, it has been difficult to appropriately evaluate the listening ability of a language.
　本発明は、上記問題を解決すべくなされたもので、その目的は、特定の言語の能力を適切に評価することができる語学能力評価装置、及び語学能力評価システムを提供することにある。 The present invention has been made to solve the above problems, and an object thereof is to provide a language ability evaluation device and a language ability evaluation system capable of appropriately evaluating the ability of a specific language.
　上記問題を解決するために、本発明の一態様は、評価対象被験者が特定言語の音声を聴いた際に計測された前記評価対象被験者の脳活動の計測データを取得するデータ取得部と、学習被験者が前記特定言語の音声を聴いた際に計測され、言語の理解に関する要素に対応する前記学習被験者の脳活動の計測データと、前記学習被験者の前記特定言語の習得レベルとを対応付けた教師付き学習データに基づいて機械学習された学習結果を取得し、前記データ取得部が取得した前記評価対象被験者の脳活動の計測データと、前記学習結果とに基づいて、前記言語の理解に関する要素に対応する前記評価対象者の前記習得レベルを判定する判定部とを備える語学能力評価装置である。 In order to solve the above problem, one aspect of the present invention is a data acquisition unit that acquires measurement data of brain activity of the evaluation subject measured when the evaluation subject listens to a voice of a specific language; A teacher who measures measurement data of the brain activity of the learning subject corresponding to an element related to understanding of the language, measured when the subject listens to the voice of the specific language, and the learning level of the particular language of the learning subject Based on the measurement data of brain activity of the subject to be evaluated acquired by the data acquisition unit, the learning result obtained by machine learning based on the added learning data, and the element related to the understanding of the language It is a language skill evaluation device provided with the judgment part which judges the acquisition level of the corresponding evaluation object person.
　また、本発明の一態様は、上記の語学能力評価装置において、前記言語の理解に関する要素には、音素、品詞、単語の難易度、文の長さ、文の構造、及び前記音声の速度のうちの少なくとも１つが含まれ、前記判定部は、前記評価対象被験者の脳活動の計測データから前記言語の理解に関する要素に対応する脳活動の要素成分データを抽出し、当該要素成分データと、前記学習結果とに基づいて、前記言語の理解に関する要素ごとの前記評価対象被験者の前記習得レベルを判定するようにしてもよい。 Further, according to one aspect of the present invention, in the language ability evaluation apparatus described above, the elements relating to the understanding of the language include phoneme, part of speech, word difficulty, sentence length, sentence structure, and speed of the speech. At least one of them, wherein the determination unit extracts element component data of brain activity corresponding to an element relating to understanding of the language from measurement data of brain activity of the subject to be evaluated; The learning level of the subject to be evaluated may be determined for each element related to the understanding of the language based on the learning result.
　また、本発明の一態様は、上記の語学能力評価装置において、前記言語の理解に関する要素は、複数の分類項目に分類されており、前記判定部は、前記分類項目に対応する前記評価対象被験者の脳活動の計測データ及び前記学習結果に基づいて、前記分類項目ごとの前記習得レベルを判定するようにしてもよい。 Further, according to one aspect of the present invention, in the language ability evaluation apparatus described above, the element relating to the understanding of the language is classified into a plurality of classification items, and the determination unit determines the evaluation subject corresponding to the classification items The learning level of each classification item may be determined based on measurement data of brain activity and the learning result.
　また、本発明の一態様は、上記の語学能力評価装置において、前記教師付き学習データに基づいて、機械学習を実行し、前記学習結果を生成する学習部を備えるようにしてもよい。 Further, according to one aspect of the present invention, the language ability evaluation apparatus described above may further include a learning unit that executes machine learning based on the supervised learning data and generates the learning result.
　また、本発明の一態様は、上記の語学能力評価装置において、前記脳活動の計測データが脳波データであってもよい。 Further, according to one aspect of the present invention, in the language ability evaluation apparatus described above, the measurement data of the brain activity may be brain wave data.
　また、本発明の一態様は、上記に記載の語学能力評価装置と、評価対象被験者に前記特定言語の音声を出力する音声出力装置と、前記評価対象被験者の脳活動を計測する脳活動計測装置とを備え、前記評価対象被験者が前記音声出力装置からの音声を聴いた際の脳活動を前記脳活動計測装置で計測し、計測した当該計測データを前記語学能力評価装置に入力して前記特定言語の習得レベルを判定することを特徴とする語学能力評価システムである。 Further, according to one aspect of the present invention, there is provided a language ability evaluation device described above, a voice output device for outputting the voice of the specific language to the evaluation subject, and a brain activity measuring device for measuring brain activity of the evaluation subject. And the brain activity measuring device measures brain activity when the subject to be evaluated listens to the voice from the voice output device, and the measured data is input to the language ability evaluation device to specify the specified It is a language ability evaluation system characterized by determining a language acquisition level.
　本発明によれば、言語の理解能力を適切に評価することができる。 According to the present invention, language comprehension ability can be appropriately evaluated.
本実施形態による英語能力評価システムの一例を示すブロック図である。It is a block diagram showing an example of the English ability evaluation system by this embodiment. 本実施形態における音声データの一例を説明する図である。It is a figure explaining an example of the voice data in this embodiment. 本実施形態における要素及び分類項目の一例を説明する図である。It is a figure explaining an example of the element in this embodiment, and a classification item. 本実施形態による英語能力評価装置の学習処理の一例を示すフローチャートである。It is a flowchart which shows an example of a learning process of the English ability evaluation apparatus by this embodiment. 本実施形態による英語能力評価装置の判定処理の一例を示すフローチャートである。It is a flowchart which shows an example of the determination processing of the English ability evaluation apparatus by this embodiment. 本実施形態による英語能力評価装置の判定結果の一例を示す図である。It is a figure which shows an example of the determination result of the English ability evaluation apparatus by this embodiment.
　以下、本発明の一実施形態による語学能力評価装置及び語学能力評価システムについて図面を参照して説明する。 Hereinafter, a language ability evaluation device and a language ability evaluation system according to an embodiment of the present invention will be described with reference to the drawings.
　図１は、本実施形態による英語能力評価システム１の一例を示すブロック図である。　図１に示すように、英語能力評価システム１は、英語能力評価装置１０と、音声出力部２０と、脳波計３０と、ヘッドギア３１とを備えている。ここで、英語能力評価システム１は、語学能力評価システムの一例である。 FIG. 1 is a block diagram showing an example of the English language ability evaluation system 1 according to the present embodiment. As shown in FIG. 1, the English language ability evaluation system 1 includes an English language ability evaluation device 10, an audio output unit 20, an electroencephalograph 30, and a head gear 31. Here, the English language ability evaluation system 1 is an example of a language ability evaluation system.
　音声出力部２０（音声出力装置の一例）は、例えば、スピーカやヘッドホンなどであり、英語能力評価装置１０が出力する英語の音声信号を英語の音声に変換して、評価対象者Ｕ１（評価対象被験者）に出力する。 The voice output unit 20 (an example of a voice output device) is, for example, a speaker, a headphone, etc., converts an English voice signal output from the English ability evaluation device 10 into English voice and evaluates an evaluation target person U1 (evaluation target Output to the subject).
　脳波計３０は、評価対象者Ｕ１が音声出力部２０から出力された英語の音声を聴いた際の脳波を、評価対象者Ｕ１の頭部に装着されたヘッドギア３１を用いて計測する。脳波計３０は、計測した脳波を、脳波信号として英語能力評価装置１０に出力する。ここで、脳波信号は、脳活動の計測結果の一例である。また、脳波計３０は、評価対象者Ｕ１の脳活動を計測する脳活動計測装置の一例である。
　ヘッドギア３１は、評価対象者Ｕ１の頭部に装着可能であり、脳波を計測するための電極を有する。 The electroencephalograph 30 measures the electroencephalogram when the evaluation subject U1 listens to the English voice output from the voice output unit 20 using the headgear 31 mounted on the head of the evaluation subject U1. The electroencephalograph 30 outputs the measured electroencephalogram to the English ability evaluation device 10 as an electroencephalogram signal. Here, the electroencephalogram signal is an example of a measurement result of brain activity. The electroencephalograph 30 is an example of a brain activity measurement device that measures the brain activity of the evaluation target person U1.
The head gear 31 is attachable to the head of the evaluation subject U1 and has an electrode for measuring an electroencephalogram.
　英語能力評価装置１０（語学能力評価装置の一例）は、英語のリスニング能力を評価する装置であり、脳波計３０によって計測された評価対象者Ｕ１の脳活動の計測データに基づいて、英語のリスニングにおける評価対象者Ｕ１の習得レベルを判定する。英語能力評価装置１０は、記憶部１１と、制御部１２とを備えている。 The English ability evaluation device 10 (an example of a language ability evaluation device) is a device that evaluates the listening ability of English, and based on measurement data of brain activity of the evaluation subject U1 measured by the electroencephalograph 30, listening of English The acquisition level of evaluation object person U1 in is determined. The English language ability evaluation apparatus 10 includes a storage unit 11 and a control unit 12.
　記憶部１１は、英語能力評価装置１０が利用する各種情報を記憶する。記憶部１１は、音声データ記憶部１１１と、学習データ記憶部１１２と、学習結果記憶部１１３と、脳波データ記憶部１１４と、評価結果記憶部１１５とを備えている。
　音声データ記憶部１１１は、評価対象者Ｕ１に聴かせる英語の音声データを記憶する。音声データ記憶部１１１は、例えば、英語の音声信号を示す英語音声データと、当該英語音声データに含まれる音素や単語などの種類とその開始時間とを対応付けて記憶する。音声データ記憶部１１１は、例えば、図２に示すような音声データを記憶する。 The storage unit 11 stores various information used by the English language ability evaluation apparatus 10. The storage unit 11 includes an audio data storage unit 111, a learning data storage unit 112, a learning result storage unit 113, an electroencephalogram data storage unit 114, and an evaluation result storage unit 115.
The voice data storage unit 111 stores English voice data that the evaluation subject U1 listens to. The voice data storage unit 111 stores, for example, English voice data representing a voice signal of English, types such as phonemes and words included in the English voice data, and their start times. The audio data storage unit 111 stores, for example, audio data as shown in FIG.
　図２は、本実施形態における音声データの一例を説明する図である。
　図２に示すように、音声データは、“Ｉ　ｆｅｌｌ　ｏｆｆ　ｍｙ　ｂｉｋｅ”などの英語の音声信号を示す英語音声データと、音声に含まれる“ａｙ”、“ｆ”などの音素情報、内容語・機能語の別、及び各音素と各単語のタイミング情報などとが対応付けられて、学習結果記憶部１１３に記憶されている。 FIG. 2 is a diagram for explaining an example of audio data in the present embodiment.
As shown in FIG. 2, the voice data includes English voice data representing an English voice signal such as "I fell off my bike", phoneme information such as "ay" and "f" included in the voice, content words, and so forth. Different functional words, each phoneme, timing information of each word, and the like are associated with each other and stored in the learning result storage unit 113.
　学習データ記憶部１１２は、学習被験者の英語の習得レベルと、学習被験者が英語の音声を聴いた際に計測された学習被験者の脳波データとが対応付けられた学習データを記憶する。ここで、学習被験者は、例えば、英語の習得度の高い熟達群（習得レベルの高いグループ）の１００名程度と、英語の習得度の低い未熟達群（習得レベルの低いグループ）の１００名程度とを含んでいる。また、学習データ記憶部１１２は、例えば、学習被験者の脳波信号を英語の理解に関する要素ごとにモデル化した要素成分データと、英語の習得レベルとを対応付けた教師付き学習データを記憶する。ここで、要素成分データは、脳波信号から要素に対応する要素成分を抽出したものであり、例えば、脳波信号と、英語音声に含まれる音素や単語などの種類とその開始時間などとに基づいて一般線形モデルや逆相関法などにより算出されたＴＲＦ（Time Response Function）である。 The learning data storage unit 112 stores learning data in which the learning subject's acquired level of English and the brain wave data of the learning subject measured when the learning subject listens to the English voice are associated. Here, for example, about 100 learning subjects with high mastery of English (group with high mastery level) and about 100 with unskilled students with low mastery level of English (group with low mastery level) And contains. Further, the learning data storage unit 112 stores, for example, supervised learning data in which element component data in which an electroencephalogram signal of a learning subject is modeled for each element concerning understanding of English is associated with an acquisition level of English. Here, the element component data is obtained by extracting an element component corresponding to an element from an electroencephalogram signal, and, for example, based on the type of an electroencephalogram signal, a phoneme or a word contained in English speech, and its start time. It is a TRF (Time Response Function) calculated by a general linear model or inverse correlation method.
　また、言語の理解に関する要素には、図３に示すように、「音素」、「品詞」、「単語レベル」（単語の難易度）、「文の長さ」、「文構造」、及び「スピード」の６つの要素が含まれる。また、各要素は、複数の分類項目に分類される。 Further, as shown in FIG. 3, the elements relating to the understanding of the language include "phoneme", "part of speech", "word level" (degree of word difficulty), "sentence length", "sentence structure", and " Six elements of "speed" are included. In addition, each element is classified into a plurality of classification items.
　ここで、要素「音素」の「分類項目」には、例えば、“短母音”、“長母音”、“二重母音”、“破裂音”、“摩擦音”、“破擦音”などが含まれる。また、要素「品詞」の「分類項目」には、例えば、“名詞”、“動詞”、“形容詞”、“副詞”、“疑問詞”、“前置詞”などが含まれる。また、要素「単語レベル」（単語の難易度）の「分類項目」には、例えば、“ＣＥＦＲ　Ａ１”、“ＣＥＦＲ　Ａ２”、“ＣＥＦＲ　Ｂ１”、“ＣＥＦＲ　Ｂ２”などが含まれる。ここで、ＣＥＦＲ（Common European Framework of Reference）は、ヨーロッパで外国語の語学力の熟達度を測る基準であり、ここでは、単語レベル（単語の難易度）をＡ１～Ｃ２の６段階にレベル分けする。 Here, the “classification item” of the element “phoneme” includes, for example, “short vowel”, “long vowel”, “double vowel”, “plosive sound”, “frictional sound”, “breaking sound”, etc. . Further, the “classification item” of the element “part of speech” includes, for example, “noun”, “verb”, “adjective”, “adverb”, “interrogative word”, “preposition” and the like. Further, the “classification item” of the element “word level” (word difficulty level) includes, for example, “CEFR A1”, “CEFR A2”, “CEFR B1”, “CEFR B2” and the like. Here, CEFR (Common European Framework of Reference) is a standard for measuring the language proficiency of foreign languages in Europe. Here, the word level (word difficulty level) is divided into six levels A1 to C2 Do.
　また、要素「文の長さ」の「分類項目」には、例えば、“５ワード以下”、“５～１０ワード以下”、“１０～２０ワード以下”、“２０ワード以上”が含まれる。また、要素「文構造」の「分類項目」には、例えば、“肯定文”、“否定文”、“疑問文”、“命令文”などが含まれる。また、要素「スピード」の「分類項目」には、例えば、“１１０ｗｐｍ（ＳＬＯＷ）”、“１６０ｗｐｍ（ＮＡＴＵＲＡＬ）”、“２１０ｗｐｍ（ＦＡＳＴ）”などが含まれる。ここで、ｗｐｍ（word per minute）は、１分間の単語数を示し、英文を読むスピードを示している。
　学習データ記憶部１１２は、各分類項目に対応する要素成分データと、習得レベルとを対応付けた教師付き学習データを記憶する。 Also, the “classification item” of the element “sentence length” includes, for example, “5 words or less”, “5 to 10 words or less”, “10 to 20 words or less”, and “20 words or more”. Further, the “classification item” of the element “statement structure” includes, for example, “affirmative sentence”, “negative sentence”, “question sentence”, “instruction sentence” and the like. Further, the “classification item” of the element “speed” includes, for example, “110 wpm (SLOW)”, “160 wpm (NATURAL)”, “210 wpm (FAST)” and the like. Here, wpm (word per minute) indicates the number of words per minute and indicates the speed of reading English sentences.
The learning data storage unit 112 stores supervised learning data in which element component data corresponding to each classification item is associated with an acquisition level.
　学習結果記憶部１１３は、学習被験者が英語の音声を聴いた際に計測され、英語の理解に関する要素に対応する学習被験者の脳活動の計測データ（脳波信号）と、学習被験者の英語の習得レベルとを対応付けた教師付き学習データに基づいて機械学習された学習結果を記憶する。ここで、学習結果は、上述した学習データ記憶部１１２が記憶する教師付き学習データに基づいて、評価対象者Ｕ１が英語の音声を聴いた際に計測された脳活動の計測データ（脳波信号）から、英語の習得レベルを推定する機械学習が行われた結果である。また、学習結果は、後述する学習処理部１２２によって生成される。
　なお、学習結果記憶部１１３は、各要素の分類項目に対応する学習結果を記憶する。 The learning result storage unit 113 measures measurement data (brain wave signal) of brain activity of the learning subject corresponding to elements related to comprehension of English, which are measured when the learning subject listens to English speech, and learning level of the learning subject's English Storing the machine-learned learning result based on the supervised learning data in which Here, the learning result is measurement data (brain wave signal) of brain activity measured when the evaluation subject U1 listens to an English voice based on the supervised learning data stored in the learning data storage unit 112 described above. The result is machine learning to estimate the level of English acquisition. The learning result is generated by a learning processing unit 122 described later.
The learning result storage unit 113 stores the learning result corresponding to the classification item of each element.
　脳波データ記憶部１１４は、評価対象者Ｕ１が英語の音声を聴いた際に脳波計３０が計測した脳波データを記憶する。
　評価結果記憶部１１５は、後述する判定処理部１２３が判定した評価対象者Ｕ１の英語の習得レベルの判定結果を記憶する。評価結果記憶部１１５は、各要素の分類項目ごとに、判定結果を記憶する。 The electroencephalogram data storage unit 114 stores the electroencephalogram data measured by the electroencephalograph 30 when the evaluation target person U1 listens to English speech.
The evaluation result storage unit 115 stores the determination result of the learning level of English of the evaluation target person U1 determined by the determination processing unit 123 described later. The evaluation result storage unit 115 stores the determination result for each classification item of each element.
　制御部１２は、例えば、ＣＰＵ（Central Processing Unit）などを含むプロセッサであり、英語能力評価装置１０を統括的に制御する。制御部１２は、脳波信号取得部１２１と、学習処理部１２２と、判定処理部１２３とを備えている。 The control unit 12 is a processor including, for example, a CPU (Central Processing Unit) and the like, and controls the English ability evaluation apparatus 10 in a centralized manner. The control unit 12 includes an electroencephalogram signal acquisition unit 121, a learning processing unit 122, and a determination processing unit 123.
　脳波信号取得部１２１（データ取得部の一例）は、評価対象者Ｕ１が英語の音声を聴いた際に計測された評価対象者Ｕ１の脳活動の計測データを取得する。すなわち、脳波信号取得部１２１は、音声データ記憶部１１１が記憶する英語音声データを、音声信号として、音声出力部２０に出力して、音声出力部２０から当該英語の音声を評価対象者Ｕ１に聴かせた際に脳波計３０が計測した脳波信号を脳波計３０から取得する。脳波信号取得部１２１は、所得した脳波信号を脳波データとして、脳波データ記憶部１１４に記憶させる。 The electroencephalogram signal acquisition unit 121 (an example of a data acquisition unit) acquires measurement data of brain activity of the evaluation subject U1 measured when the evaluation subject U1 listens to an English voice. That is, the brain wave signal acquisition unit 121 outputs the English voice data stored in the voice data storage unit 111 as a voice signal to the voice output unit 20, and the voice output unit 20 outputs the English voice to the evaluation target person U1. An electroencephalogram signal measured by the electroencephalograph 30 when being heard is acquired from the electroencephalograph 30. The electroencephalogram signal acquisition unit 121 stores the acquired electroencephalogram signal as electroencephalogram data in the electroencephalogram data storage unit 114.
　学習処理部１２２（学習部の一例）は、学習データ記憶部１１２が記憶する教師付き学習データに基づいて、機械学習を実行し、学習結果を生成する。学習処理部１２２は、例えば、音声データ記憶部１１１が記憶する英語音声データを、音声信号として、音声出力部２０に出力して、音声出力部２０から当該英語の音声を学習被験者に聴かせた際に脳波計３０が計測した脳波信号から一般線形モデルや逆相関法などを用いて抽出した各要素の各分類項目に対応する要素成分データ（ＴＲＦ）と、習得レベルとを対応付けた教師付き学習データに基づいて、新たな要素成分データから習得レベルを推定する機械学習を実行する。 The learning processing unit 122 (an example of a learning unit) executes machine learning based on supervised learning data stored in the learning data storage unit 112, and generates a learning result. For example, the learning processing unit 122 outputs the English speech data stored in the speech data storage unit 111 as a speech signal to the speech output unit 20, and the speech output unit 20 causes the learning subject to listen to the speech of the English language Supervised by associating element component data (TRF) corresponding to each classification item of each element extracted from the electroencephalogram signal measured by the electroencephalograph 30 using the general linear model or the inverse correlation method with the learning level Machine learning is performed to estimate the acquisition level from new element component data based on the learning data.
　ここで、学習処理部１２２は、例えば、サポートベクターマシンやリッジ回帰を利用した機械学習を実行する。学習処理部１２２は、機械学習を実行した学習結果を生成し、当該学習結果を学習結果記憶部１１３に記憶させる。学習処理部１２２は、各要素の各分類項目に対応する学習結果を学習結果記憶部１１３に記憶させる。 Here, the learning processing unit 122 executes machine learning using, for example, a support vector machine or ridge regression. The learning processing unit 122 generates a learning result obtained by executing machine learning, and stores the learning result in the learning result storage unit 113. The learning processing unit 122 causes the learning result storage unit 113 to store the learning result corresponding to each classification item of each element.
　判定処理部１２３（判定部の一例）は、学習結果記憶部１１３が記憶する学習結果を取得し、脳波信号取得部１２１が取得した評価対象者Ｕ１の脳波データと、当該学習結果とに基づいて、各要素に対応する評価対象者Ｕ１の習得レベルを判定する。ここで、要素には、例えば、音素、品詞、単語の難易度、文の長さ、文の構造、及び音声の速度のうちの少なくとも１つが含まれている。判定処理部１２３は、評価対象者Ｕ１の脳波データから各要素の各分類項目に対応する脳活動の要素成分データを抽出し、当該要素成分データと、学習結果とに基づいて、各要素の分類項目ごとの評価対象者Ｕ１の習得レベルを判定する。 The determination processing unit 123 (an example of the determination unit) acquires the learning result stored in the learning result storage unit 113, and based on the brain wave data of the evaluation subject U1 acquired by the brain wave signal acquiring unit 121 and the learning result. , Determine the acquisition level of the evaluation target person U1 corresponding to each element. Here, the elements include, for example, at least one of a phoneme, a part of speech, a word difficulty level, a sentence length, a sentence structure, and an audio speed. The determination processing unit 123 extracts element component data of brain activity corresponding to each classification item of each element from the electroencephalogram data of the evaluation subject U1, and classifies each element based on the element component data and the learning result. The acquisition level of the evaluation target person U1 for each item is determined.
　判定処理部１２３は、判定した判定結果（習得レベル）を、各要素の分類項目ごとに評価結果記憶部１１５に記憶させる。また、判定処理部１２３は、判定した当該判定結果（習得レベル）を英語能力評価装置１０の外部に出力する。 The determination processing unit 123 stores the determined determination result (learning level) in the evaluation result storage unit 115 for each classification item of each element. In addition, the determination processing unit 123 outputs the determined determination result (acquisition level) to the outside of the English language ability evaluation apparatus 10.
　次に、図面を参照して、本実施形態による英語能力評価システム１の動作について説明する。
　図４は、本実施形態による英語能力評価装置１０の学習処理の一例を示すフローチャートである。 Next, with reference to the drawings, the operation of the English language ability evaluation system 1 according to the present embodiment will be described.
FIG. 4 is a flowchart showing an example of learning processing of the English language ability evaluation apparatus 10 according to the present embodiment.
　図４に示すように、まず、脳波計３０は、学習被験者が英語の音声を聴いた際の脳波データを計測する（ステップＳ１０１）。例えば、制御部１２の学習処理部１２２は、音声データ記憶部１１１が記憶する英語音声データを、音声信号として、音声出力部２０に出力し、音声出力部２０から当該英語の音声を学習被験者に出力させる。学習処理部１２２は、音声出力部２０から当該英語の音声を学習被験者に聴かせた際に脳波計３０が計測した脳波信号を脳波計３０から取得する。 As shown in FIG. 4, first, the electroencephalograph 30 measures electroencephalogram data when the learning subject listens to English voice (step S101). For example, the learning processing unit 122 of the control unit 12 outputs the English voice data stored in the voice data storage unit 111 to the voice output unit 20 as a voice signal, and the voice output unit 20 sends the English voice to the learning subject Make it output. The learning processing unit 122 acquires, from the electroencephalograph 30, an electroencephalogram signal measured by the electroencephalograph 30 when the learning subject makes the voice of the English sound listen to from the voice output unit 20.
　ここで、学習被験者は、例えば、英語の習得度の高い熟達群（習得レベルの高いグループ）の１００名程度と、英語の習得度の低い未熟達群（習得レベルの低いグループ）の１００名程度とである。学習処理部１２２は、取得した脳波信号を、例えば、一般線形モデルや逆相関法を利用して要素ごとの要素成分データ(ＴＲＦ)を抽出する。学習処理部１２２は、要素ごとに抽出した要素成分データと、英語の習得レベルとを対応付けた教師付き学習データを、学習データ記憶部１１２に記憶させる。 Here, for example, about 100 learning subjects with high mastery of English (group with high mastery level) and about 100 with unskilled students with low mastery level of English (group with low mastery level) And The learning processing unit 122 extracts element component data (TRF) for each element of the acquired electroencephalogram signal using, for example, a general linear model or an inverse correlation method. The learning processing unit 122 causes the learning data storage unit 112 to store supervised learning data in which element component data extracted for each element is associated with the learning level of English.
　次に、学習処理部１２２は、各要素に対応する脳波データと、習得レベルとを対応付けた教師付き学習データに基づいて機械学習する（ステップＳ１０２）。学習処理部１２２は、例えば、学習データ記憶部１１２が記憶する学習データを取得し、例えば、サポートベクターマシンやリッジ回帰などの学習手法を利用して、習得レベルを推定（判定）する基準を機械学習する。なお、学習処理部１２２は、例えば、各要素の分類項目ごとに、習得レベルを推定（判定）する基準を機械学習する。 Next, the learning processing unit 122 performs machine learning based on supervised learning data in which the brain wave data corresponding to each element and the learning level are associated (step S102). The learning processing unit 122 obtains, for example, learning data stored in the learning data storage unit 112, and uses, for example, a standard for estimating (determining) an acquisition level using a learning method such as a support vector machine or ridge regression. learn. The learning processing unit 122 machine-learns, for example, a criterion for estimating (determining) an acquisition level for each classification item of each element.
　次に、学習処理部１２２は、学習結果を学習結果記憶部１１３に記憶させる（ステップＳ１０３）。学習処理部１２２は、例えば、各要素の分類項目ごとに機械学習した学習結果を学習結果記憶部１１３に記憶させる。ステップＳ１０３の処理後に、学習処理部１２２は、学習処理を終了する。 Next, the learning processing unit 122 stores the learning result in the learning result storage unit 113 (step S103). For example, the learning processing unit 122 causes the learning result storage unit 113 to store the learning result obtained by machine learning for each classification item of each element. After the process of step S103, the learning processing unit 122 ends the learning process.
　次に、図５を参照して、本実施形態による英語能力評価装置１０の判定処理について説明する。
　図５は、本実施形態による英語能力評価装置１０の判定処理の一例を示すフローチャートである。 Next, with reference to FIG. 5, the determination process of the English language ability evaluation apparatus 10 according to the present embodiment will be described.
FIG. 5 is a flowchart showing an example of the determination process of the English language ability evaluation apparatus 10 according to the present embodiment.
　図５に示すように、まず、脳波計３０は、評価対象者Ｕ１が英語の音声を聴いた際の脳波データを計測する（ステップＳ２０１）。例えば、制御部１２の脳波信号取得部１２１は、音声データ記憶部１１１が記憶する英語音声データを、音声信号として、音声出力部２０に出力し、音声出力部２０から当該英語の音声を評価対象者Ｕ１に出力させる。脳波計３０は、評価対象者Ｕ１の頭部に装着されたヘッドギア３１を用いて、評価対象者Ｕ１の脳波を計測して、脳波信号として出力する。 As shown in FIG. 5, first, the electroencephalograph 30 measures electroencephalogram data when the evaluation subject U1 listens to English voice (step S201). For example, the brain wave signal acquisition unit 121 of the control unit 12 outputs the English voice data stored in the voice data storage unit 111 as a voice signal to the voice output unit 20, and the voice output unit 20 evaluates the English voice Output to the user U1. The electroencephalograph 30 measures the brain wave of the evaluation subject U1 using the headgear 31 mounted on the head of the evaluation subject U1, and outputs it as an electroencephalogram signal.
　次に、脳波信号取得部１２１は、評価対象者Ｕ１の脳波データを取得する（ステップＳ２０２）。脳波信号取得部１２１は、音声出力部２０から当該英語の音声を評価対象者Ｕ１に聴かせた際に脳波計３０が計測した脳波信号を脳波計３０から取得する。脳波信号取得部１２１は、取得した脳波信号を脳波データとして、脳波データ記憶部１１４に記憶させる。 Next, the electroencephalogram signal acquisition unit 121 acquires brain wave data of the evaluation subject U1 (step S202). The electroencephalogram signal acquisition unit 121 acquires, from the electroencephalograph 30, an electroencephalogram signal measured by the electroencephalograph 30 when the evaluation target person U1 causes the voice output unit 20 to listen to the English voice. The brain wave signal acquisition unit 121 stores the acquired brain wave signal as brain wave data in the brain wave data storage unit 114.
　次に、制御部１２の判定処理部１２３は、脳波データから各要素に対応する要素成分データを抽出する（ステップＳ２０３）。判定処理部１２３は、例えば、脳波データ記憶部１１４が記憶する脳波データと、音声データ記憶部１１１が記憶する英語音声データ及び音素・単語情報とに基づいて、一般線形モデルや逆相関法を利用して、各要素の分類項目に対応する要素成分データ(ＴＲＦ)を抽出する。 Next, the determination processing unit 123 of the control unit 12 extracts element component data corresponding to each element from the electroencephalogram data (step S203). The determination processing unit 123 uses, for example, a general linear model or an inverse correlation method based on brain wave data stored in the brain wave data storage unit 114 and English speech data and phoneme / word information stored in the speech data storage unit 111. Then, element component data (TRF) corresponding to the classification item of each element is extracted.
　次に、判定処理部１２３は、各要素に対応する要素成分データと学習結果とに基づいて各要素の習得レベルを推定する（ステップＳ２０４）。判定処理部１２３は、例えば、各要素の分類項目に対応する要素成分データから、学習結果記憶部１１３が記憶する各要素の分類項目に対応する学習結果を用いて、各要素の分類項目ことの評価対象者Ｕ１の習得レベルを推定する。判定処理部１２３は、推定した各要素の分類項目ことの評価対象者Ｕ１の習得レベルを評価結果記憶部１１５に記憶させる。 Next, the determination processing unit 123 estimates the learning level of each element based on the element component data corresponding to each element and the learning result (step S204). The determination processing unit 123 uses, for example, the learning result corresponding to the classification item of each element stored in the learning result storage unit 113 from the element component data corresponding to the classification item of each element. The acquisition level of the evaluation subject person U1 is estimated. The determination processing unit 123 causes the evaluation result storage unit 115 to store the acquired level of the evaluation target person U1 of the estimated classification item of each element.
　次に、判定処理部１２３は、推定した各要素の習得レベルを評価判定結果として出力する（ステップＳ２０５）。判定処理部１２３は、例えば、図６に示すように、評価結果記憶部１１５が記憶する、各要素の分類項目ことの評価対象者Ｕ１の習得レベルを、評価判定結果として出力する。ステップＳ２０５の処理後に、判定処理部１２３は、判定処理を終了する。 Next, the determination processing unit 123 outputs the acquired learning level of each element as an evaluation determination result (step S205). For example, as shown in FIG. 6, the determination processing unit 123 outputs, as an evaluation determination result, the learning level of the evaluation target person U1 that is the classification item of each element stored by the evaluation result storage unit 115. After the process of step S205, the determination processing unit 123 ends the determination process.
　図６は、本実施形態による英語能力評価装置１０の判定結果の一例を示す図である。　図６に示すように、英語能力評価装置１０は、評価対象者Ｕ１の脳波データと、学習結果記憶部１１３が記憶する学習結果とに基づいて、評価対象者Ｕ１の習得レベルを、各要素の分類項目ことの評価判定結果として出力する。 FIG. 6 is a diagram showing an example of the determination result of the English language ability evaluation apparatus 10 according to the present embodiment. As shown in FIG. 6, the English language ability evaluation apparatus 10 determines the learning level of the evaluation subject U1 based on the brain wave data of the evaluation subject U1 and the learning result stored in the learning result storage unit 113. It is output as an evaluation judgment result of the classification item.
　図６に示す例では、要素「音素」の“短母音”の評価対象者Ｕ１の習得レベルが、“Ｈ”（習得度が高い）であることを示し、“長母音”の評価対象者Ｕ１の習得レベルが、“Ｍ”（習得度が中程度）であることを示している。まお、習得レベルは、“Ｈ”、“Ｌ”（習得度が低い）の２レベルでなく、３以上のレベルであってもよく、この図に示す例では、英語能力評価装置１０は、“Ｈ”、“Ｍ”、“Ｌ”の３レベルに判定した場合の一例を示している。 The example shown in FIG. 6 shows that the learning level of the evaluation subject person U1 of the "short vowel" of the element "phoneme" is "H" (high acquisition degree), and the evaluation subject person U1 of the "long vowel" The master's acquisition level of "M" indicates that the acquisition level is medium. The learning level may be three or more levels instead of two levels of “H” and “L” (low learning level). In the example shown in this figure, the English language ability evaluation apparatus 10 An example is shown in the case where the judgment is made to three levels of H "," M "and" L ".
　以上説明したように、本実施形態による英語能力評価装置１０（語学能力評価装置）は、脳波信号取得部１２１（データ取得部）と、判定処理部１２３（判定部）とを備える。脳波信号取得部１２１は、評価対象者Ｕ１（評価対象被験者）が英語の音声を聴いた際に計測された評価対象者Ｕ１の脳活動の計測データ（例えば、脳波データ）を取得する。判定処理部１２３は、機械学習された学習結果を取得し、脳波信号取得部１２１が取得した評価対象者Ｕ１の脳活動の計測データと、学習結果とに基づいて、言語の理解に関する要素に対応する評価対象者Ｕ１の習得レベルを判定する。ここで、学習結果は、学習被験者が英語の音声（スピーチ）を聴いた際に計測され、言語の理解に関する要素（例えば、音素、品詞、単語の難易度、文の長さ、文の構造、及び音声の速度など）に対応する学習被験者の脳活動の計測データと、学習被験者の英語の習得レベルとを対応付けた教師付き学習データに基づいて機械学習されたものである。 As described above, the English language ability evaluation apparatus 10 (language ability evaluation system) according to the present embodiment includes the electroencephalogram signal acquisition unit 121 (data acquisition unit) and the determination processing unit 123 (determination unit). The electroencephalogram signal acquisition unit 121 acquires measurement data (for example, electroencephalogram data) of the brain activity of the evaluation subject U1 measured when the evaluation subject U1 (the evaluation target subject) listens to an English voice. The determination processing unit 123 acquires machine-learned learning results, and responds to elements related to language understanding based on the measurement data of brain activity of the evaluation subject U1 acquired by the electroencephalogram signal acquisition unit 121 and the learning results. The acquisition level of evaluation object person U1 to judge is determined. Here, the learning result is measured when the learning subject listens to English speech (speech), and elements related to understanding of the language (eg, phoneme, part of speech, word difficulty, sentence length, sentence structure, And the speed of voice, etc.) is machine-learned based on supervised learning data in which measurement data of brain activity of the learning subject corresponding to the learning speed of the learning subject and the learning level of the learning subject are associated.
　これにより、本実施形態による英語能力評価装置１０は、脳活動の計測データ（例えば、脳波データ）から英語の理解に関する要素に対応する評価対象者Ｕ１の習得レベルを判定するため、定量的且つ客観的に英語のリスニング能力を評価することができる。すなわち、本実施形態による英語能力評価装置１０は、例えば、ユーザに解答の正誤を提示する必要なく、英語のリスニング能力を評価することができる。よって、本実施形態による英語能力評価装置１０は、英語のリスニング能力を適切に評価することができる。さらに、本実施形態による英語能力評価装置１０は、例えば、正答であっても、英語の音声（スピーチ）をどの程度、聴き取れているのか、どこが聴き取り不十分なのか、等を認識することができ、効果的に英語を学習することが可能になる。 Thereby, the English language ability evaluation apparatus 10 according to the present embodiment is quantitative and objective in order to determine the acquisition level of the evaluation subject person U1 corresponding to the element related to understanding of English from measurement data of brain activity (for example, electroencephalogram data). It is possible to assess English listening skills. That is, the English language ability evaluation apparatus 10 according to the present embodiment can, for example, evaluate the English listening ability without having to present the user with the correctness of the answer. Therefore, the English language ability evaluation apparatus 10 according to the present embodiment can appropriately evaluate English listening ability. Furthermore, the English language ability evaluation apparatus 10 according to the present embodiment recognizes, for example, even if it is a correct answer, how much and how well it can hear English speech (speech), which place is insufficient to listen to English, etc. To effectively learn English.
　また、本実施形態では、言語の理解に関する要素には、音素、品詞、単語の難易度、文の長さ、文の構造、及び音声の速度のうちの少なくとも１つが含まれる。判定処理部１２３は、評価対象者Ｕ１の脳活動の計測データから言語の理解に関する要素に対応する脳活動の要素成分データを抽出し、当該要素成分データと、機械学習された学習結果とに基づいて、言語の理解に関する要素ごとの評価対象者Ｕ１の習得レベルを判定する。
　これにより、本実施形態による英語能力評価装置１０は、要素ごとのより詳細な評価が可能になる。 Further, in the present embodiment, the elements relating to the understanding of the language include at least one of a phoneme, a part of speech, a word difficulty level, a sentence length, a sentence structure, and an audio speed. The determination processing unit 123 extracts element component data of brain activity corresponding to an element related to understanding of a language from measurement data of brain activity of the evaluation target person U1, and based on the element component data and a learning result by machine learning. Then, the learning level of the evaluation subject U1 is determined for each element related to the understanding of the language.
Thereby, the English language ability evaluation apparatus 10 according to the present embodiment enables more detailed evaluation of each element.
　また、本実施形態では、言語の理解に関する要素は、複数の分類項目に分類されている。判定処理部１２３は、分類項目に対応する評価対象者Ｕ１の脳活動の計測データ及び学習結果に基づいて、分類項目ごとの習得レベルを判定する。
　これにより、本実施形態による英語能力評価装置１０は、分類項目ごとの習得レベルを判定することが可能になり、聴き取り出来ている分類項目と、聴き取り不充分な分類項目を明確にあることができる。このことから、本実施形態による英語能力評価装置１０は、さらに効果的に英語の語学学習をすることが可能になる。 Further, in the present embodiment, elements related to understanding of the language are classified into a plurality of classification items. The determination processing unit 123 determines the learning level for each classification item based on the measurement data and the learning result of the brain activity of the evaluation target person U1 corresponding to the classification item.
As a result, the English language ability evaluation apparatus 10 according to the present embodiment can determine the acquisition level for each classification item, and clearly identify the classification items that can be heard and the classification items that are insufficient for listening. Can. From this, the English language ability evaluation apparatus 10 according to the present embodiment can learn English language learning more effectively.
　また、本実施形態による英語能力評価装置１０は、上述した教師付き学習データに基づいて、機械学習を実行し、学習結果を生成する学習処理部１２２（学習部）を備える。　これにより、本実施形態による英語能力評価装置１０は、新たに機械学習を実行し、学習結果を生成することが可能になり、再学習により学習結果を更新することができる。 In addition, the English language ability evaluation apparatus 10 according to the present embodiment includes a learning processing unit 122 (learning unit) that executes machine learning based on the above-described supervised learning data and generates a learning result. As a result, the English language ability evaluation apparatus 10 according to the present embodiment can newly execute machine learning, generate a learning result, and can update the learning result by relearning.
　また、本実施形態による英語能力評価方法は、データ取得ステップと、判定ステップとを含む。データ取得ステップにおいて、脳波信号取得部１２１が、評価対象者Ｕ１が英語の音声を聴いた際に計測された評価対象者Ｕ１の脳活動の計測データを取得する。判定ステップにおいて、判定処理部１２３が、上述した機械学習された学習結果を取得し、データ取得ステップによって取得された評価対象者Ｕ１の脳活動の計測データと、学習結果とに基づいて、言語の理解に関する要素に対応する評価対象者Ｕ１の習得レベルを判定する。
　これにより、本実施形態による英語能力評価方法は、上述した英語能力評価装置１０と同様の効果を奏し、英語のリスニング能力を適切に評価することができる。 In addition, the English language ability evaluation method according to the present embodiment includes a data acquisition step and a determination step. In the data acquisition step, the electroencephalogram signal acquisition unit 121 acquires measurement data of brain activity of the evaluation subject U1, which is measured when the evaluation subject U1 listens to an English voice. In the determination step, the determination processing unit 123 acquires the machine-learned learning result described above, and based on the measurement data of the brain activity of the evaluation target person U1 acquired in the data acquisition step and the learning result, The acquisition level of the person U1 to be evaluated corresponding to the element concerning understanding is determined.
Thereby, the English language ability evaluation method according to the present embodiment can achieve the same effect as the above-described English language ability evaluation apparatus 10, and can appropriately evaluate English listening ability.
　また、本実施形態による英語能力評価システム１（語学能力評価システム）は、上述した英語能力評価装置１０（語学能力評価装置）と、音声出力部２０（音声出力装置）と、脳波計３０（脳活動計測装置）とを備える。音声出力部２０は、評価対象者Ｕ１に英語の音声を出力する。脳波計３０は、評価対象者Ｕ１の脳活動を計測する。英語能力評価システム１は、評価対象者Ｕ１が音声出力部２０からの音声を聴いた際の脳活動を脳波計３０で計測し、計測した当該計測データを英語能力評価装置１０に入力して英語の習得レベルを判定する。
　これにより、本実施形態による英語能力評価システム１は、上述した英語能力評価装置１０と同様の効果を奏し、英語のリスニング能力を適切に評価することができる。 In addition, the English language ability evaluation system 1 (language ability evaluation system) according to the present embodiment includes the above-described English language ability evaluation device 10 (language ability evaluation device), a voice output unit 20 (voice output device), and an electroencephalograph 30 (brain Activity measuring device). The voice output unit 20 outputs English voice to the evaluation subject U1. The electroencephalograph 30 measures brain activity of the evaluation subject U1. The English ability evaluation system 1 measures brain activity when the evaluation target person U1 listens to the voice from the voice output unit 20 with the electroencephalograph 30, and inputs the measured data to the English ability evaluation device 10 for English Determine your mastery level.
Thereby, the English language ability evaluation system 1 according to the present embodiment can achieve the same effect as the above-described English language ability evaluation apparatus 10, and can appropriately evaluate English listening ability.
　なお、本発明は、上記の各実施形態に限定されるものではなく、本発明の趣旨を逸脱しない範囲で変更可能である。
　例えば、上記の実施形態において、英語能力評価装置１０は、音声出力部２０又は脳波計３０の一部又は全部を含む形態であってもよい。また、音声出力部２０と脳波計３０とヘッドギア３１は、英語能力評価装置１０と異なる場所に置かれてもよく、その場合クラウドコンピューティングの形態でも利用できる。また、記憶部１１の一部又は全部を英語能力評価装置１０の外部に備える形態であってもよい。また、英語能力評価装置１０は、学習処理部１２２を備えない形態であってもよいし、外部に学習処理部１２２と同様の処理を実行する学習装置を備えるようにしてもよい。また、脳波計３０は、ヘッドギア３１を用いない形態であってもよい。また、評価対象者Ｕ１に聴かせる音声は、学習被験者に聞かせる音声とは異なるものであってもよい。また、言語の理解に関する要素に対応する評価対象者Ｕ１の習得レベルを判定した後、例えば、聴き取り不充分な要素を多く含む音声を選択するまたは音声の速度を変更して再生するなどして評価対象者Ｕ１に聴かせ、再度習得レベルを判定する構成とした場合、評価対象者Ｕ１個々人の習得レベルを精緻に判別することができる。 In addition, this invention is not limited to said each embodiment, It can change in the range which does not deviate from the meaning of this invention.
For example, in the above embodiment, the English language ability evaluation apparatus 10 may be configured to include part or all of the audio output unit 20 or the electroencephalograph 30. Further, the audio output unit 20, the electroencephalograph 30, and the headgear 31 may be placed at a different place from the English language ability evaluation apparatus 10, and in that case, the form of cloud computing can also be used. In addition, part or all of the storage unit 11 may be provided outside the English language ability evaluation apparatus 10. In addition, the English language ability evaluation apparatus 10 may not include the learning processing unit 122, or may have an external learning device that executes the same processing as the learning processing unit 122. In addition, the electroencephalograph 30 may have a form in which the head gear 31 is not used. Further, the sound to be heard by the evaluation target person U1 may be different from the sound to be heard by the learning subject. In addition, after determining the learning level of evaluation target person U1 corresponding to an element relating to the understanding of language, for example, a voice including a large number of insufficient listening elements is selected or the speed of voice is changed and reproduced. When the evaluation target person U1 is made to listen and the learning level is determined again, the learning level of the individual evaluation target person U1 can be finely determined.
　また、上記の実施形態において、語学能力評価システム及び語学能力評価装置の一例として、英語能力評価システム１及び英語能力評価装置１０を説明したが、これに限定されるものではなく、英語以外の日本語、ドイツ語、フランス語、イタリア語、スペイン語、ロシア語、ヒンディー語、中国語、韓国語などの特定言語に適用してもよい。
　また、上記の実施形態において、脳活動の計測データの一例として、脳波データを計測する例を説明したが、これに限定されるものではなく、例えば、ｆＭＲＩ（functional Magnetic Resonance Imaging）などによる脳活動の計測データでもよいし、他の脳活動の計測データでもよい。 In the above embodiment, the English language ability evaluation system 1 and the English language ability evaluation device 10 have been described as an example of the language ability evaluation system and the language ability evaluation device, but the invention is not limited thereto. It may be applied to specific languages such as German, French, Italian, Spanish, Russian, Hindi, Chinese, Korean and so on.
In the above embodiment, an example of measuring electroencephalogram data as an example of measurement data of brain activity has been described. However, the present invention is not limited to this. For example, brain activity by fMRI (functional Magnetic Resonance Imaging) or the like Or other brain activity measurement data.
　また、上記の実施形態において、言語の理解に関する要素が、音素、品詞、単語の難易度、文の長さ、文の構造、及び音声の速度である例を説明したが、これに限定されるものではなく、例えば、なまり、リンキング（単語の最後の音と次の単語の最初の音の連結）、アクセント、文の複雑さなどを含めてもよい。
　また、上記の実施形態において、脳活動の要素成分データの抽出に一般線形モデル・逆相関法を利用する例を説明したが、これに限定されるものではなく、脳波データから各要素に対応する脳活動の要素成分データが抽出できれば、他の手法を用いてもよい。機械学習の一例として、サポートベクターマシンやリッジ回帰を利用する機械学習などの例を説明したが、これに限定されるものではなく、ニューラルネットワークを用いるものなど、他の手法を用いた機械学習を適用してもよい。 Also, in the above embodiment, although the elements related to understanding of the language are phoneme, part of speech, word difficulty, sentence length, sentence structure, and speech speed, the present invention is limited thereto. For example, it may include dullness, linking (connection of the last sound of the word and the first sound of the next word), an accent, a sentence complexity, and the like.
In the above embodiment, although an example in which the general linear model / inverse correlation method is used to extract component data of brain activity has been described, the present invention is not limited thereto. Other methods may be used as long as component data of brain activity can be extracted. Although examples of machine learning using support vector machines and ridge regression have been described as an example of machine learning, the present invention is not limited to this, and machine learning using other methods such as using neural networks is described. It may apply.
　なお、上述した英語能力評価システム１及び英語能力評価装置１０が備える各構成は、内部に、コンピュータシステムを有している。そして、上述した英語能力評価システム１及び英語能力評価装置１０が備える各構成の機能を実現するためのプログラムをコンピュータ読み取り可能な記録媒体に記録して、この記録媒体に記録されたプログラムをコンピュータシステムに読み込ませ、実行することにより上述した英語能力評価システム１及び英語能力評価装置１０が備える各構成における処理を行ってもよい。ここで、「記録媒体に記録されたプログラムをコンピュータシステムに読み込ませ、実行する」とは、コンピュータシステムにプログラムをインストールすることを含む。ここでいう「コンピュータシステム」とは、ＯＳや周辺機器等のハードウェアを含むものとする。
　また、「コンピュータシステム」は、インターネットやＷＡＮ、ＬＡＮ、専用回線等の通信回線を含むネットワークを介して接続された複数のコンピュータ装置を含んでもよい。また、「コンピュータ読み取り可能な記録媒体」とは、フレキシブルディスク、光磁気ディスク、ＲＯＭ、ＣＤ－ＲＯＭ等の可搬媒体、コンピュータシステムに内蔵されるハードディスク等の記憶装置のことをいう。このように、プログラムを記憶した記録媒体は、ＣＤ－ＲＯＭ等の非一過性の記録媒体であってもよい。 In addition, each structure with which the English language ability evaluation system 1 mentioned above and the English language ability evaluation apparatus 10 are equipped has a computer system inside. Then, a program for realizing the function of each component included in the above-described English language ability evaluation system 1 and the English language ability evaluation device 10 is recorded in a computer readable recording medium, and the program recorded in the recording medium is a computer system The processing in each configuration provided in the above-described English language ability evaluation system 1 and the English language ability evaluation device 10 may be performed by reading and executing the above. Here, "to read and execute the program recorded on the recording medium into the computer system" includes installing the program on the computer system. The “computer system” mentioned here includes an OS and hardware such as peripheral devices.
Also, the “computer system” may include a plurality of computer devices connected via a network including communication lines such as the Internet, WAN, LAN, and dedicated lines. The “computer-readable recording medium” means a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in a computer system. As described above, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.
　１　英語能力評価システム、１０　英語能力評価装置、１１　記憶部、１２　制御部、２０　音声出力部、３０　脳波計、３１　ヘッドギア、１１１　音声データ記憶部、１１２　学習データ記憶部、１１３　学習結果記憶部、１１４　脳波データ記憶部、１１５　評価結果記憶部、１２１　脳波信号取得部、１２２　学習処理部、１２３　判定処理部、Ｕ１　評価対象者 DESCRIPTION OF SYMBOLS 1 English ability evaluation system, 10 English ability evaluation apparatus, 11 storage part, 12 control part, 20 audio output part, 30 electroencephalograph, 31 headgear, 111 audio data storage part, 112 learning data storage part, 113 learning result storage part, 114 electroencephalogram data storage unit, 115 evaluation result storage unit, 121 electroencephalogram signal acquisition unit, 122 learning processing unit, 123 determination processing unit, U1 evaluation target person

Claims (6)

  1. 　評価対象被験者が特定言語の音声を聴いた際に計測された前記評価対象被験者の脳活動の計測データを取得するデータ取得部と、
    　学習被験者が前記特定言語の音声を聴いた際に計測され、言語の理解に関する要素に対応する前記学習被験者の脳活動の計測データと、前記学習被験者の前記特定言語の習得レベルとを対応付けた教師付き学習データに基づいて機械学習された学習結果を取得し、前記データ取得部が取得した前記評価対象被験者の脳活動の計測データと、前記学習結果とに基づいて、前記言語の理解に関する要素に対応する前記評価対象被験者の前記習得レベルを判定する判定部と
    　を備える語学能力評価装置。     A data acquisition unit for acquiring measurement data of brain activity of the evaluation subject measured when the evaluation subject listens to a voice of a specific language;
    The measurement data of the brain activity of the learning subject corresponding to the element concerning the understanding of the language, which is measured when the learning subject listens to the voice of the specific language, is associated with the learning level of the specific language of the learning subject An element related to understanding of the language based on measurement data of brain activity of the subject to be evaluated acquired by learning data obtained by machine learning based on supervised learning data and the data acquisition unit acquired A language skill evaluation apparatus comprising: a determination unit that determines the acquisition level of the evaluation target subject corresponding to.
  2. 　前記言語の理解に関する要素には、音素、品詞、単語の難易度、文の長さ、文の構造、及び前記音声の速度のうちの少なくとも１つが含まれ、
    　前記判定部は、前記評価対象被験者の脳活動の計測データから前記言語の理解に関する要素に対応する脳活動の要素成分データを抽出し、当該要素成分データと、前記学習結果とに基づいて、前記言語の理解に関する要素ごとの前記評価対象被験者の前記習得レベルを判定する
    　請求項１に記載の語学能力評価装置。     The elements related to understanding the language include at least one of a phoneme, a part of speech, a word difficulty, a sentence length, a sentence structure, and the speed of the speech.
    The determination unit extracts element component data of brain activity corresponding to an element relating to the understanding of the language from measurement data of brain activity of the evaluation target subject, and based on the element component data and the learning result, The language skill evaluation apparatus according to claim 1, wherein the acquisition level of the evaluation subject is determined for each element related to understanding of language.
  3. 　前記言語の理解に関する要素は、複数の分類項目に分類されており、
    　前記判定部は、前記分類項目に対応する前記評価対象被験者の脳活動の計測データ及び前記学習結果に基づいて、前記分類項目ごとの前記習得レベルを判定する
    　請求項１又は請求項２に記載の語学能力評価装置。     Elements related to the understanding of the language are classified into a plurality of classification items,
    The said determination part determines the said acquisition level for every said classification item based on the measurement data and said learning result of the brain activity of the said evaluation object test subject corresponding to the said classification item. Language ability evaluation device.
  4. 　前記教師付き学習データに基づいて、機械学習を実行し、前記学習結果を生成する学習部を備える
    　請求項１から請求項３のいずれか一項に記載の語学能力評価装置。     The language ability evaluation device according to any one of claims 1 to 3, further comprising a learning unit that executes machine learning based on the supervised learning data and generates the learning result.
  5. 　前記脳活動の計測データが脳波データである
    　請求項１から請求項４のいずれか一項に記載の語学能力評価装置。     The language ability evaluation device according to any one of claims 1 to 4, wherein the measurement data of the brain activity is electroencephalogram data.
  6. 　請求項１から請求項５のいずれか一項に記載の語学能力評価装置と、
    　評価対象被験者に前記特定言語の音声を出力する音声出力装置と、
    　前記評価対象被験者の脳活動を計測する脳活動計測装置と
    　を備え、
    　前記評価対象被験者が前記音声出力装置からの音声を聴いた際の脳活動を前記脳活動計測装置で計測し、計測した当該計測データを前記語学能力評価装置に入力して前記特定言語の習得レベルを判定する
    　語学能力評価システム。     The language ability evaluation device according to any one of claims 1 to 5;
    An audio output device for outputting the voice of the specific language to a subject to be evaluated;
    A brain activity measuring device for measuring the brain activity of the subject to be evaluated;
    The brain activity measuring device measures brain activity when the subject to be evaluated listens to the voice from the voice output device, and the measured data is input to the language ability evaluation device to acquire the specific language acquisition level Language ability evaluation system to determine the.
PCT/JP2019/002452 2018-01-26 2019-01-25 Language proficiency assessment device using brain activity, and language proficiency assessment system WO2019146753A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018011431A JP7125042B2 (en) 2018-01-26 2018-01-26 Language ability evaluation device and language ability evaluation system using brain activity
JP2018-011431 2018-01-26

Publications (1)

Publication Number Publication Date
WO2019146753A1 true WO2019146753A1 (en) 2019-08-01

Family

ID=67396074

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2019/002452 WO2019146753A1 (en) 2018-01-26 2019-01-25 Language proficiency assessment device using brain activity, and language proficiency assessment system

Country Status (2)

Country Link
JP (1) JP7125042B2 (en)
WO (1) WO2019146753A1 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113476057A (en) * 2021-07-08 2021-10-08 先端智能科技(天津)有限公司 Content evaluation method and device, electronic device and storage medium
EP4147636A4 (en) * 2020-05-27 2024-05-15 Tokyo Inst Tech Information processing device and information processing method

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6792091B1 (en) * 2020-01-20 2020-11-25 泰宏 中野 Speech learning system and speech learning method
CN112107318B (en) * 2020-09-24 2024-02-27 自达康(北京)科技有限公司 Physical activity ability evaluation system

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017201342A (en) * 2016-05-02 2017-11-09 良一 春日 Language Learning Robot Software

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2017201342A (en) * 2016-05-02 2017-11-09 良一 春日 Language Learning Robot Software

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHANG, MING: "Unconscious improvement in foreign language learning using mismatch negativity neurofeedback: A preliminary study", PLOS ONE, vol. 12, no. 6, 15 June 2017 (2017-06-15), pages 1 - 13, XP055627299 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4147636A4 (en) * 2020-05-27 2024-05-15 Tokyo Inst Tech Information processing device and information processing method
CN113476057A (en) * 2021-07-08 2021-10-08 先端智能科技(天津)有限公司 Content evaluation method and device, electronic device and storage medium
CN113476057B (en) * 2021-07-08 2023-04-07 先端智能科技(天津)有限公司 Content evaluation method and device, electronic device and storage medium

Also Published As

Publication number Publication date
JP2019128533A (en) 2019-08-01
JP7125042B2 (en) 2022-08-24

Similar Documents

Publication Publication Date Title
Thomson Measurement of accentedness, intelligibility, and comprehensibility
WO2019146753A1 (en) Language proficiency assessment device using brain activity, and language proficiency assessment system
Pelzl et al. ADVANCED SECOND LANGUAGE LEARNERS’PERCEPTION OF LEXICAL TONE CONTRASTS
McCrocklin ASR-based dictation practice for second language pronunciation improvement
Roberts et al. Word recognition
US9449522B2 (en) Systems and methods for evaluating difficulty of spoken text
Sultana et al. SUST Bangla Emotional Speech Corpus (SUBESCO): An audio-only emotional speech corpus for Bangla
Michael Automated Speech Recognition in language learning: Potential models, benefits and impact
Cucchiarini et al. Second language learners' spoken discourse: Practice and corrective feedback through automatic speech recognition
De Jong et al. Evidence for featural units in the acquisition of speech production skills: Linguistic structure in foreign accent
Dahlen et al. Rehearsal and aptitude in foreign vocabulary learning
Baer‐Henney et al. The interaction of language‐specific and universal factors during the acquisition of morphophonemic alternations with exceptions
Inoue et al. A Study of Objective Measurement of Comprehensibility through Native Speakers' Shadowing of Learners' Utterances.
Swingley et al. Lexical learning may contribute to phonetic learning in infants: A corpus analysis of maternal Spanish
Escudero et al. “Mummy, keep it steady”: phonetic variation shapes word learning at 15 and 17 months
Bobb et al. Second language learners' listener impressions of foreigner-directed speech
Mokari et al. Perceptual assimilation predicts acquisition of foreign language sounds: The case of Azerbaijani learners’ production and perception of Standard Southern British English vowels
KR20050089043A (en) Systems and methods for dynamically analyzing temporality in speech
Ambrazaitis Nuclear intonation in Swedish: Evidence from experimental-phonetic studies and a comparison with German
Georgiou Speakers of different L1 dialects with acoustically proximal vowel systems present with similar nonnative speech perception abilities: Data from Greek listeners of Dutch
Kaland et al. Red and blue bananas: Time-series f0 analysis of contrastively focused noun phrases in Papuan Malay and Dutch
JP2019078989A (en) Multilingual learning system and language ability measurement system
Rato et al. Designing speech perception tasks with TP
Jevtović et al. One-to-one or one too many? Linking sound-to-letter mappings to speech sound perception and production in early readers
JP6594273B2 (en) Questioning utterance determination device, method and program thereof

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19743751

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19743751

Country of ref document: EP

Kind code of ref document: A1