JP2021110895A - Hearing impairment determination device, hearing impairment determination system, computer program and cognitive function level correction method - Google Patents

Abstract

To provide a hearing impairment determination device, a hearing impairment determination system, a computer program and a hearing impairment determination method, which can determine hearing impairment of an object person.SOLUTION: A hearing impairment determination device includes: an acquisition unit for acquiring interaction voice between an object person and an interacting person; an extraction unit for extracting a feature amount on voice of the interacting person on the basis of the acquired interaction voice; a calculation unit for calculating delay time of a response of the object person with respect to utterance of the interacting person on the basis of the acquired interaction voice; and a hearing impairment level determination unit for determining a hearing impairment level of the object person on the basis of the extracted feature amount and the calculated delay time.SELECTED DRAWING: Figure 1

Description

The present invention relates to a deafness determination device, a deafness determination system, a computer program, and a deafness determination method.

In recent years, there is concern that the number of dementia patients will increase, and techniques for early detection of dementia have been developed using various approaches. Patent Document 1 discloses a device that extracts prosodic features based on user's voice data and calculates the risk of cognitive dysfunction using a pre-constructed learning model.

Japanese Unexamined Patent Publication No. 2011-255106

A person's hearing declines with age, and older people are more likely to develop deafness. In addition, cognitive function may decline and cognitive dysfunction may occur in old age. However, in the past, it was difficult to distinguish between dementia and deafness. Therefore, if a subject with hearing loss cannot answer the question of the questioner correctly, the subject may be erroneously determined to have dementia.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a deafness determination device, a deafness determination system, a computer program, and a deafness determination method capable of determining a subject's deafness.

The hearing loss determination device according to the embodiment of the present invention extracts a feature amount related to the voice of the interlocutor based on the acquisition unit that acquires the dialogue voice between the target person and the interlocutor and the dialogue voice acquired by the acquisition unit. The extraction unit, the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the calculation unit. It is provided with a hearing loss level determining unit that determines the hearing loss level of the subject based on the calculated delay time.

The hearing loss determination system according to the embodiment of the present invention extracts a feature amount related to the voice of the interlocutor based on the acquisition unit that acquires the dialogue voice between the target person and the interlocutor and the dialogue voice acquired by the acquisition unit. The extraction unit, the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the calculation unit. It is provided with a hearing loss level determining unit that determines the hearing loss level of the subject based on the calculated delay time.

The computer program according to the embodiment of the present invention includes a process of acquiring a dialogue voice between a target person and an interlocutor on a computer, and a process of extracting a feature amount related to the voice of the interlocutor based on the acquired dialogue voice. , A process of calculating the delay time of the subject's response to the interlocutor's utterance based on the acquired dialogue voice, and a process of determining the hearing loss level of the subject based on the extracted feature amount and the calculated delay time. To execute.

In the method for determining hearing loss according to the embodiment of the present invention, the dialogue voice between the subject and the interlocutor is acquired, the feature amount related to the voice of the interlocutor is extracted based on the acquired dialogue voice, and the acquired dialogue is performed. The delay time of the response of the subject to the speech of the interlocutor is calculated based on the voice, and the hearing loss level of the subject is determined based on the extracted feature amount and the calculated delay time.

According to the present invention, it is possible to determine the deafness of a subject.

It is a schematic diagram which shows an example of the structure of the deafness determination system of embodiment. It is a schematic diagram which shows an example of the voice waveform of the dialogue voice. It is a schematic diagram which shows an example of the hearing level by age. It is explanatory drawing which shows an example of the feature amount of an interlocutor. It is a schematic diagram which shows an example of the deafness level determination method. It is a schematic diagram which shows the 1st example of the structure of the cognitive function level determination part. It is a schematic diagram which shows the 2nd example of the structure of the cognitive function level determination part. It is a schematic diagram which shows the 3rd example of the structure of the cognitive function level determination part. It is explanatory drawing which shows an example of the correction method of the cognitive function level. It is a schematic diagram which shows the 1st example of the display method of the determination result. It is a schematic diagram which shows the 2nd example of the display method of the determination result. It is a schematic diagram which shows the 3rd example of the display method of the determination result. It is a schematic diagram which shows the 4th example of the display method of the determination result. It is a flowchart which shows an example of the processing procedure of a deafness determination system.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram showing an example of the configuration of the hearing loss determination system of the present embodiment. The deafness determination system includes a deafness determination device 50 and a terminal device 10. The deafness determination device 50 and the terminal device 10 are connected via the communication network 1. The terminal device 10 can be composed of, for example, an information processing device such as a personal computer, a tablet, a smartphone, or a smart speaker. A microphone 11 is connected to the terminal device 10. The microphone 11 can acquire the voice of the target person and the interlocutor interacting with the target person. The microphone 11 may be built in the terminal device 10 as long as the voices of the target person and the interlocutor can be acquired. The subject is a subject for deafness determination, and the interlocutor is a person who conducts a dialogue for deafness determination of the subject such as a doctor, a nurse, a counselor, and a caregiver. When there is an interlocutor, the subject can talk with the interlocutor, and when there is no interlocutor, a predetermined sentence or the like can be read aloud. The voice of the target person or the voice of the target person and the interlocutor is collected by the microphone 11 and transmitted to the deafness determination device 50 via the terminal device 10.

The dialogue between the target person and the interlocutor may be performed online via the communication network 1. In this case, the voice of the target person is acquired by the microphone 11A connected to the terminal device 10A used by the target person, and the voice of the interlocutor is acquired by the microphone 11B connected to the terminal device 10B used by the interlocutor. The voices acquired in 11A and B are transmitted to the hearing loss determination device 50.

The deafness determination device 50 includes a control unit 51, a communication unit 52, a voice identification unit 53, a feature amount extraction unit 54, a delay time calculation unit 55, a storage unit 56, an anxiety level determination unit 57, and a deafness level determination unit that control the entire device. It includes 58, a cognitive function level determination unit 59, a correction unit 60, and an output unit 61. The control unit 51 can be composed of a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The voice identification unit 53, the feature amount extraction unit 54, the delay time calculation unit 55, the anxiety level determination unit 57, the hearing loss level determination unit 58, the cognitive function level determination unit 59, the correction unit 60, and the output unit 61 are CPUs and GPUs (Graphics). It can be configured by any of hardware such as Processing Units), DSP (Digital Signal Processors), FPGA (Field-Programmable Gate Arrays), or a combination thereof. The control unit 51, the voice identification unit 53, the feature amount extraction unit 54, the delay time calculation unit 55, the anxiety level determination unit 57, the hearing loss level determination unit 58, the cognitive function level determination unit 59, and the correction unit 60 of the hearing loss determination device 50. Each function of the above may be provided in the terminal device 10, and some functions of the hearing loss determination device 50 may be provided in the terminal device 10. Further, each function of the deafness determination device 50 may be distributed and provided in a plurality of devices.

The communication unit 52 can be configured with a required communication module, has a function of communicating with the terminal device 10 via the communication network 1, and transmits / receives required information with the terminal device 10. It can be performed. The communication unit 52 has a function as an acquisition unit, and can acquire the dialogue voice between the target person and the interlocutor from the terminal device 10.

FIG. 2 is a schematic diagram showing an example of a voice waveform of a dialogue voice. The vertical axis shows the amplitude of the audio signal, and the horizontal axis shows the time. In the example of FIG. 2, the voice 1 of the interlocutor, the voice 1 of the subject, the voice 2 of the interlocutor, the voice 2 of the subject, and the voice 3 of the interlocutor follow. There is a response delay time between the interlocutor's voice 1 and the target person's voice 1, and similarly, there is a response delay time between the interlocutor's voice 2 and the target person's voice 2. There is.

The voice identification unit 53 can discriminate between the voice of the target person and the voice of the interlocutor from the dialogue voice acquired via the communication unit 52. The voice can be identified by storing the voice data of the target person and the interlocutor in the storage unit 56 in advance and collating the voice data with the stored voice data. The storage unit 56 can be composed of a hard disk, a flash memory, or the like. In addition, machine learning may be used for voice identification. For example, the voices of interlocutors such as doctors, nurses, counselors, and caregivers can be identified by machine learning. Further, the voice of the target person may be machine-learned in advance. Further, as another method of voice identification, when the interlocutor operates an operation button or the like provided on the terminal device 10 when the target person speaks or when the interlocutor speaks, the terminal device 10 operates. An identification flag indicating that the button or the like has been operated is synchronized with the voice data and transmitted to the hearing loss determination device 50. The voice identification unit 53 can acquire the identification flag and identify whether it is the voice of the target person or the voice of the interlocutor depending on the presence or absence of the identification flag. Further, as another method of voice identification, the directivity of the microphone 11 can be used. For example, the microphone 11 can be arranged so that the target person can enter the region having high directivity, and can be distinguished by the magnitude of the distortion of the sound.

The feature amount extraction unit 54 has a function as an extraction unit, and extracts a feature amount related to the voice of the interlocutor based on the dialogue voice acquired via the communication unit 52. The feature amount may be any feature amount that represents difficulty in hearing for the hearing impaired. The characteristics of age-related deafness are that high frequencies cannot be heard, that the frequency resolution is reduced because the subtle differences in frequencies contained in the voice cannot be heard, or that the time resolution is reduced because the fast-spoken voice cannot be heard. Features etc. can be mentioned. In particular, the feature that high frequencies are hard to hear becomes more prominent as the age goes up.

FIG. 3 is a schematic diagram showing an example of hearing level by age group. In the figure, the vertical axis represents the hearing level, which is the lowest loudness that a person can hear. The horizontal axis shows the frequency of voice. In the example of the figure, the hearing level of each age group in the 50s, 60s, and 70s is shown. As can be seen from the figure, the higher the frequency, the less audible it tends to be, and the tendency increases with age. Therefore, the case of using the feature amount indicating the degree of the high frequency region included in the voice will be described below. As described above, the feature amount is not limited to this, and any feature amount indicating difficulty in hearing for a deaf person can be used.

FIG. 4 is an explanatory diagram showing an example of the feature amount of the voice of the interlocutor. As shown in FIG. 4, as the feature amount, for example, power, time, number of times, frequency, voice recognition difficulty, and the like can be mentioned. Power indicates the weight (power) for each frequency in the frequency analysis of sound, and is different from the loudness and intensity (volume) of sound perceived by human hearing. The feature amount extraction unit 54 can extract at least one of consonants or power, time, number of times, frequency or voice recognition difficulty in a high frequency range equal to or higher than a predetermined threshold value as a feature amount. The power can be the intensity of consonants or speech in the high frequency range above the threshold. The time can be the time during which a consonant or a sound in a high frequency range equal to or higher than the threshold value is output. The number of times indicates how many times a word containing a consonant or a high frequency range equal to or higher than a threshold value appears in one phrase of the interlocutor's utterance. The frequency indicates how many times a word containing a high frequency region above the threshold value appears per unit time. The voice recognition difficulty can be the difficulty of language recognition when consonants or high frequency voices are cut. The difficulty level of language recognition can be the failure rate of automatic speech recognition. By cutting consonants or high-frequency voices, it is possible to make the utterance content of the interlocutor linguistically difficult to understand. All of these features indicate the degree of the high frequency range included in the voice.

The delay time calculation unit 55 has a function as a calculation unit, and calculates the delay time (response delay time) of the response of the target person to the utterance of the interlocutor based on the dialogue voice acquired via the communication unit 52. As illustrated in FIG. 2, the delay time can be the time from the end of the interlocutor's utterance to the start of the subject's response. Deaf people tend to have longer delays than healthy people.

The deafness level determination unit 58 has a function as a determination unit, and can determine the deafness level of the subject based on the feature amount extracted by the feature amount extraction unit 54 and the delay time calculated by the delay time calculation unit 55. can. That is, the hearing loss level determination unit 58 targets according to how difficult it is for the hearing-impaired person to hear the interlocutor's utterance and how much the subject's utterance is delayed with respect to the interlocutor's utterance. Determine the level of hearing loss of a person.

The anxiety level determination unit 57 can determine the anxiety level of the target person with respect to the utterance of the interlocutor based on the dialogue voice acquired via the communication unit 52. The magnitude of the anxiety level corresponds to the magnitude of the self-confidence level. For determining the anxiety level, for example, at least one of the voice volume of the subject's utterance (amount representing the loudness and strength of the voice), the specificity of the utterance content, and the clarity of the utterance content can be used. For example, when the subject's voice is low, it can be determined that the anxiety level is high. In addition, when the utterance content of the subject is unclear or unclear, it can be determined that the anxiety level is high.

When determining the anxiety level, a camera provided in the terminal device 10 or a separate camera is connected to the terminal device 10, and the facial expression of the target person is based on an image obtained by photographing the target person with the camera. , Gestures, gestures, gaze of the subject, etc. may be extracted to determine the anxiety level of the subject.

The deafness level determination unit 58 may further determine the deafness level of the subject based on the anxiety level determined by the anxiety level determination unit 57. For example, when the level of anxiety is high, it can be determined that the level of deafness is high because it is considered that the cause is that the interlocutor's utterance cannot be heard or is difficult to hear. As a result, the deafness level of the subject can be determined more accurately.

FIG. 5 is a schematic diagram showing an example of a method for determining the level of deafness. It is assumed that the dialogue voice between the interlocutor and the target person A includes the voices 1 to 5 (question) of the interlocutor and the voices 1 to 5 (answer) of the target person corresponding to each of them. It is assumed that the feature level X for each of the interlocutor's voices 1, 2, 3, 4, and 5 is 2, 2, 5, 1, 4. The feature level X can be divided into 5 stages of 1 to 5, for example, by weighting each feature amount extracted by the feature amount extraction unit 54. The larger the value of the feature level X, the more difficult it is for the deaf person to hear the interlocutor's utterance. Further, it is assumed that the delay time Y of the response to each of the voices 1, 2, 3, 4, and 5 of the subject A is 1, 1, 2, 1, 3. The delay time Y can be divided into 5 stages of 1 to 5, for example. The larger the value of the delay time Y, the longer the delay time can be expressed. Further, it is assumed that the anxiety level Z for each of the voices 1, 2, 3, 4, and 5 of the subject A is 1, 2, 3, 1, 5. The anxiety level Z can be divided into 5 stages of 1 to 5, for example. The larger the value of the anxiety level Z, the stronger the anxiety of the subject A can be shown.

The feature level X represents the difficulty of hearing for the hearing impaired. Therefore, when the subject A is not deaf, the delay time Y and the anxiety level Z of the subject A do not depend on the feature level X of the interlocutor's voice. On the other hand, when the subject A has hearing loss, the delay time Y and the anxiety level Z of the subject A depend on the voice feature level X of the interlocutor. Specifically, when the subject A has hearing loss, the larger the feature level X, the larger the corresponding delay time Y and the anxiety level Z. This tendency becomes stronger as the degree of deafness of subject A becomes stronger (the level of deafness becomes higher).

Therefore, the deafness level determination unit 58 determines the deafness level higher as the correlation of the delay time Y with respect to the feature level X is stronger. Similarly, the deafness level determination unit 58 determines the deafness level higher as the correlation between the feature level X and the anxiety level Z is stronger. The deafness level is, for example, a correlation coefficient of at least one of the delay time Y and the anxiety level Z with respect to the feature level X, but is not limited to this.

The cognitive function level determination unit 59 determines the cognitive function level of the subject based on the dialogue voice acquired via the communication unit 52. For determining the cognitive function level, for example, the voice feature amount of the dialogue voice (for example, the pitch related to the pitch of the voice, the formant frequency related to the characteristics of vowels and consonants, and the mel frequency spectrum coefficient related to the vocal tract characteristic (for example). It can be done based on MFCC) etc.). For the determination of the cognitive function level, for example, a learning model such as a rule base, a support vector machine (SVM) which is a method of machine learning, or a neural network can be used.

Further, the cognitive function level determination unit 59 can determine the cognitive function of the subject based not only on the voice of the subject but also on the voice of the interlocutor. That is, not only the voice of the subject but also the voice of the interlocutor who interacts with the subject can be an element for determining the cognitive function. Since the response such as the answer of the subject to the utterance of the interlocutor's question can be used to judge the cognitive function of the subject, it is possible to judge how the subject responds to the question of the person. It can be used as a material, and the accuracy of determination of cognitive function can be improved.

Next, the details of the cognitive function level determination unit 59 will be described.

FIG. 6 is a schematic diagram showing a first example of the configuration of the cognitive function level determination unit 59. As shown in FIG. 6, the cognitive function level determination unit 59 includes a voice feature amount extraction unit 591 and a DNN (Deep Neural Network) 592. The voice feature amount extraction unit 591 is based on the voice waveform of the interlocutor and the voice waveform of the target person (for example, a voice waveform that combines the question of the interlocutor and the answer of the target person to the question). The feature amount (for example, pitch, formant frequency, mel frequency spectrum coefficient (MCFF)) is extracted, and the voice feature amount of the interlocutor (for example, pitch, formant frequency, mel frequency spectrum coefficient (MCFF)) is extracted. Of the three elements of speech (prosody, sound quality, and phonology), prosody is known to be particularly important nonverbal information for identifying cognitive dysfunction. Therefore, DNN592 can be learned by using the pitch, formant frequency, and Mel frequency spectrum coefficient as the speech features that characterize the prosody.

An identification flag can be input to the voice feature amount extraction unit 591. The identification flag can be a target person flag and an interlocutor flag. For example, when the voice of the interlocutor is input to the voice feature amount extraction unit 591, the interlocutor flag may be continuously input while the voice is being input, and the interlocutor flag may be input at the start and end of the voice of the interlocutor. May be entered. When the voice of the target person is input to the voice feature amount extraction unit 591, the target person flag may be continuously input while the voice is being input, and the target person flag is input at the start and end of the voice of the target person. You may. As a result, the voice feature amount extraction unit 591 distinguishes between the target person and the interlocutor regardless of whether the voice of only the target person is input or the voice of the target person and the voice of the interlocutor are repeatedly input in order. Can be identified. The voice feature amount extraction unit 591 inputs the extracted feature amount of the target person and the feature amount of the interlocutor into the DNN 592.

Further, the response time such as the answer of the target person to the utterance of the interlocutor's question or the like may be input to the DNN 592. DNN592 is generated using learning data including the response times of the healthy and cognitively impaired persons to the utterances of the interlocutors who interact with the healthy and cognitively impaired persons. The response time can be the time from the end of the interlocutor's utterance to the start of the response of the healthy person and the cognitively impaired person. Since it is considered that the response time tends to increase as the cognitive function declines, the accuracy of determining the cognitive function of DNN592 can be improved by including the response time in the learning data.

When a person's voice is input, the DNN 592 can determine the cognitive function level of the person. In the example of FIG. 6, the cognitive function level (level of cognitive dysfunction) is divided into m from level "1" to level "m". The cognitive function level m corresponds to severe cognitive dysfunction, and the smaller the numerical value indicating the level, the less the cognitive dysfunction.

The cognitive function level determination unit 59 may include a learning processing unit for generating (learning) DNN592. The learning processing unit may be another computer. The cognitive function level determination unit 59 gives voice data of each of the healthy person and the cognitively impaired person to the input layer, and outputs the cognitive function level of each of the healthy person and the cognitively impaired person corresponding to the voice data given to the input layer to the output layer. Can be given and generated. In this case, a voice feature amount (for example, pitch, formant frequency, mel frequency spectrum coefficient, etc.) can be extracted from the voice data of a healthy person and a cognitively impaired person, and the extracted voice feature amount can be used as learning data. Further, the cognitive function level may be numerically divided into 5 stages such as 1 to 5 (m = 5 in the example of FIG. 6), and may be divided into 3 stages such as normal, mild dementia and severe dementia. It may be divided into two stages such as normal and dementia. Thereby, the DNN 592 can determine the cognitive function level of the subject based on the voice.

Further, the DNN 592 gives the voice data of the interlocutor to the input layer in addition to the voices of the healthy person and the cognitively impaired person, and the cognitive function of each of the healthy person and the cognitively impaired person corresponding to the voice data given to the input layer. It can be generated by giving a level to the output layer.

When learning DNN592, the hearing loss level of a healthy person and a cognitively impaired person may be given as a teacher label as learning data. As a result, the DNN 592 can determine the cognitive function level in consideration of the deafness level of the subject, so that the correction unit 60 does not have to be provided.

FIG. 7 is a schematic diagram showing a second example of the configuration of the cognitive function level determination unit 59. As shown in FIG. 7, the cognitive function level determination unit 59 includes an RNN (Recurrent Neural Network) 593. As shown in FIG. 7, when the voice waveform of the interlocutor and the voice waveform of the subject (for example, the voice waveform that combines the question of the interlocutor and the answer of the subject to the question) are input to the RNN593, the RNN593 Can output the cognitive function level of the subject. In the example of FIG. 7, the cognitive function level is divided into m from level "1" to level "m". The cognitive function level m corresponds to severe cognitive dysfunction, and the smaller the numerical value indicating the level, the less the cognitive dysfunction. Further, when inputting the voice waveform of the target person and the voice waveform of the interlocutor, a speaker flag indicating which speaker's voice waveform is used may be input to the RNN563.

An identification flag can be input to the RNN593. The identification flag can be a target person flag and an interlocutor flag. For example, when the interlocutor's voice is input to RNN593, the interlocutor flag may be continuously input while the voice is input, or the interlocutor flag may be input at the start and end of the interlocutor's voice. good. When the voice of the target person is input to the RNN593, the target person flag may be continuously input while the voice is input, or the target person flag may be input at the start and end of the voice of the target person. Thereby, the RNN593 can distinguish between the target person and the interlocutor regardless of whether the voice of the target person only is input or the voice of the target person and the voice of the interlocutor are repeatedly input in order. .. The interlocutor flag does not have to be entered. For example, if only the voice of the target person is input to the RNN593, or if the voice of the target person and the voice of the interlocutor are identified in advance, the interlocutor flag is unnecessary.

The cognitive function level determination unit 59 can generate the trained RNN593 using the learning data. The RNN593 is generated by giving the voice data of each of the healthy person and the cognitively impaired person to the input layer and giving the cognitive function level of each of the healthy person and the cognitively impaired person corresponding to the voice data given to the input layer to the output layer. Can be done. In this case, the voice data as the learning data may be voice data of a healthy person and a cognitively impaired person, or may be voice data of both a healthy person and a cognitively impaired person and an interlocutor. The voice data can be directly used as learning data as it is. Further, the speaker flag may be input to the RNN593 for learning. The RNN593 can determine the cognitive function level of the subject based on the input voice.

Further, the RNN593 gives the voice data of the interlocutor to the input layer in addition to the voices of the healthy person and the cognitively impaired person, and the cognitive function of each of the healthy person and the cognitively impaired person corresponding to the voice data given to the input layer. It can be generated by giving a level to the output layer.

FIG. 8 is a schematic diagram showing a third example of the configuration of the cognitive function level determination unit 59. As shown in FIG. 8, the cognitive function level determination unit 59 includes an FFT conversion unit 595 and a CNN (Convolutional Neural Network) 596. The FFT (Fast Fourier Transform) transform unit 595 converts the interlocutor's voice waveform and the subject's voice waveform (for example, a voice waveform that combines the interlocutor's question and the subject's answer to the question). It is converted into a spectrogram, and the converted spectrograms of the subject and the interlocutor are output to CNN596. The spectrogram is a two-dimensional map, the vertical axis shows the frequency, the horizontal axis shows the time, and the amplitude (strength) of the frequency at that point depends on the brightness or color of each point (coordinates) in the two dimensions. ) Can be expressed. The spectrogram can show what frequency components are included in the speech waveforms of the interlocutor and the subject.

An identification flag can be input to the FFT conversion unit 595. The identification flag can be a target person flag and an interlocutor flag. For example, when the voice of the interlocutor is input to the FFT conversion unit 595, the interlocutor flag may be continuously input while the voice is being input, and the interlocutor flag is input at the start and end of the voice of the interlocutor. You may. When the voice of the target person is input to the FFT conversion unit 595, the target person flag may be continuously input while the voice is being input, and the target person flag is input at the start and end of the voice of the target person. May be good. As a result, the FFT converter 595 distinguishes between the target person and the interlocutor regardless of whether the voice of only the target person is input or the voice of the target person and the voice of the interlocutor are repeatedly input in order. be able to. The CNN596 can output the cognitive function level of the subject when the spectrogram is input. In the example of FIG. 8, the cognitive function level is divided into m from level "1" to level "m".

The cognitive function level determination unit 59 can generate the trained CNN596 using the learning data. CNN596 can generate a spectrogram converted from voice data of a healthy person and a cognitively impaired person and a cognitive function level of the healthy person and the cognitively impaired person by using the learning data. If the audio waveform is captured as a two-dimensional map instead of the spectrogram, the two-dimensional map can indicate the presence or absence of an audio signal at each point (coordinates) by the brightness or color of each point. Therefore, the voice waveform captured as a two-dimensional map may be input to CNN564.

Further, the CNN596 includes a spectrogram converted from the voice data of the interlocutor's voice in addition to the spectrogram converted from the voice data of the healthy person and the cognitively impaired person, and the cognition of the healthy person and the cognitively impaired person. The functional level and can be generated using the training data.

In the present embodiment, the determination of the cognitive function may use any of the configurations illustrated in FIGS. 6 to 8 or may combine the configurations. For example, the cognitive function may be determined using both the configurations of FIGS. 6 and 7, and the cognitive function may be determined using both the configurations of FIGS. 6 and 8. When combining the configurations, the determination results of each configuration may be comprehensively determined and used as the final determination.

As described above, DNN592, RNN593, and CNN596 are generated using learning data including voice data of interlocutors interacting with healthy and cognitively impaired persons. Not only the voices of healthy persons and cognitively impaired persons, but also the voices of interlocutors who interact with healthy persons and cognitively impaired persons can be a factor for determining cognitive function. That is, since the responses such as the answers of the healthy person and the cognitively impaired person to the utterance of the interlocutor's question can be used to judge the cognitive function, which of the healthy person and the cognitively impaired person responds to the human question. It is possible to learn how to react in this way, and it is possible to improve the accuracy of determining the cognitive function of DNN592, RNN593, and CNN596.

In the present embodiment, the DNN592, RNN593, and CNN596 can relearn the cognitive function level of the subject determined by themselves using the learning data updated to the cognitive function level determined by the doctor. For example, assume that DNN592 determines the cognitive function level of a subject as level "3". If the doctor determines that the subject's cognitive function level is level "4" by medical examination, the DNN592 can be relearned using the learning data with the subject's voice and cognitive function level updated to "4". can. The same applies to RNN593 and CNN596. This makes it possible to improve the accuracy of determining the cognitive function of DNN592, RNN593, and CNN596.

Next, a method for correcting the cognitive function level will be described.

The correction unit 60 can correct the cognitive function level determined by the cognitive function level determination unit 59 based on the deafness level determined by the deafness level determination unit 58. If the subject has a high level of deafness, it is highly likely that the subject's dementia level is determined to be higher than the original dementia level due to the effects of deafness. Therefore, the correction unit 60 corrects the cognitive function level of the subject having a high level of deafness so as to be low (mild).

FIG. 9 is an explanatory diagram showing an example of a method for correcting the cognitive function level. As shown in the figure, the cognitive function level is represented by numerical values 1 to 5, and the deafness level is represented by numerical values 1 to 5. The larger the cognitive function level and the deafness level, the more severe the condition. The values in each cell of FIG. 9 indicate the corrected cognitive function level. Further, in the figure, for convenience, the area representing the corrected cognitive function level is divided into four.

The region where the cognitive function level before correction is 1 to 3 and the deafness level is 1 to 3 (no pattern in the figure) is, for example, a region of non-dementia / non-deafness. Since the effect of deafness on the cognitive function level is small, the correction unit 60 does not correct the cognitive function level.

The region where the cognitive function level before correction is 1 to 3 and the hearing loss level is 4 to 5 (left diagonal pattern in the figure) is, for example, a region of non-dementia / deafness. Although there is an effect on the cognitive function level due to deafness, the correction unit 60 does not correct the cognitive function level because it is determined to be non-dementia.

The region where the cognitive function level before correction is 4 to 5 and the hearing loss level is 1 to 3 (right diagonal pattern in the figure) is, for example, a dementia / non-deafness region. Since the effect of deafness on the cognitive function level is small, the correction unit 60 does not correct the cognitive function level.

The region where the cognitive function level before correction is 4 to 5 and the hearing loss level is 4 to 5 (net pattern in the figure) is, for example, a region of dementia / deafness. Since deafness has an effect on the cognitive function level, the correction unit 60 corrects the cognitive function level. Specifically, the correction unit 60 reduces the cognitive function level by 0.5 when the deafness level is 4, and reduces the cognitive function level by 1 when the deafness level is 5.

In this case, if the subject who had a cognitive function level of 4 before the correction and was determined to have dementia had a deafness level of 5, the corrected cognitive function level would be corrected from 4 to 3. It can be determined that there is no dementia. The numerical change at the time of correction is not limited to the example of FIG.

As described above, when the subject is determined to have dementia and hearing loss, the subject is corrected by correcting the determined cognitive function level according to the level of hearing loss. It may be possible to determine that you do not have dementia. This makes it possible to identify a subject who has been determined to have dementia due to deafness and prevent the subject from being erroneously determined to have dementia.

Next, a method of displaying the determination result will be described.

The output unit 61 can output display data for displaying the cognitive function level determined by the cognitive function level determination unit 59 and the deafness level determined by the deafness level determination unit 58 in association with each other.

FIG. 10 is a schematic diagram showing a first example of a determination result display method. The deafness determination device 50 can record the history of the determination result of the deafness level and the cognitive function level of the subject in the storage unit 56. In addition, instead of the storage unit 56, it may be recorded in an external database. As shown in FIG. 10, it is divided into two categories according to the cognitive function level, dementia and non-dementia, and according to the deafness level, it is divided into two categories, deafness and non-deafness. As a result, the judgment results can be divided into four areas: non-dementia / deafness, non-dementia / non-deafness, dementia / deafness, and dementia / non-deafness, and the relationship between dementia and deafness can be easily identified. can do. Further, the output unit 61 can output the determined cognitive function level and the corrected cognitive function level in association with each other. In the example of FIG. 10, the subject determined to have dementia before the correction has the cognitive function level corrected by considering the deafness level of the subject, and is determined not to have dementia. In this way, although the subject was determined to have dementia, it is possible to easily confirm whether or not the subject actually has dementia.

FIG. 11 is a schematic diagram showing a second example of a determination result display method. The output unit 61 can output display data for displaying at least one of the cognitive function level and the deafness level of the subject in chronological order. In the example of FIG. 11, it can be seen that the subject does not have dementia, but the level of deafness gradually increases over the years. In the figure, the solid line part shows the actual results from the past to the present, and the broken line part shows the future forecast. In the example of FIG. 11, it is possible to easily confirm how the deafness level of the subject changes. The subject can take measures against deafness at an early stage.

FIG. 12 is a schematic diagram showing a third example of a determination result display method. In the example of FIG. 12, although the subject does not have dementia, it can be seen that the level of deafness gradually increases over the years and the level of cognitive function also increases. In the figure, the solid line part shows the actual results from the past to the present, and the broken line part shows the future forecast. In the example of FIG. 12, it is possible to easily confirm how the deafness level and the cognitive function level of the subject change. The subject can take measures against deafness and dementia at an early stage.

FIG. 13 is a schematic diagram showing a fourth example of a determination result display method. In the example of FIG. 13, the statistical distribution is shown so that the relationship between the cognitive function level and the deafness level can be understood by age group. In the figure, the statistical distributions of the 50s, 60s, 70s, and 80s are shown. In addition, the positions corresponding to the cognitive function level and the deafness level of the subject (“you” in the figure) are also displayed. From this, it is possible to know how much the subject corresponds to the statistical value of the age compared to his / her own age, and it is possible to easily determine whether dementia or deafness is more advanced than that of the same age. ..

FIG. 14 is a flowchart showing an example of the processing procedure of the deafness determination system. The terminal device 10 acquires the dialogue voice (S11) and transmits the acquired dialogue voice to the deafness determination device 50 (S12). The deafness determination device 50 receives the dialogue voice (S13) and discriminates between the voice of the dialogue person and the voice of the target person (S14). The deafness determination device 50 extracts the feature amount of each voice of the interlocutor (S15), calculates the delay time of each answer of the subject (S16), and determines the anxiety level of each answer of the subject (S17). ..

The deafness determination device 50 determines the deafness level of the subject (S18) and determines the cognitive function level of the subject (S19). The deafness determination device 50 corrects the cognitive function level of the subject based on the determined deafness level (S20), transmits data associated with the deafness level and the cognitive function level to the terminal device 10 (S21), and performs processing. finish. The terminal device 10 receives and displays data associated with the deafness level and the cognitive function level (S22), and ends the process.

The deafness determination device 50 can also be realized by using a general-purpose computer including a CPU (processor), a GPU, a RAM (memory), and the like. That is, as shown in FIG. 14, a computer program that defines the procedure for each process is loaded into a RAM (memory) provided in the computer, and the computer program is executed by the CPU (processor) to determine hearing loss on the computer. The device 50 can be realized. The computer program may be recorded and distributed on a recording medium. The learned DNN592, RNN593, and CNN596 may be generated by another server or the like having a learning processing unit, and downloaded to the deafness determination device 50.

The deafness determination device 50 of the present embodiment can be incorporated into various devices. For example, by incorporating a microphone and the hearing loss determination device 50 of the present embodiment into the hearing aid, it is possible to realize a hearing aid capable of determining cognitive function. By acquiring the subject's voice with the microphone of the hearing aid, it is possible to determine not only the deafness level but also the cognitive function level. The determination result can be displayed on a display display (for example, a smartphone, a tablet, a personal computer, a TV, etc.) capable of wireless communication with the hearing aid.

Further, the hearing loss determination device 50 of the present embodiment can be incorporated into a robot or a smart speaker. By interacting with the subject, the robot or smart speaker can acquire the subject's voice and determine the hearing loss level and the cognitive function level. In this case, the utterance of the robot or the smart speaker can acquire the reaction of the target person by outputting the voices of both the difficult-to-hear and easy-to-hear speeches, for example. The determination result may be output to the target person's mobile terminal (for example, a smartphone or tablet), or the determination result may be notified by voice. Such robots can be installed in hospitals, clinics, government offices, stores, and the like. Further, by installing the smart speaker at the home of the target person or family, for example, a watching service can be realized.

In addition, the deafness determination device 50 of the present embodiment is incorporated into a smartphone, tablet, personal computer, camera, or the like, and when the subject makes a videophone, the voice is acquired and the deafness level and the cognitive function level are also determined. Can be done. The determination result may be recorded in a smartphone, tablet, personal computer, or camera, and may be displayed or output as needed or periodically. As a result, the subject can check the history of his / her deafness level and cognitive function level at any time.

The deafness determination device of the present embodiment has an acquisition unit that acquires a dialogue voice between the target person and the interlocutor, and an extraction unit that extracts a feature amount related to the voice of the interlocutor based on the dialogue voice acquired by the acquisition unit. And the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the delay calculated by the calculation unit. It is provided with a deafness level determination unit that determines the deafness level of the subject based on time.

The deafness determination system of the present embodiment has an acquisition unit that acquires a dialogue voice between the target person and the interlocutor, and an extraction unit that extracts a feature amount related to the voice of the interlocutor based on the dialogue voice acquired by the acquisition unit. And the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the delay calculated by the calculation unit. It is provided with a deafness level determination unit that determines the deafness level of the subject based on time.

The computer program of the present embodiment has acquired a process of acquiring a dialogue voice between the target person and the interlocutor and a process of extracting a feature amount related to the voice of the interlocutor based on the acquired dialogue voice. A process of calculating the delay time of the subject's response to the interlocutor's utterance based on the dialogue voice and a process of determining the hearing loss level of the subject based on the extracted feature amount and the calculated delay time are executed. Let me.

The hearing loss determination method of the present embodiment acquires the dialogue voice between the target person and the interlocutor, extracts the feature amount related to the voice of the interlocutor based on the acquired dialogue voice, and is based on the acquired dialogue voice. The delay time of the response of the subject to the speech of the interlocutor is calculated, and the hearing loss level of the subject is determined based on the extracted feature amount and the calculated delay time.

The acquisition unit acquires the dialogue voice between the target person and the interlocutor. The subject is a subject who determines deafness. The interlocutor is a person who interacts with the subject, and includes, for example, a doctor, a nurse, a counselor, a caregiver, and the like. The extraction unit extracts the feature amount related to the voice of the interlocutor based on the acquired dialogue voice. The feature amount may be any feature amount that can express the difficulty of hearing for a hearing-impaired person.

The calculation unit calculates the delay time of the target person's response to the interlocutor's utterance based on the acquired dialogue voice. The delay time can be the time from the end of the interlocutor's utterance to the start of the subject's response. Deaf people tend to have longer delays than healthy people. The determination unit determines the hearing loss level of the subject based on the extracted feature amount and the calculated delay time. That is, the level of deafness of the subject is determined according to how difficult it is for the subject to hear the interlocutor's utterance and how much the subject's utterance is delayed with respect to the interlocutor's utterance. , The deafness of the subject can be determined.

The hearing loss determination device of the present embodiment includes an anxiety level determination unit that determines the anxiety level of the subject with respect to the utterance of the interlocutor based on the dialogue voice acquired by the acquisition unit. The hearing loss level of the subject is determined based on the anxiety level determined by the anxiety level determination unit.

The anxiety level determination unit determines the anxiety level of the subject with respect to the utterance of the interlocutor based on the acquired dialogue voice. The magnitude of the anxiety level corresponds to the magnitude of the self-confidence level. For the determination of the anxiety level, for example, the voice volume of the subject's utterance, the specificity and clarity of the utterance content, and the like can be used.

The deafness level determination unit determines the deafness level of the subject based on the determined anxiety level. For example, when the level of anxiety is high, it can be determined that the level of deafness is high because it is considered that the cause is that the interlocutor's utterance cannot be heard or is difficult to hear. As a result, it is possible to accurately determine the deafness of the subject.

The anxiety level determination unit determines the anxiety level based on at least one of the subject's voice volume and the content of the answer. For example, when the subject's voice is low, it can be determined that the anxiety level is high. In addition, when the utterance content of the subject is unclear or unclear, it can be determined that the anxiety level is high.

In the hearing loss determination device of the present embodiment, the extraction unit extracts at least one of the power, time, number of times, or frequency in a high frequency range above a predetermined threshold value included in the voice of the interlocutor as the feature amount. do.

The extraction unit extracts at least one of consonants or power, time, number of times, or frequency in a high frequency range equal to or higher than a predetermined threshold value included in the voice of the interlocutor as a feature amount. The power can be the intensity of consonants or speech in the high frequency range above the threshold. The time can be the time during which a consonant or a sound in a high frequency range equal to or higher than the threshold value is output. The number of times indicates how many times a word containing a consonant or a high frequency range equal to or higher than a threshold value appears in one phrase of the interlocutor's utterance. Frequency indicates how many times a word containing a consonant or a high frequency range above a threshold value appears per unit time.

The hearing loss determination device of the present embodiment is based on a cognitive function determination unit that determines the cognitive function level of the subject based on the dialogue voice acquired by the acquisition unit and a hearing loss level determined by the hearing loss level determination unit. A correction unit for correcting the cognitive function level determined by the cognitive function determination unit is provided.

The cognitive function determination unit determines the cognitive function level of the subject based on the acquired dialogue voice. For determining the cognitive function level, for example, the voice feature amount of the dialogue voice (for example, the pitch related to the pitch of the voice, the formant frequency related to the characteristics of vowels and consonants, and the mel frequency spectrum coefficient related to the vocal tract characteristic (for example). It can be done based on MFCC) etc.). For the determination of the cognitive function level, for example, a learning model such as a rule base, a support vector machine (SVM) which is a method of machine learning, or a neural network can be used.

The correction unit corrects the cognitive function level determined by the cognitive function determination unit based on the deafness level determined by the deafness level determination unit. For example, if the subject is determined to have dementia and has hearing loss, the subject may have dementia by correcting the determined cognitive function level according to the level of hearing loss. It may be possible to determine that there is no such thing. This makes it possible to identify a subject who has been determined to have dementia due to deafness and prevent the subject from being erroneously determined to have dementia.

The deafness determination device of the present embodiment includes an output unit that outputs display data for displaying the cognitive function level determined by the cognitive function determination unit in association with the deafness level determined by the deafness level determination unit. ..

The output unit outputs display data for displaying the cognitive function level determined by the cognitive function determination unit in association with the deafness level determined by the deafness level determination unit. For example, it is divided into two categories, dementia and non-dementia, according to the cognitive function level, and divided into two categories, deafness and non-deafness, according to the level of deafness. As a result, the judgment results can be divided into four areas: non-dementia / deafness, non-dementia / non-deafness, dementia / deafness, and dementia / non-deafness, and the relationship between dementia and deafness can be easily identified. can do.

In the deafness determination device of the present embodiment, the output unit outputs display data for displaying at least one of the cognitive function level and the deafness level of the subject in time series.

The output unit outputs display data for displaying at least one of the cognitive function level and the deafness level of the subject in chronological order. This makes it possible to easily confirm how at least one of the cognitive function level and the deafness level of the subject changes. The subject can take measures against deafness or dementia at an early stage.

In the deafness determination device of the present embodiment, the output unit outputs the cognitive function level determined by the cognitive function determination unit in association with the cognitive function level corrected by the correction unit.

The output unit outputs the determined cognitive function level in association with the corrected cognitive function level. As a result, although the subject was determined to have dementia, it is possible to easily confirm whether the implementation is not dementia or also dementia.

1 Communication network 10 Terminal device 11 Microphone 50 Deafness judgment device 51 Control unit 52 Communication unit 53 Voice identification unit 54 Feature extraction unit 55 Delay time calculation unit 56 Storage unit 57 Anxiety level determination unit 58 Deafness level determination unit 59 Cognitive function level determination Part 591 Voice feature extraction part 592 DNN
593 RNN
595 FFT converter 596 CNN
60 Correction unit 61 Output unit

The present invention relates to a deafness determination device, a deafness determination system, a computer program, and a cognitive function level correction method.

The present invention has been made in view of such circumstances, and an object of the present invention is to provide a deafness determination device, a deafness determination system, a computer program, and a cognitive function level correction method capable of determining a subject's deafness. ..

The deafness determination device according to the embodiment of the present invention extracts a feature amount related to the voice of the interlocutor based on the acquisition unit that acquires the dialogue voice between the target person and the interlocutor and the dialogue voice acquired by the acquisition unit. The extraction unit, the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the calculation unit. It is provided with a deafness level determination unit that determines the deafness level of the subject based on the calculated delay time, and a correction unit that corrects the cognitive function level of the subject based on the deafness level determined by the deafness level determination unit. ..

The deafness determination system according to the embodiment of the present invention extracts a feature amount related to the voice of the interlocutor based on the acquisition unit that acquires the dialogue voice between the target person and the interlocutor and the dialogue voice acquired by the acquisition unit. The extraction unit, the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the calculation unit. It is provided with a deafness level determination unit that determines the deafness level of the subject based on the calculated delay time, and a correction unit that corrects the cognitive function level of the subject based on the deafness level determined by the deafness level determination unit. ..

The computer program according to the embodiment of the present invention includes a process of acquiring a dialogue voice between the target person and the interlocutor and a process of extracting a feature amount related to the voice of the interlocutor based on the acquired dialogue voice. , A process of calculating the delay time of the subject's response to the interlocutor's utterance based on the acquired dialogue voice, and a process of determining the deafness level of the subject based on the extracted feature amount and the calculated delay time. And the process of correcting the cognitive function level of the subject based on the determined deafness level .

In the cognitive function level correction method according to the embodiment of the present invention, the acquisition unit acquires the dialogue voice between the subject and the interlocutor, and the feature amount related to the voice of the interlocutor is extracted based on the acquired dialogue voice. Is extracted, and the calculation unit calculates the delay time of the response of the subject to the speech of the interlocutor based on the acquired dialogue voice, and the subject is based on the extracted feature amount and the calculated delay time. the deafness level determined hearing loss level determination unit, correction unit cognitive function level of the subject based on the determined hearing loss level is corrected.

The deafness determination device according to the embodiment of the present invention extracts a feature amount related to the voice of the interlocutor based on the acquisition unit that acquires the dialogue voice between the target person and the interlocutor and the dialogue voice acquired by the acquisition unit. The extraction unit, the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the calculation unit. A deafness level determination unit that determines the deafness level of the subject based on the calculated delay time, a correction unit that corrects the cognitive function level of the subject based on the deafness level determined by the deafness level determination unit, and the above. The correction unit includes a cognitive function determination unit that determines the cognitive function level of the subject using at least one of the voice feature amount, the voice waveform, and the voice waveform of the dialogue voice of the target person or the interlocutor. is that to correct the cognitive level determined by the cognitive function determination unit based on the hearing loss level determined by the hearing level determination unit.

The deafness determination system according to the embodiment of the present invention extracts a feature amount related to the voice of the interlocutor based on the acquisition unit that acquires the dialogue voice between the target person and the interlocutor and the dialogue voice acquired by the acquisition unit. The extraction unit, the calculation unit that calculates the delay time of the response of the target person to the speech of the interlocutor based on the dialogue voice acquired by the acquisition unit, the feature amount extracted by the extraction unit, and the calculation unit. A deafness level determination unit that determines the deafness level of the subject based on the calculated delay time, a correction unit that corrects the cognitive function level of the subject based on the deafness level determined by the deafness level determination unit, and the above. The correction unit includes a cognitive function determination unit that determines the cognitive function level of the subject using at least one of the voice feature amount, the voice waveform, and the voice waveform of the dialogue voice of the target person or the interlocutor. is that to correct the cognitive level determined by the cognitive function determination unit based on the hearing loss level determined by the hearing level determination unit.

The computer program according to the embodiment of the present invention includes a process of acquiring a dialogue voice between a target person and an interlocutor on a computer, and a process of extracting a feature amount related to the voice of the interlocutor based on the acquired dialogue voice. , A process of calculating the delay time of the subject's response to the interlocutor's speech based on the acquired dialogue voice, and a process of determining the hearing loss level of the subject based on the extracted feature amount and the calculated delay time. And the process of determining the cognitive function level of the subject using at least one of the voice feature amount of the dialogue voice of the subject or the interlocutor, the voice waveform, and the spectrogram obtained by converting the voice waveform, and the determined hearing loss level. Based on this, the process of correcting the cognitive function level of the subject is executed.

In the cognitive function level correction method according to the embodiment of the present invention, the acquisition unit acquires the dialogue voice between the subject and the interlocutor, and the feature amount related to the dialogue voice of the interlocutor is extracted based on the acquired dialogue voice. Is extracted, and the calculation unit calculates the delay time of the response of the subject to the speech of the interlocutor based on the acquired dialogue voice, and the subject is based on the extracted feature amount and the calculated delay time. The hearing loss level is determined by the hearing loss level determination unit , and the cognitive function level of the subject is determined by using at least one of the voice feature amount, the voice waveform, and the voice waveform of the dialogue voice of the subject or the interlocutor. The cognitive function determination unit determines, and the correction unit corrects the cognitive function level of the subject based on the determined hearing loss level.

Claims (11)

The acquisition unit that acquires the dialogue voice between the target person and the interlocutor,
An extraction unit that extracts features related to the voice of the interlocutor based on the dialogue voice acquired by the acquisition unit, and an extraction unit.
A calculation unit that calculates the delay time of the response of the target person to the utterance of the interlocutor based on the dialogue voice acquired by the acquisition unit, and a calculation unit.
A deafness determination device including a deafness level determination unit that determines the deafness level of the subject based on the feature amount extracted by the extraction unit and the delay time calculated by the calculation unit. It is provided with an anxiety level determination unit that determines the anxiety level of the target person with respect to the utterance of the interlocutor based on the dialogue voice acquired by the acquisition unit.
The deafness level determination unit
The deafness determination device according to claim 1, wherein the deafness level of the subject is determined based on the anxiety level determined by the anxiety level determination unit. The anxiety level determination unit
The deafness determination device according to claim 2, wherein the anxiety level is determined based on at least one of the subject's voice volume and the content of the answer. The extraction unit
Any one of claims 1 to 3 that extracts at least one of a consonant or a power, time, number of times, or frequency in a high frequency range equal to or higher than a predetermined threshold value included in the voice of the interlocutor as the feature amount. The hearing loss determination device described in 1. A cognitive function determination unit that determines the cognitive function level of the subject based on the dialogue voice acquired by the acquisition unit, and
The deafness determination device according to any one of claims 1 to 4, further comprising a correction unit that corrects the cognitive function level determined by the cognitive function determination unit based on the deafness level determined by the deafness level determination unit. .. The deafness determination device according to claim 5, further comprising an output unit that outputs display data for displaying the cognitive function level determined by the cognitive function determination unit in association with the deafness level determined by the deafness level determination unit. The output unit
The deafness determination device according to claim 6, which outputs display data for displaying at least one of the cognitive function level and the deafness level of the subject in time series. The output unit
The deafness determination device according to claim 6 or 7, wherein the cognitive function level determined by the cognitive function determination unit and the cognitive function level corrected by the correction unit are output in association with each other. The acquisition unit that acquires the dialogue voice between the target person and the interlocutor,
An extraction unit that extracts features related to the voice of the interlocutor based on the dialogue voice acquired by the acquisition unit, and an extraction unit.
A calculation unit that calculates the delay time of the response of the target person to the utterance of the interlocutor based on the dialogue voice acquired by the acquisition unit, and a calculation unit.
A deafness determination system including a deafness level determination unit that determines the deafness level of the subject based on the feature amount extracted by the extraction unit and the delay time calculated by the calculation unit. On the computer
The process of acquiring the dialogue voice between the target person and the interlocutor,
A process of extracting features related to the voice of the interlocutor based on the acquired dialogue voice, and
A process of calculating the delay time of the subject's response to the interlocutor's utterance based on the acquired dialogue voice, and
A computer program that executes a process of determining the deafness level of the subject based on the extracted features and the calculated delay time. Acquires the dialogue voice between the target person and the interlocutor,
Based on the acquired dialogue voice, the feature amount related to the voice of the interlocutor is extracted.
Based on the acquired dialogue voice, the delay time of the response of the subject to the utterance of the interlocutor is calculated.
Deafness level of the subject is determined based on the extracted features and the calculated delay time.
Deafness determination method.

Images