JP2021058573A - Cognitive function prediction device, cognitive function prediction method, program and system - Google Patents

Abstract

To provide a device, method, program and system for predicting the state or symptom due to the reduction in a cognitive function.SOLUTION: A cognitive function prediction device comprises: an information transmission unit 11 which transmits voice information and image information to a user; a measurement unit 12 which measures a user voice and/or user image; a feature data creation unit 131 which creates at least one kind of feature data from voice feature data, language feature data and image feature data on the basis of the measurement result obtained in the measurement unit; a feature amount extraction unit 132 which extracts a feature amount from the feature data; and a cognitive function prediction unit 133 which predicts whether or not the user has a tendency of the state or symptom due to the reduction in the cognitive function on the basis of the feature amount. The information transmitted from the information transmission unit includes an atypical question. The atypical question includes a question about a past event according to the user's age.SELECTED DRAWING: Figure 1

Description

The present invention relates to a cognitive function prediction device, a cognitive function prediction method, a program and a system.

In recent years, the aging of the population has progressed in developed countries. In particular, the aging rate of Japan (the ratio of the total population aged 65 and over) was 27.5% in 2017, according to a survey by the Ministry of Health, Labor and Welfare. We are entering society. The aging rate is expected to increase in the future, and medical measures for the elderly have become an important issue. One of them is dementia. Dementia refers to a condition in which cognitive functions such as memory, thinking, and behavior are impaired, which interferes with daily life. Currently, no effective treatment for conditions or symptoms caused by cognitive decline, including dementia, has been established, and post-diagnosis support such as future planning by patients and their families is required. For that purpose, early detection of conditions or symptoms caused by cognitive decline is important.

Early detection of conditions or symptoms caused by cognitive decline is performed by combining neuropsychological tests, blood tests, and brain imaging tests. However, this test includes an invasive test, which causes anxiety and stress to the subject, and thus places a heavy burden on the subject. Therefore, a non-invasive and easy detection method is required. Therefore, many methods have been proposed so far that are non-invasive and easily detect a condition or symptom caused by a decrease in cognitive function.

Proposed methods include research using linguistic information and research using voice information (for example, Non-Patent Documents 1, 2 and 3). Most of these are analyzes of unilateral utterances during photographic narratives and neuropsychological tests. In addition, a method of interactively detecting using an agent has also been proposed (Non-Patent Documents 4 and 5). In this method, three questions created based on a neuropsychiatric test are prepared, and detection is attempted from the voice information and linguistic information of the responses. However, the questions of the neuropsychological test could be remembered by the elderly and taken in advance.

Aramaki, E.I. , Shikata, S.K. , Miyabe, M.M. and Kinoshita, A. : Vocabulary size in speech may be an early indicator of cog-night impairment, PLOS one, Vol. 11, No. 5, p. e0155195 (2016) McKhann, G.M. M. , Knopman, D.I. S. , Chertkow, H. et al. , Hy-man, B. T. , Jack, C.I. R. , Kawas, C.I. H. , Krunk, W. et al. E. , Koroshetz, W. et al. J. , Many, J. et al. J. and Mayeux, R.M. : The diagnosis of dementia due to Alzheimer's disease: Recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease, Alzheimer's & dementia: the journal of the Alzheimer's Association, Vol .. 7, No. 3, pp. 263 {269 (2011)} Roark, B.I. , Mitchell, M.M. , Hosom, J. et al. -P. , Hollinghead, K.K. and Kaye, J.M. : Spoken language diverged massages for detecting milled cognitive impairment, IEEE transitions on audio, speech, and language. 19, No. 7, pp. 2081 {2090 (2011)} Tanaka, H.M. , Adachi, H. et al. , Ukita, N.M. , Ikeda, M.I. , Kazui, H. et al. , Kudo, T.M. and Nakamura, S.A. : Detecting Dementia Through Interactive Computer Avatars, IEEE journal of transitional engineering in health and medicine, Vol. 5, pp. 1 {11 (2017)} Mirheidari, B.M. , Blackburn, D.I. , Harkness, K.K. , Walker, T.M. , Venneri, A. , Reuber, M.D. and Christensen, H. et al. : An avatar-based system for dementia, individuals likely to develop dementia, Proc. Interspeech 2017, pp. 3147 {3151 (2017)}

In order to detect the condition or symptom caused by the cognitive decline at an early stage, it is necessary to monitor it regularly and for a long period of time. When detecting, it was necessary to make it impossible to take measures in advance.

An object of the present invention is to provide a device, a method, a program and a system for predicting a condition or symptom caused by a decrease in cognitive function.

(1) The information transmission unit for transmitting voice information and image information to the user, the measurement unit for measuring the user voice and / or the user image, and the measurement results obtained by the above measurement unit. Based on this, a feature data creation unit for creating at least one type of feature data selected from a group consisting of voice feature data, linguistic feature data, and image feature data, and a feature data creation section obtained by the above feature data creation section. Based on the feature amount extraction unit for extracting the feature amount from the feature data and at least one kind of feature amount obtained by the feature amount extraction unit, the tendency of the state or symptom caused by the cognitive decline to the user. The information transmitted from the information transmission unit includes a cognitive function prediction unit configured to predict whether or not there is an atypical question.
The above-mentioned atypical question is a cognitive function prediction device characterized in that it includes a question about a past event according to the age of the user.
(2) A question creation unit configured to create a question based on a response from a user is further provided, and the atypical question is described in (1) including a question created by the question creation unit. Cognitive function predictor.
(3) The feature amount extraction unit is based on at least one of the voice feature data and the linguistic feature data created by the feature data creation unit based on the user voice obtained by the measurement unit. , The cognitive function predictor according to (1) or (2), which is configured to extract a feature amount.
(4) The feature amount extraction unit is selected from a group consisting of a line-of-sight pattern, a facial action coding system, and a facial landmark feature created by the feature data creation unit based on the user image obtained by the measurement unit. At least one kind of image feature data, and at least one kind of feature data among the voice feature data and the linguistic feature data created by the feature data creation section based on the user voice obtained by the measurement section. The cognitive function predictor according to (1) or (2), which is configured to extract a feature amount based on the above.
(5) Based on the feature amount extracted by the feature amount extraction unit, the cognitive function prediction unit determines whether or not the user has a tendency of a state or symptom caused by cognitive decline by SVM (Support Vector). The cognitive function predictor according to any one of (1) to (4), which is configured to predict using at least one selected from the group consisting of Machine), logistic regression analysis, and deep learning.
(6) Voice feature data based on the information transmission step of transmitting voice information and image information to the user, the measurement step of measuring the user voice and / or the user image, and the measurement result of the measurement unit. , The feature amount is extracted from the feature data creation step for creating at least one kind of feature data selected from the group consisting of linguistic feature data and image feature data, and the feature data obtained in the above feature data creation step. Cognitive function prediction that predicts whether or not the user is prone to a condition or symptom caused by cognitive decline based on the feature amount extraction step and at least one kind of feature amount obtained by the feature amount extraction unit. The information transmitted from the information transmission unit includes atypical questions, and the atypical questions include questions about past events according to the age of the user. Cognitive function prediction. Method.
(7) A program for causing a computer to execute the cognitive function prediction method described in (6).
(8) A user terminal including an information transmission unit for transmitting voice information and image information to a user, and a measurement unit for measuring user voice and / or user image, and the measurement unit. A feature data creation unit for creating at least one type of feature data selected from a group consisting of voice feature data, linguistic feature data, and image feature data based on the measurement results of the above, and the above-mentioned feature data creation section. A condition or symptom caused by cognitive decline in the user based on the feature amount extraction unit for extracting the feature amount from the feature data obtained in the above and at least one kind of feature amount extracted by the feature amount extraction unit. The information transmitted from the information transmission unit includes a cognitive function prediction unit configured to predict whether or not there is a tendency of the above information transmission unit, and a cognitive function prediction device including the cognitive function prediction unit. The above atypical question is a cognitive function prediction system characterized by including questions about past events according to the age of the user.

According to the present invention, it is possible to easily predict a state or a symptom caused by an early deterioration of cognitive function.

It is a figure for demonstrating the cognitive function prediction apparatus of one Embodiment of this invention. It is a figure for demonstrating the information transmission part and the measurement part in one Embodiment of this invention. It is a figure for demonstrating the cognitive function prediction system of one Embodiment of this invention. It is a flowchart for demonstrating the cognitive function prediction method of one Embodiment of this invention. It is an example of a question used in the cognitive function predictor of one embodiment of the present invention. It is a table which showed the feature amount extracted by the feature amount extraction part in the control part. It is a graph which showed the verification result of the determination accuracy of dementia which is one kind of the state or symptom caused by the deterioration of the cognitive function in the cognitive function prediction apparatus of one Embodiment of this invention. FIG. 7 (a) is a graph showing the verification result of the prediction accuracy when SVM is adopted as the determination method to predict dementia, and FIG. 7 (b) is a graph showing the verification result of the prediction accuracy when dementia is predicted by adopting the logistic regression analysis. It is a graph which showed the verification result of the prediction accuracy in the case of predicting. It is a graph which showed the verification result of the determination accuracy of dementia which is one kind of the state or symptom caused by the deterioration of the cognitive function using only the image feature data used in one Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the following description, and can be appropriately modified without departing from the gist of the present invention.

[Cognitive function predictor]
FIG. 1 is a diagram for explaining a cognitive function prediction device according to an embodiment of the present invention.
As shown in FIG. 1, the cognitive function prediction device 10 includes an information transmission unit 11, a measurement unit 12, a control unit 13, an external information input unit 21, and a database 31. The control unit 13 includes a feature data creation unit 131, a feature amount extraction unit 132, a cognitive function prediction unit 133, and a question creation unit 134. The cognitive function prediction device 10 according to the present embodiment is configured to have a dialogue with the user, and the user is in a state caused by a deterioration of the cognitive function based on the response of the user in the dialogue. Or it is configured to predict whether it is a symptom or not.

The information transmission unit 11 is for transmitting voice information and image information to the user of the cognitive function prediction device 10 (hereinafter, simply referred to as “user”), and is, for example, a display unit 111 such as a display. And an audio output unit 112 such as a speaker. Here, an example of the information transmission unit 11 will be described with reference to the drawings. FIG. 2 is a schematic view showing an example of an information transmission unit and a measurement unit.

As shown in FIG. 2, the information transmission unit 11 includes a display unit 111 for displaying an image and an audio output unit 112 for outputting audio. The display unit 111 can display, for example, an image (avatar) 111a and / or text 111b that imitates a person. The information transmission unit 11 can output a question to the user as a voice in the voice output unit 112 in order to have a dialogue with the user. Further, the avatar 111a displayed on the display unit 111 can be moved according to the voice information output by the voice output unit 112. Therefore, when the avatar 111a is displayed on the display unit 111, the mouth and facial expression of the avatar 111a displayed on the display unit 111 can be configured to move according to the voice information output by the voice output unit 112. , You can create an environment where the user is interacting with the avatar. As a result, it is possible to alleviate the discomfort and tension of the user who answers the question transmitted from the information transmission unit 11 by using the cognitive function prediction device 10.

Further, the avatar 111a displayed on the display unit 111 can be set according to the user's preference. For example, an image imitating a family member, a pet, a friend, a celebrity, an animal, a character, or the like can be set as an avatar according to the user's preference. By displaying the avatar 111a according to the user's preference on the display unit 111 in this way, the regular use of the cognitive function prediction device 10 by the user is promoted, and the state caused by the deterioration of the cognitive function at an early stage. Or the symptom can be predicted.

Further, the voice output from the voice output unit 112 can be set according to the user's preference. For example, it is possible to change the pitch of the voice, set the voice color, the way of speaking, the speed, the volume, and the like. As a result, the optimum voice can be set for the avatar 111a according to the user's preference, the discomfort and tension of the user can be further alleviated, and the ease of listening can be improved.

The image data displayed on the display unit 111 and the audio data output from the audio output unit 112 are given by the control unit 13.
For example, the user can input favorite image data and audio data from the external information input unit 21 and / or the sound collecting unit 121, and the image data and the audio data are stored in the database 31. The control unit 13 can read the data stored in the database 31, give the image data to the display unit 111, and give the audio data to the audio output unit 112.

The measuring unit 12 includes, for example, a sound collecting unit 121 composed of a microphone or the like, and an imaging unit 122 composed of, for example, a CCD (Charge Coupled Device) image sensor, a CMOS (Complementary Metal Oxide Sensor) image sensor, or the like. The sound collecting unit 121 is provided to collect the sound emitted by the user, and the imaging unit 122 is provided to capture the facial expression of the user and the like. For example, the sound collecting unit 121 is preferably provided at a position where the user's voice can be easily collected when the user faces the display unit 111 shown in FIG. The same applies to the image pickup unit 122, and it is preferable that the image pickup unit 122 is provided at a position where it is easy to image the user's facial expression or the like when the user faces the display unit 111 shown in FIG. Specifically, for example, it is preferable that the sound collecting unit 121 and the imaging unit 122 are provided on the bezel or the like around the display unit 111.

The external information input unit 21 shown in FIG. 1 may be any as long as it can perform a predetermined input operation, and examples thereof include an input interface such as a mouse, a keyboard, and a touch panel.

The control unit 13 includes a feature data creation unit 131, a feature amount extraction unit 132, and a cognitive function prediction unit 133. The control unit 13 transmits data necessary for the information transmission unit 11 to transmit information to the user (image data displayed on the display unit 111 and audio data output from the audio output unit 112) to the information transmission unit 11. Can be given. Further, the control unit 13 is a feature data creation unit 131 based on the measurement result of the measurement unit 12 (audio data obtained from the sound collecting unit 121 and / or image data obtained from the imaging unit 122). Create at least one feature data selected from the group consisting of data, linguistic feature data and image feature data. Further, the feature amount extraction unit 132 of the control unit 13 extracts the feature amount from the feature data obtained from the feature data creation unit 131, and the cognitive function prediction unit 133 obtains at least one type obtained by the feature amount extraction unit 132. Based on the feature amount of, it is predicted whether or not the user has a tendency of a condition or a symptom due to a decrease in cognitive function.

The control unit 13 is a so-called computer, and includes an arithmetic unit such as a CPU (Central Processing Unit) (not shown), and a known configuration such as a data memory, a memory for storing data, and a working memory. The feature data creation unit 131, the feature amount extraction unit 132, and the cognitive function prediction unit 133 each include hardware and a program that operate on a computer and exert a predetermined function.

The control unit 13 can adjust the volume and / or speed of the voice information output from the voice output unit 112 so that the user can easily understand it or according to the user's preference. Further, in the control unit 13, the text 111b can be added to the voice information by the user's selection.

The database 31 is composed of a recording device such as a hard disk or a semiconductor memory, and data required for various processes performed by the control unit 13 (for example, data for asking a user, data for displaying an avatar 111a, etc.). Is being recorded.

Here, the information transmission unit 11 is configured to transmit information including an atypical question to the user. In the present specification, the "atypical question" is a question other than the fixed question that is always asked every time the user uses the cognitive function prediction device.
Examples of standard questions include questions such as date and time, place, and calculation included in neuropsychological tests such as the Hasegawa Dementia Scale (HDS-R) and the MMSE. In addition, the atypical questions include questions randomly selected from a plurality of predetermined questions, questions about past events according to the user's age, questions created based on the user's response, and the like. .. The atypical question transmitted from the information transmission unit 11 to the user is configured to always include a question about a past event according to the age of the user. Further, the atypical question may be an atypical question set in which three questions are randomly selected from an atypical question set composed of at least 10 questions.

Here, the question about the past event according to the age of the user is preferably an event when the user was in his 10s to 40s (for example, an event, an incident, an event, an event, a fashion, etc. at that time). .. This is because including questions about past events when the user was in his 10s and 40s in the above atypical questions may improve the prediction accuracy of the condition or symptom caused by the deterioration of cognitive function.
As for the age of the user, the age at the time of using the cognitive function prediction device 10 of the user is determined by the control unit 13 via the external information input unit 21, the sound collecting unit 121 in the measuring unit 12, and the imaging unit 122. You may check each time you use it. Further, in the initial setting at the start of use of the cognitive function prediction device 10, the age of the user is input in advance via the external information input unit 21, the sound collecting unit 121 in the measuring unit 12, and the imaging unit 122, and is input to the database 31. Based on the stored user initial information, the control unit 13 may calculate the user age at the time of use.
Questions about past events according to the user's age are selected from past events stored in the database in advance based on the user's age, based on the user's age confirmed or calculated by the control unit 13 as described above. It may be something that is done. Further, when the cognitive function prediction device 10 can be connected to a network line such as the Internet, the control unit 13 has a past event in which the user is in a specific age based on the confirmed or calculated age of the user. Questions about can be selected and created from web information and information obtained from external databases.

In addition, the information transmission unit 11 transmits at least one fixed question selected from the date and time, place, and human relations, and at least one atypical question about the event when the user was in his 10s and 40s. It is preferably configured. Of the standard questions, the above-mentioned questions regarding date and time, place, and relationships are questions that are answered based on orientation and short-term memory (immediate memory). In addition, questions about events when users were in their 10s and 40s are questions that are answered based on long-term memory (remote memory). It is known that long-term memory declines following short-term memory decline as the condition or symptom progresses due to cognitive decline. By combining these two questions and asking the user, cognition The degree of progression of the initial symptoms of the condition or symptom caused by the deterioration of function can be measured.

The control unit 13 can further include a question creation unit 134 configured to create a free question based on a response from the user. The question creation unit 134, like the feature data creation unit 131, the feature amount extraction unit 132, and the cognitive function prediction unit 133, includes hardware and a program that operate on a computer and exert a predetermined function.
The question created by the question creation unit 134 (hereinafter, simply referred to as a free question) may be a question created based on the response of the user, and is selected from the questions prepared in advance. The question may be selected based on the user's response. For example, the free question may be a question selected by the question creation unit 134 in the control unit 13 based on the user's response from a plurality of questions stored in the database in advance, and is based on the user's response. , The question may be created by combining the data stored in the database in advance. When the cognitive function prediction device 10 can be connected to a network line such as the Internet, the above-mentioned free question is selected from web information and information acquired from an external database based on the user's response in the question creation unit 134. It may be a created question.
By transmitting a free question based on the response from the user in the information transmission unit 11, the user can get a feeling of interacting with the avatar 111a when using the cognitive function prediction device 10, and is nervous. This is because the feeling is relaxed, the user can be measured in a natural state, and the accuracy of cognitive function prediction is improved.

Information on the user's hobbies, interests, etc. is collected by the external information input unit 21 and the measurement unit 12 when the user's cognitive function prediction device 10 is used or when the cognitive function prediction device 10 is initially set. It may be obtained via the unit 121 and the imaging unit 122 and stored in the database 31 from the control unit 13. When a free question is created in the question creation unit 134, it is recognized by reading out information on the user's hobbies and interests from the database 31 and specifying the field of the free question as the user's hobby and interest field. The user's interest in the question transmitted from the function prediction device 10 is increased, and the regular use of the cognitive function prediction device 10 can be promoted. This enables early prediction of cognitive function.
In addition, the cognitive function prediction unit 133 detects the difference in the feature amount between the question about the user's hobby or interest field and the question about the other field, which is caused by the deterioration of the cognitive function with higher accuracy. It may be possible to predict the condition or symptoms.

The feature amount extraction unit 132 in the control unit 13 converts the feature data into at least one of the voice feature data and the linguistic feature data created by the feature data creation unit 131 based on the user voice obtained by the measurement unit 12. Based on this, it is preferable that the feature amount is extracted. Further, the feature amount extraction unit 132 of the control unit 13 extracts the feature amount based on the voice feature data and the linguistic feature data created by the feature data creation unit 131 based on the user voice obtained by the measurement unit 12. It is more preferable that it is configured as such. In other words, it is preferable that two or more types of a feature amount based on the phonetic feature data and a feature amount based on the linguistic feature data are extracted.
By extracting the feature amount based on the voice feature data and the linguistic feature data created by the feature data creation unit 131, a user who does not have a state or a symptom due to a decrease in cognitive function and a user who has a mild cognitive function. This is because it is possible to classify users who have a condition or symptom caused by a decrease in the amount of data with high accuracy. In particular, when the condition or symptom caused by cognitive decline is dementia, it is possible to classify non-dementia users and longitude cognitive impairment (MCI) users with high accuracy (90% or more). it can.
Examples of the feature amount extracted from the voice feature data include pitch (basic frequency, pitch), voice volume (power), voice quality, reaction time, pause (speech interval), and the like.
The feature quantities extracted from the linguistic feature data include the number of tokens (morphemes), fillers, type-token ratio (TTR (Type Taken Radio)), part of speech information (number of nouns, verbs, adjectives, adverbs, etc.). ), Syntax complexity, vocabulary selection and vocabulary difficulty. The type-token ratio is the number of tokens, which is the total number of words included in the voice uttered by the user when responding, and the number of types, which is the total number of words included in the voice uttered by the user when the user does not allow duplication. Is divided by the number of tokens.

The feature amount extraction unit 132 of the control unit 13 is a group consisting of a line-of-sight pattern, a facial action coding system (FACS), and a facial landmark feature created by the feature data creation unit 131 based on the user image obtained by the measurement unit 12. At least one of the at least one image feature data selected from the above, and the audio feature data and the linguistic feature data created by the feature data creation unit 131 based on the user voice obtained by the measurement unit 12. It is preferable that the feature amount is extracted based on the feature data. In this case, a total of two or more types, a feature amount extracted from the image feature data and a feature amount extracted from the phonetic feature data and / or the linguistic feature data, are extracted. Further, the feature data creation unit 131 is created based on the above-mentioned image feature data created by the feature data creation unit based on the user image obtained by the measurement unit 12 and the user voice obtained by the measurement unit 12. It is more preferable that the feature amount is extracted based on the phonetic feature data and the linguistic feature data. In this case, a total of three or more types of the feature amount extracted from the image feature data, the feature amount extracted from the audio feature data, and the feature amount extracted from the linguistic feature data are extracted.
This is because it is possible to predict the cognitive function with higher accuracy by extracting the feature amount from the image feature data in addition to the voice feature data and / or the linguistic feature data. In addition, depending on the user, noise such as external noise is included in the audio feature data, but there is a possibility that the accuracy of cognitive function prediction can be improved by using the feature amount extracted from the image feature data together. Because there is.

The image feature data is created by using a known image processing method and / or image recognition method. For example, Intel® OpenCV (Intel Open Source Computer Vision Library) or a free tool OpenFace Etc. may be used to create the image. Further, for example, a face recognition program may be created by using an object detection program registered in OpenCV or the like, and image processing and / or image recognition may be performed. It is not always necessary to use the open CV for the image recognition program, or an existing program or a chip equipped with an existing image recognition circuit may be used.

Further, when the feature amount is extracted by the feature amount extraction unit 132, the feature amount may be updated by a means such as a maximum likelihood linear regression method (MLLR: Maximum Likelihood Linear Regression) in order to absorb individual differences. This is a method of adaptively changing the feature distribution for an individual by maximum likelihood estimation from the characteristics of a small amount of an individual and the characteristics of a large number of people collected so far. This is because it is possible to predict the cognitive function with higher accuracy by performing a process for absorbing individual differences in extracting the feature amount.

The information transmission unit 11 can display the prediction result (cognitive function prediction result) of the state or symptom caused by the deterioration of the cognitive function for the user obtained by the cognitive function prediction unit 133 in the control unit 13.
In this case, the control unit 13 is configured to send the cognitive function prediction result obtained by the cognitive function prediction unit 133 to the information transmission unit 11.

Although not shown, the control unit 13 is a cognitive function prediction result obtained by the cognitive function prediction unit 133 via, for example, a printing device such as a printer, a communication device that can be connected to the Internet or a LAN (Local Area Network), or the like. Can be printed or data transmitted.
Further, for example, the control unit 13 can transmit the cognitive function prediction result obtained by the cognitive function prediction unit 133 to the relatives of the user, the family medical institution, or the like via the communication means.
The printing device, communication device, and the like may be included in the cognitive function prediction device 10 of the present invention, or may be connected to the cognitive function prediction device 10 of the present invention via a communication line.

Further, in the embodiment of the present invention, the configuration of the cognitive function prediction device 10 described above may be realized by, for example, a personal computer, a smartphone, a tablet, a television or the like used by the user. Further, the embodiment of the present invention is not limited to the one in which all the constituent elements are composed of an integrated device as in the above-mentioned cognitive function prediction device 10, but is composed of a plurality of devices. You may. For example, a user terminal including an information transmission unit 11 and a measurement unit 12, and a cognitive function prediction device that calculates and processes the measurement results obtained by the measurement unit 12 to predict a state or symptom caused by a decrease in cognitive function. May be a separate body.

[Cognitive function prediction system]
FIG. 3 is a diagram showing an example in which the information transmission unit 11 and the measurement unit 12 in the cognitive function prediction device 10 shown in FIG. 1 are configured as separate bodies, and is a diagram showing a cognitive function prediction system according to an embodiment of the present invention. It is a figure for demonstrating.
As shown in FIG. 3, the cognitive function prediction system 40 has an information transmission unit 11 for transmitting voice information and image information to the user, and a measurement unit for measuring the user voice and / or the user image. A group consisting of voice feature data, linguistic feature data, and image feature data based on the measurement results from the user terminal 50 having the external information input unit 21 and the external information input unit 21 and the measurement unit 12 on the user terminal 50. A feature data creation unit 131 for creating at least one selected feature data, a feature amount extraction unit 132 for extracting a feature amount from the feature data obtained by the feature data creation unit 131, and a feature amount extraction unit. The cognitive function prediction unit 133, which is configured to predict whether or not the user has a tendency of a state or symptom caused by a decrease in cognitive function, and a question creation unit, based on the feature amount obtained in the unit 132. A cognitive function prediction device 60 having 134 and a database 31 is provided as a separate body. When the user terminal 50 and the cognitive function prediction device 60 for predicting a state or symptom caused by a decrease in cognitive function are separated in this way, the user terminal 50 and the cognitive function prediction device 60 are connected to a communication line. Is connected by.

As the communication line, a commonly used communication line such as an infrared ray, a short-range wireless communication line, a network line, a local area network line, and a telephone line can be appropriately selected. Further, the connection between the user terminal 50 and the cognitive function prediction device 60 is a transmitter / receiver and a communication unit necessary for using a generally used communication line regardless of whether it is wired or wireless. Can be provided.

When the present embodiment is divided into the user terminal 50 and the cognitive function prediction device 60 as shown in FIG. 3, for example, the information transmission unit 11 such as a smartphone, tablet, personal computer, or television already used by the user. A terminal having a device and a measuring unit 12 and capable of connecting to a communication line can be used as the user terminal 50. By doing so, the terminal that the user has been using conventionally can be the user terminal 50, so that the psychological burden on the user who uses the cognitive function prediction system, which is one embodiment, can be reduced. Can be done. This promotes the regular use of the cognitive function prediction system, and can predict the condition or symptom caused by the deterioration of cognitive function at an early stage.

[Cognitive function prediction method]
Next, a cognitive function prediction method, which is one of the present embodiments, will be described.
The cognitive function prediction method of the present embodiment includes an information transmission step of transmitting voice information and image information to the user, a measurement step of measuring the user voice and / or the user image, and the measurement result of the measurement unit. Based on this, a feature data creation step for creating at least one type of feature data selected from a group consisting of voice feature data, linguistic feature data, and image feature data, and feature data obtained in the above feature data creation step. A cognitive function that predicts whether or not the user has a tendency of a state or symptom due to a decrease in cognitive function based on the feature amount extraction step of extracting the feature amount from the feature amount and the feature amount obtained in the above feature amount extraction step. The information transmitted from the information transmission unit including the prediction step includes an atypical question, and the atypical question includes a question about a past event according to the age of the user.

FIG. 4 is a flowchart for explaining a cognitive function prediction method according to an embodiment of the present invention. Further, the flowchart of FIG. 4 shows an operation example of the cognitive function prediction device 10 shown in FIG. 1 and the cognitive function prediction system 40 shown in FIG. In FIG. 4, for the sake of simplicity, the image data displayed by the display unit 111 in the information transmission unit 11 is referred to as the avatar 111a, and the voice information output from the voice output unit 112 is as inquired by the avatar 111a. Describe using anthropomorphic expressions so that they can be recognized.

When the control unit detects that the user activates the cognitive function prediction device, the control unit transmits a question including an atypical question to the information transmission unit. A question including the above-mentioned atypical question is transmitted to the user via the information transmission unit as if the avatar is asking a question, and the measurement unit measures the user's response (ST1). The measurement result in the measurement unit is delivered to the control unit, and the feature data creation unit in the control unit creates the feature data (ST2). Next, the feature amount extraction unit extracts the feature amount from the obtained feature data (ST3). Then, the cognitive function prediction unit predicts whether or not the condition or symptom is caused by the deterioration of the cognitive function from the obtained feature amount (ST4). The cognitive function prediction result obtained by the cognitive function prediction unit is transmitted from the control unit to the information transmission unit, and the cognitive function prediction result is output (ST5).

Although not specified in the flow chart of FIG. 4, the measurement result obtained in ST1, the feature data obtained in ST2, the feature amount obtained in ST3, and the cognitive function prediction result obtained in ST4 are appropriately determined by the control unit 13. It is stored in the database 31.
Further, the question used in ST1 may be a question selected from the question data stored in the database 31 in advance, and may be a question created in the question creation unit 134 in the control unit 13 according to the response of the user. There may be.

Further, the output of the cognitive function prediction result is not limited to the case where it is output by using the information transmission unit, and can be output by using a printing device or a communication device as described above. Further, the timing of outputting the cognitive function prediction result in ST5 may be any time after the end of ST4, for example, at any timing set by the user himself / herself, the user's family and / or the family doctor, the user himself / herself. It may be provided to the user's family and / or family doctor.

At least ST1 to ST4 of the flowchart described above are processed by, for example, a program realized in a cognitive function prediction device configured as a computer.

In the present embodiment described above, the user's response to an atypical question including a question about a past event according to the user's age is measured, and one or more feature data is created from the obtained measurement result, and the feature is characterized. Based on one or more features obtained from the data, it is predicted whether or not the user has a condition or symptom caused by a decrease in cognitive function. Therefore, this embodiment includes questions about user orientation, short-term memory and long-term memory, and can predict cognitive function with higher accuracy. Among the states or symptoms caused by the deterioration of cognitive function, it is possible to predict the cognitive function with higher accuracy for dementia.

In the present specification, as states or symptoms caused by cognitive decline, forgetfulness, memory decline, concentration decline, attention loss, judgment weakness, spatial cognitive decline, neural activity decline, and nerve transmission function decline. , Cognitive inflexibility, executive function, information processing slowdown, depression-like symptoms, dementia (including those caused by diseases such as Alzheimer's disease) and other conditions or symptoms.
The cognitive function prediction device in the present specification is preferably a dementia prediction device for predicting dementia, and the cognitive function prediction method is preferably a dementia prediction method for predicting dementia. , The cognitive function prediction system is preferably a dementia prediction system for predicting dementia.
In the present specification, a state or symptom caused by a decrease in cognitive function can be read as dementia, a cognitive function predictor can be read as a dementia predictor, and a cognitive function predictor can be read as a dementia predictor. The cognitive function prediction system can be read as a dementia prediction system.

Next, an example of the operation of the cognitive function prediction device 10 according to the embodiment of the present invention will be described. The operation of the cognitive function prediction device 10 described below is common to the operation of the cognitive function prediction system 40 including the user terminal 50 and the cognitive function prediction device 60 connected by the communication line shown in FIG. It can be read as an example of the operation of the cognitive function prediction system 40.

First, for example, the control unit 13 performs a predetermined input operation (for example, an operation via the external information input unit 21 and / or the measurement unit 12) to the cognitive function prediction device 10 by an operator (for example, a relative of a subject) or a user. ) Is detected, the operation of the cognitive function prediction device 10 is started.

As an initial setting, user information may be registered in the cognitive function prediction device 10. In such registration of user information, for example, the control unit 13 detects that predetermined information such as user information (name, age, etc.) has been input to the external information input unit 21 or the measurement unit 12. Then, the control unit 13 is achieved by storing the input user information in the database 31. The user information also includes, for example, data regarding the avatar 111a.
Further, as user information, biometric information such as fingerprint authentication, iris authentication, vein authentication, voice print authentication and face authentication may be registered in advance via the external information input unit 21 or the measurement unit 12. By first registering the biometric information of the user, the biometric information is detected by the control unit 13 via the measurement unit 12 and / or the external information input unit 21 when the cognitive function prediction device 10 is used next time. The user is identified by the above, and the cognitive function prediction device 10 can be operated smoothly.

When the cognitive function prediction device 10 is activated, the control unit 13 first reads necessary image data and audio data from the database 31 so that the user feels that the avatar 111a is introducing himself (name himself). , The data is input to the information transmission unit 11. In the following, for the sake of simplification of the explanation, only the anthropomorphic operation of the avatar 111a will be described, and the operation of the control unit 13 required for the avatar 111a to perform the anthropomorphic operation (image required from the database 31). The operation of reading data and audio data and inputting the data to the information transmission unit 11) will be omitted.

When the avatar 111a introduces himself / herself, it is possible to alleviate the discomfort and tension of the user who interacts with the machine. Also, after the avatar 111a introduces himself, the avatar 111a encourages the user to introduce himself (for example, the avatar 111a asks the question "What is your name?") To interact with the machine. It is possible to further alleviate the discomfort and tension of the person.

Next, the avatar 111a asks the user a question. For example, Avatar 111a said, "What month and what day is it today?", "Tell us what you know about the Osaka Expo.", "Tell us about the events that left an impression on you when you were in your twenties. Ask multiple questions such as "Please." The multiple questions asked by Avatar 111a include fixed questions (questions such as date and time, place, calculation, etc. included in neuropsychiatric tests such as Hasegawa dementia scale and MMSE), as well as atypical questions. Includes questions about past events according to the user's age.
For example, the avatar 111a randomly selects at least three questions from the question set as shown in FIG. 5 and asks questions. In FIG. 5, the questions related to Q5 to Q13 are set according to the age of the user. Since it corresponds to the question regarding the past event that was responded to, the question of any of Q5 to Q13 is included.
The questions by the avatar 111a are preferably at least 3 or more, including questions about past events according to the age of the user, and more preferably 5 or more.

The measuring unit 12 measures the user's reaction from the time when the question is asked by the avatar 111a to the time when the user's answer is completed. When the avatar 111a asks the user a question but the user does not give any answer (that is, after one question to the user is completed, the sound collecting unit 121 collects the user's voice. When the control unit 13 detects that the state of not collecting sound continues for a predetermined time), the next question may be moved to without waiting for the answer of the question. The predetermined time may be determined according to, for example, the conversation rhythm of the user, or may be uniformly set to, for example, 15 seconds. Further, the question that is difficult for the user to answer may be set so that the avatar 111a waits for an answer longer.
Further, when the avatar 111a is silenced after the user utters some voice and answers (that is, the sound collecting unit 121 collects the user's voice after one question to the user is completed, and then the avatar 111a collects the user's voice. When the control unit 13 detects that the sound collecting unit 121 does not collect the user's voice for a predetermined time), the following operation may be performed assuming that the user's answer is completed.

Then, the feature data creation unit 131 of the control unit 13 selects at least one type of feature data from the group consisting of voice feature data, linguistic feature data, and image feature data based on the measurement result of the measurement unit 12. To create. Specifically, the feature data creation unit 131 performs various processes such as voice section detection processing and voice recognition processing on the user voice data obtained by the sound collection unit 121 in the measurement unit 12, and the voice feature. Data and linguistic feature data can be prepared. Further, the feature data creation unit 131 may perform facial expression feature extraction processing on the user image data obtained by the image pickup unit 122 in the measurement unit 12 with a free tool OpenFace or the like to prepare data related to the image features. it can. The feature data creation unit 131 preferably creates voice feature data and / or linguistic feature data and image feature data, and creates voice feature data, linguistic feature data, and image feature data. Is more preferable.

Next, the feature amount extraction unit 132 in the control unit 13 extracts the feature amount from the feature data created by the feature data creation unit 131. The feature amount is extracted based on the feature data created by the feature data creation unit 131. Here, the feature amount calculated by the feature amount extraction unit 132 will be described with reference to the drawings. FIG. 6 is a table explaining the feature amount extracted by the feature amount extraction unit 132 in the control unit.

As shown in FIG. 6, the feature amount extraction unit 132 is configured to extract various feature amounts from the voice feature data, the linguistic feature data, and the image feature data created by the feature data creation unit 131. There is. The feature amount extraction unit 132 preferably extracts the feature amount extracted from the audio feature data and / or the feature amount extracted from the linguistic feature data, and the feature amount extracted from the image feature data. It is more preferable to extract the feature amount extracted from the audio feature data, the feature amount extracted from the linguistic feature data, and the feature amount extracted from the image feature data. Each feature amount shown in FIG. 6 is extracted by, for example, the following processing.

Features such as pitch, voice volume (power), voice quality, reaction time, and pause (speech interval) are extracted from the voice feature data by using, for example, a voice analysis tool such as Snack sound toolkit or OpenSmile. can do.
The feature amount obtained from the voice feature data is a feature amount related to the voice itself, which can be calculated without analyzing the content of the user's voice (morphemes, words, etc. contained in the voice).
"Pitch" is the pitch and fundamental frequency of the voice. The pause is characterized by the total number of times the silence is counted for 1 second or more and the time when the utterance interval is the longest in the user's utterance (response). The reaction time is the time difference between the end of the question of the avatar 111a and the start of the response of the user. For the fundamental frequency, the coefficient of variation, the average value, the maximum value, the median value, the minimum value, and the range can be used as feature quantities. Further, regarding the voice volume (power), the average value, the maximum value, and the minimum value can be set as feature quantities. Regarding the voice quality, it is the amplitude difference between the first harmonic overtone (h1) and the third formant (a3) in the user's voice.

From the linguistic feature data, for example, by using a morphological analysis engine such as MeCab, Japanese morphological analysis can be performed, and the number of tokens (morphological elements), filler, and type-token ratio (TTR (Type Taken)). Ratio)), part of speech information (number of nouns, verbs, adjectives and adjuncts, etc.), syntactic complexity, vocabulary selection, vocabulary difficulty, and feature quantities such as speech speed can be extracted.
That is, the feature amount obtained from the linguistic feature data is a feature amount related to the content of the user's voice, and is extracted by analyzing the content of the user's voice (morphemes, words, etc. contained in the voice).
For each of the features classified into language features, the morphological element and word information contained in the voice is, for example, a character string of the user's voice by applying a well-known voice recognition method to the user's voice data. It can be obtained by applying a well-known morphological element or word analysis method (for example, Mecab) to the character string after converting to.
The number of tokens is the total number of words contained in the user's voice. A filler is a number of words that do not have a specific meaning, such as "uh" or "ah". "TTR" is a value obtained by dividing the number of types, which is the total number of non-duplicate words contained in the user's voice, by the number of tokens. The value of TTR becomes smaller as the user uses the same word because the number of types does not increase and the number of tokens increases. The "difficulty level" is an intermediate value of the difficulty level of all nouns determined based on a predetermined dictionary in which the difficulty level of a phrase is defined numerically. The "speaking speed" is a value obtained by dividing the speaking time of the subject by the number of words.

Examples of the image feature data include image feature data such as a facial action unit (FACS), a facial landmark feature, and a line-of-sight pattern by using Openface, and feature amounts and facial landmarks extracted from FACS. The feature amount extracted from the feature, the feature amount extracted from the line-of-sight pattern, and the like can be extracted. Further, the response time (mouth reaction time) from the end of the question of the avatar 111a to the movement of the user may be used as the feature amount extracted from the image feature data. The feature amount obtained from the image feature data is a feature amount extracted only from the appearance of the user regardless of the voice of the user.

It is preferable that the feature amount is extracted based on at least one of the voice feature data and the linguistic feature data created based on the user voice obtained by the measuring unit 12, and the voice feature data and the linguistic feature data It is more preferable that the feature amount is extracted based on the linguistic feature data. Further, the above-mentioned feature amount includes a line-of-sight pattern created based on a user image obtained by the measuring unit 12 in addition to the feature amount extracted based on the voice feature data and / or the linguistic feature data, and a facial action coding system. And it is more preferable to include a feature amount extracted from at least one type of image feature data among the facial landmark features. Further, it is most preferable to include the audio feature data, the linguistic feature data, and the feature amount extracted from the image feature data. This is because the feature amount obtained by the feature amount extraction unit 132 includes the feature amount obtained from the image feature data, so that the cognitive function prediction accuracy is improved.
In addition, the feature amount extraction unit 132 has at least one kind of voice feature selected from the group consisting of pitch (pitch, basic frequency), voice volume (power), voice quality, reaction time, and pause (speech interval). , Token (morphological element) number, filler, type-token ratio (TTR (Type Taken Radio)) and part-of-speech information (number of nouns, verbs, adjectives, adjectives, etc.) at least one linguistic feature selected from the group. And at least one type of image feature selected from the group consisting of the feature amount extracted from the facial action unit, the feature amount extracted from the facial landmark feature, and the feature amount extracted from the line-of-sight pattern. It is more preferable that 10 or more kinds of feature amounts are extracted from the above feature amounts. Moreover, it is preferable to include each feature amount extracted from each of the above feature data. This is because the accuracy of the cognitive function prediction result can be improved. This is because each of the above-mentioned features is considered to be a feature that more reflects the information processing and cognitive activity of the brain during dialogue.

When the avatar 111a asks a plurality of questions as described above, for example, by averaging 10 types of features calculated for each question, the number of questions x 10 features is finally obtained. Is extracted.

The cognitive function prediction unit 133 in the control unit 13 predicts whether or not the user has a tendency of a state or symptom due to a decrease in cognitive function based on at least one feature amount obtained by the feature amount extraction unit 132. To do. Specifically, the cognitive function prediction unit 133 extracts one or more feature amounts in the feature amount extraction unit 132 based on the measurement result of the measurement unit 12, and the user causes a decrease in cognitive function based on the feature amount. Determine if it is a condition or symptom.
In addition, it is preferable that the cognitive function prediction unit 133 predicts whether or not the user has a tendency of a state or a symptom due to a decrease in cognitive function based on 10 or more kinds of feature quantities. Further, it is more preferable that the above 10 or more kinds of feature amounts include each feature amount extracted from each of the above feature data. This is because the cognitive function prediction accuracy is high.

By applying a predetermined determination method to the above 10 types (10 dimensions) of the feature amount, the cognitive function prediction unit 133 determines whether or not the user is in a state or symptom caused by a decrease in cognitive function. Predict. As this prediction method, for example, it is preferable that the prediction is made using at least one discriminative model selected from the group consisting of SVM (Support Vector Machine), logistic regression analysis, and deep learning. As input of the discriminative model, the feature amount extracted by the feature amount extraction unit 132 is normalized so as to have a minimum value of 0 and a maximum value of 1, respectively, and a group having a state or symptom due to deterioration of cognitive function (for example, cognition). A determination method can be constructed by training a model that classifies a group (for example, a non-dementia group) and a group (for example, a non-dementia group) that has no condition or symptom due to a decrease in cognitive function. In constructing the determination method, it is preferable to construct the determination method by learning 10 or more types of features including the reaction time (speech feature amount).

Then, the control unit 13 outputs the cognitive function prediction result data obtained by the cognitive function prediction unit 133 to the information transmission unit 11 and the like.

Here, the verification result of the prediction accuracy of the cognitive function in the cognitive function prediction device 10 will be described with reference to the drawings. FIG. 7 is a graph showing the verification results of the prediction accuracy of dementia, which is one of the states or symptoms caused by the deterioration of cognitive function in the cognitive function prediction device according to the embodiment of the present invention. FIG. 7 (a) is a graph showing the verification result of the prediction accuracy when SVM is adopted as the determination method to predict dementia, and FIG. 7 (b) is a graph showing the verification result of the prediction accuracy when dementia is predicted by adopting the logistic regression analysis. It is a graph which showed the verification result of the prediction accuracy in the case of predicting.

The graphs shown in FIGS. 7A and 7B are graphs showing ROC curves and AUC (Area Under the Curve) values calculated using leave-one-participant-out cross-validation. In leave-one-participant-out cross-validation, a judgment method is constructed using the remaining sample (study sample) extracted from a plurality of samples, and one sample extracted (for testing). This is a verification method for obtaining a judgment result by applying the judgment method to the sample), and is a verification method for obtaining a plurality of judgment results so that all the samples are once used as test samples. In addition, the verification results shown in FIGS. 7 (a) and 7 (b) obtained 21 types of features from each of about 24 collaborators, each of whom has half the dementia group and half the non-dementia group, and these 21 types. This is a set of features calculated as one sample.
The 21 feature quantities are the time taken to answer the question (response time), the total number of silences between utterances of 1 second or more (pause count), and utterances and utterances. Time average value (pause time average value) between utterances (1 second or more), maximum value of time between utterances (1 second or more) (pause time maximum value), basic frequency average value, basic frequency minimum value , Maximum value of basic frequency, median value of basic frequency, difference between maximum value and minimum value of basic frequency, standard deviation of basic frequency, fluctuation coefficient of basic frequency, average value of voice volume, standard deviation of voice volume, length of utterance (Speech time), number of tokens in utterance, total number of fillers in utterance, total number of verbs in utterance, total number of nomenclature in utterance, total number of adjectives in utterance, total number of adjuncts in utterance, total time The number of tokens per. In this embodiment, the avatar 111a asks a question set (5 questions) including at least one question about a past event according to the age of the user of Q5 to Q13 out of the 19 questions shown in FIG. We chose randomly and asked each collaborator a question. The above few utterances may be all the user's answers to the plurality of questions from the avatar 111a (all the user's response utterances), and at least three questions including questions about past events according to the user's age. It may be the answer to the question (the utterance of the user response).

The ROC curve is a line connecting a plurality of verification results obtained by performing verification while changing the parameters that determine the boundary between positive and negative in a specific determination method, and the horizontal axis is the false positive rate (determined as positive). However, the percentage that is actually negative) and the vertical axis are the true positive percentage (the percentage that is determined to be positive and is actually positive). Further, in the graphs shown in FIGS. 7A and 7B, the diagonal broken line indicating the case where the false positive rate and the true positive rate match corresponds to the case where positive and negative are completely randomly determined. Therefore, it can be said that the more the ROC curve is on the upper side of the broken line (the region where the true positive rate is higher than that of the random judgment) and the more deviated from the broken line, the higher the determination accuracy of dementia. Further, the AUC value represents the size of the area on the lower side of the ROC curve, and it can be said that the closer this value is to 1, which is the maximum value, the higher the determination accuracy of dementia.

As shown in FIGS. 7A and 7B, the ROC curve when either SVM or logistic regression is adopted is above the diagonal diagonal line described above, and is sufficiently deviated from the diagonal line. Further, the AUC value when either SVM or logistic regression is adopted is a value extremely close to the maximum value of 1 (SVM: 0.95, logistic regression: 0.92). Therefore, it can be seen that the cognitive function prediction device according to the embodiment of the present invention has extremely high prediction accuracy of dementia, which is one of the states or symptoms caused by the deterioration of cognitive function.

As described above, the cognitive function predictor according to the embodiment of the present invention measures the user's response to an atypical question including a question about a past event according to the user's age, and from the obtained measurement result. One or more feature data is created, and based on the one or more feature amounts obtained from the feature data, it is predicted whether or not the user is in a state or symptom caused by a decrease in cognitive function. Therefore, the present embodiment includes questions regarding the user's short-term memory and long-term memory, and enables more accurate prediction of the state or symptom caused by the deterioration of cognitive function.

The verification results illustrated in FIGS. 7 (a) and 7 (b) show whether or not the user has dementia, which is one of the states or symptoms caused by the deterioration of cognitive function, using the above-mentioned 21 types of features. Although it is a verification result when it is determined whether or not, it is possible to improve the determination accuracy of dementia without necessarily using all of these 21 types of feature quantities. In particular, among these 21 types of feature quantities, those that contribute to the improvement of the determination accuracy of dementia are selectively used to determine whether or not the user has dementia, thereby improving the determination accuracy of dementia. Can be expected. That is, the cognitive decline is determined by the user determining whether the condition or symptom is caused by the cognitive decline by selectively using the feature amount that contributes to the improvement of the cognitive decline or the symptom. It is expected that the accuracy of determining the condition or symptom caused by the above will be improved.

Further, the verification results illustrated in FIGS. 7A and 7B do not include the feature amount extracted from the image feature data, but include the feature amount extracted from the image feature data. May be good. Here, the graph shown in FIG. 8 is a graph showing the ROC curve and the AUC (Area Under the Curve) value calculated by using the Leave-one-participant-out intersection verification as in FIG. 7, and is an image feature. It is a graph which showed the verification of the determination accuracy of dementia which is one of the states or symptoms caused by the deterioration of cognitive function using data. The verification results shown in FIG. 8 obtained three types of features extracted from the image feature data from each of about 24 collaborators, half of whom were in the dementia group and half of the non-dementia group, and these three types. It is calculated by extracting from each collaborator using the set of features as the feature quantity.
The three types of features are the action units (Action Units (Australia)) extracted from the facial action coding system, the features extracted from the line-of-sight pattern, and the mouth reaction time extracted from the facial landmark features. ..

As shown in FIG. 8, the ROC curve when logistic regression is adopted using a combination of three types of features extracted from the image feature data is in any case of two different questions (Q1 and Q2). It is above the diagonal line and is sufficiently deviated from the diagonal line. Further, the AUC value is also close to the maximum value of 1 (Q1: 0.78, Q2: 0.82). Therefore, even when only the feature amount extracted from the image feature data is used, the cognitive function prediction device according to the embodiment of the present invention has high prediction accuracy of the state or symptom caused by the deterioration of the cognitive function, and specifically. It can be seen that the prediction accuracy of dementia is high. In addition, Q1 and Q2 in FIG. 8 are standard questions, Q1 is "what month and what day is it today", and Q2 is "tell me the memories you have enjoyed so far".
Therefore, by combining the feature amount extracted from the image feature data and the feature amount extracted from the linguistic feature data and / or the phonetic feature data, the variation due to the individual difference in the cognitive function prediction accuracy can be eliminated. It is thought that consistent cognitive function prediction will be possible.

Here, the significance of each feature amount in the prediction of dementia, which is one of the states or symptoms caused by the deterioration of cognitive function, will be described.
According to the verification results by the logistic regression method shown in FIG. 7 (b), among the 21 types of features, 1) reaction time, 2) difference between the maximum and minimum values of the basic frequency, and 3) the maximum value of the basic frequency, 4) Mean value of pause time, 5) Total number of verbs in speech is an important feature in detecting dementia, and reaction time is the feature that has the greatest influence on the prediction of dementia by logistic regression. It shows the most effective attributes for early detection of dementia. Therefore, it is considered to show the most effective attributes for early detection of conditions or symptoms caused by cognitive decline.

Further, in the present invention, the question asked by the avatar 111a to the user includes an atypical question, and the atypical question includes a question about a past event according to the age of the user. Among patients with conditions or symptoms caused by cognitive decline, those in the early stages do not show a statistically significant difference in reaction time to questions about past events from reaction times to the same questions in healthy individuals. Therefore, by including this question, it is possible to predict the degree of progression of the condition or symptom caused by the deterioration of cognitive function, especially the degree of progression of dementia.
In addition to the above questions, the questions asked by Avatar 111a include questions in which there is a statistically significant difference in response time between patients in the early stages of conditions or symptoms caused by cognitive decline and healthy subjects. It is preferable to do so.

In addition, it is preferable to determine the content of the question according to the situation in which the cognitive function prediction device is expected to be used. For example, when it is assumed that a TV, a personal computer, a tablet, a smartphone, or the like at home is used as a user terminal 50 in the cognitive function prediction system 40, as a question that is easy to answer at home, for example, "What did you eat for breakfast?" You may ask questions such as "Is it?"

In the above embodiment, the case where the avatar 111a is an image is illustrated (see FIG. 2), but the avatar may be a three-dimensional object such as a robot. However, it is preferable to use the avatar as an image because the configuration of the cognitive function prediction device can be simplified.

The present invention can be used for a cognitive function prediction device, a cognitive function prediction system, a cognitive function prediction method and a program for predicting whether or not a user has a state or symptom caused by a decrease in cognitive function.

10, 60: Cognitive function prediction device 11: Information transmission unit 111: Display unit 111a: Avatar 111b: Text 112: Audio output unit 12: Measurement unit 121: Sound collection unit 122: Imaging unit 13: Control unit 131: Feature data creation Department 132: Feature extraction unit 133: Cognitive function prediction unit 134: Question creation unit 21: External information input unit 31: Database 40: Cognitive function prediction system 50: User terminal

Claims (8)

An information transmission unit for transmitting audio and image information to users,
A measuring unit for measuring user voice and / or user image,
A feature data creation unit for creating at least one type of feature data selected from a group consisting of voice feature data, linguistic feature data, and image feature data based on the measurement results obtained by the measurement unit. ,
A feature amount extraction unit for extracting a feature amount from the feature data obtained by the feature data creation unit, and a feature amount extraction unit.
A cognitive function prediction configured to predict whether or not the user is prone to a condition or symptom caused by cognitive decline based on at least one feature amount obtained by the feature amount extraction unit. With a department
The information transmitted from the information transmission unit includes atypical questions.
The atypical question is a cognitive function predictor including a question about a past event according to a user's age. It also has a question-making unit that is configured to create questions based on user responses.
The cognitive function prediction device according to claim 1, wherein the atypical question includes a question created by the question creation unit. The feature amount extraction unit is based on the feature data of at least one of the voice feature data and the linguistic feature data created by the feature data creation unit based on the user voice obtained by the measurement unit. The cognitive function predictor according to claim 1 or 2, which is configured to extract data. The feature amount extraction unit is at least one selected from a group consisting of a line-of-sight pattern, a facial action coding system, and a facial landmark feature created by the feature data creation unit based on a user image obtained by the measurement unit. Features based on the image feature data of the above and at least one of the voice feature data and the linguistic feature data created by the feature data creation section based on the user voice obtained by the measurement section. The cognitive function predictor according to claim 1 or 2, which is configured to extract a quantity. Based on the feature amount extracted by the feature amount extraction unit, the cognitive function prediction unit determines whether or not the user has a tendency of a state or a symptom caused by cognitive decline in SVM (Support Vector Machine). The cognitive function predictor according to any one of claims 1 to 4, which is configured to predict using at least one selected from a group consisting of logistic regression analysis and deep learning. Information transmission steps to convey audio and image information to users,
Measurement steps to measure user voice and / or user image,
A feature data creation step of creating at least one type of feature data selected from a group consisting of voice feature data, linguistic feature data, and image feature data based on the measurement results of the measurement unit.
A feature amount extraction step for extracting a feature amount from the feature data obtained in the feature data creation step, and a feature amount extraction step.
Based on at least one feature amount obtained by the feature amount extraction unit, the user is provided with a cognitive function prediction step for predicting whether or not the user is prone to a state or symptom caused by cognitive decline.
The information transmitted from the information transmission unit includes atypical questions.
The atypical question is a cognitive function prediction method including a question about a past event according to a user's age. A program for causing a computer to execute the cognitive function prediction method according to claim 6. A user terminal including an information transmission unit for transmitting voice information and image information to a user, and a measurement unit for measuring user voice and / or user image.
A feature data creation unit for creating at least one type of feature data selected from a group consisting of voice feature data, linguistic feature data, and image feature data based on the measurement results of the measurement unit, and the feature data creation unit. Based on the feature amount extraction unit for extracting the feature amount from the feature data obtained by the data creation unit and at least one kind of feature amount extracted by the feature amount extraction unit, the user suffers from cognitive decline. It has a cognitive function predictor that is configured to predict whether or not there is a tendency for the resulting condition or symptom, and a cognitive function predictor.
The information transmitted from the information transmission unit includes atypical questions.
The atypical question is a cognitive function prediction system characterized by including questions about past events according to the age of the user.

Images