JP2019055064A - Diagnosis support information providing device and diagnosis support information providing method - Google Patents

Abstract

To provide information useful for a diagnosis of cerebral function diseases such as dementia with easy measurement.SOLUTION: In a diagnosis support information providing device 1, a facial direction measurement part 12 measures a facial direction of a subject in each time-series image data, a change amount calculation part 16 calculates a time-series facial direction change amount on the basis of the facial direction measured by the facial direction measurement part 12, a diagnosis support information generation part 18 calculates a ratio at which the facial direction change amount is equal to or less than a prescribed value in a prescribed period on the basis of the facial direction change amount calculated by the change amount calculation part 16, and generates diagnosis support information indicating a calculation result, and an output part 19 outputs the diagnosis support information generated by the diagnosis support information generation part 18.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a diagnosis support information providing apparatus and a diagnosis support information providing method.

  There is a technique for providing information useful for diagnosis of a brain function disorder such as dementia by analyzing an image of a subject (for example, see Patent Documents 1 and 2). On the other hand, there is a technique for analyzing the state of mind and body of a person from an image (for example, see Patent Document 3) and a technique for analyzing a facial expression from an image (for example, see Patent Document 4).

  It has been found that dementia and smiles of patients with dementia are related, for example, people with dementia have fewer smiles. Therefore, in recent years, research using smiles for diagnosis related to dementia has been conducted. As an example, there is a study that evaluates QOL (quality of life) of patients with dementia based on the result of quantifying smiles by image analysis.

  Images used for smile analysis can be taken without imposing a burden on the subject. However, there is data that smile time is less than 30 seconds per day even for healthy adult women, and the facial expression composition rate of smiles per day is low. In addition, smiles tend to decrease depending on age. Thus, since smiles generally appear at a low frequency, it has been difficult to obtain information useful for diagnosis related to dementia using smiles.

JP 2013-169375 A Japanese Examined Patent Publication No. 6-53107 JP 2005-237561 A JP 2014-206903 A

  The present invention has been made in consideration of the above circumstances. Provided are a diagnosis support information providing apparatus and a diagnosis support information providing method capable of providing useful information for diagnosis of a brain function disorder such as dementia by easy measurement.

  One aspect of the present invention is a face orientation measurement unit that measures the face orientation of a subject in each of time series image data, and a time series face orientation change based on the face orientation measured by the face orientation measurement unit. A change amount calculation unit that calculates an amount, and a ratio that the face direction change amount is equal to or less than a predetermined value during a predetermined period is calculated based on the face direction change amount calculated by the change amount calculation unit, and the diagnosis indicates the calculation result A diagnosis support information providing apparatus comprising: a diagnosis support information generation unit that generates support information; and an output unit that outputs the diagnosis support information calculated by the diagnosis support information generation unit.

  One aspect of the present invention is the above-described diagnosis support information providing apparatus, further including a gaze direction measurement unit that measures the gaze direction of the subject in each of the time-series image data, and the change amount calculation unit is Then, based on the line-of-sight direction measured by the line-of-sight direction measurement unit, a time-series line-of-sight change amount is calculated, and the diagnosis support information generation unit calculates a rate at which the line-of-sight change amount is a predetermined value or less during the predetermined period. Calculation is performed based on the time-series gaze change amount calculated by the change amount calculation unit, and the calculation result is further set in the diagnosis support information.

  One aspect of the present invention is the above-described diagnosis support information providing device, wherein the diagnosis support information generation unit is configured to determine whether the face direction change amount is based on the ratio in which the face direction change amount is equal to or less than a predetermined value. Based on the ratio that is equal to or less than a predetermined value and the ratio that the amount of line-of-sight change is equal to or less than a predetermined value, the possibility or degree of a brain function disease is determined, Output the result.

  One aspect of the present invention is the above-described diagnosis support information providing apparatus, further comprising a facial expression estimation unit that estimates the facial expression of the subject in each of the time-series image data, and the diagnostic support information generation unit includes: The facial expression estimated by the facial expression estimation unit for the predetermined period is further used to determine the possibility or degree of a brain function disorder.

  One aspect of the present invention is a face orientation measurement step for measuring a face orientation of a subject in each of time series image data, and based on the face orientation measured in the face orientation measurement step, A change amount calculating step for calculating a change amount; and a ratio of the face direction change amount being equal to or less than a predetermined value in a predetermined period based on the face direction change amount calculated in the change amount calculating step; A diagnostic support information providing method comprising: a diagnostic support information generating step for generating diagnostic support information to be shown; and an output step for outputting the diagnostic support information calculated in the diagnostic support information generating step.

  According to the present invention, it is possible to provide information useful for diagnosis of a brain function disease such as dementia by easy measurement.

It is a functional block diagram of the diagnostic assistance information provision apparatus by 1st Embodiment. It is a figure which shows the example of face direction measurement information by the same embodiment, gaze direction measurement information, and facial expression estimation information. It is a figure which shows the example of the diagnostic assistance information by the same embodiment. It is a flowchart which shows the analysis process which the diagnosis assistance information provision apparatus by the embodiment performs. It is a figure which shows evaluation of the peripheral symptom (BPSD) peculiar to dementia and a facial expression regarding the test subject using the diagnostic assistance information provision apparatus by the embodiment. It is a figure which shows the test result using the diagnostic assistance information provision apparatus by the embodiment. It is a block diagram of the diagnostic assistance information provision system by 2nd Embodiment. It is a block diagram of the diagnosis assistance system by 3rd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
Previous studies have shown that smiles are reduced in patients with dementia. Caregivers of patients with dementia have the impression that as the symptoms of dementia progress, not only smiles but also other facial expressions decrease, and the time without facial expressions seems to increase. Based on this, in order to obtain information useful for diagnosis related to dementia, it can be considered to measure the time of expressionlessness and use it to determine the symptoms of dementia. However, an expressionless expression is a state that is a straight face but does not move and is not thinking or operating in any way, and it is relatively easy for a person to judge a face that is not expressionless and an expressionless state based on comprehensive information. On the other hand, when the apparatus is used, it is not possible to accurately measure the expressionlessness even if the state of the true face including the expressionlessness can be measured.
Therefore, in the present embodiment, information useful for diagnosis of a brain function disease, particularly dementia, is provided using the movement of the subject's face and the movement of the line of sight.

  FIG. 1 is a block diagram illustrating a configuration of a diagnosis support information providing apparatus 1 according to the present embodiment. The diagnosis support information providing apparatus 1 includes an image acquisition unit 11, a face orientation measurement unit 12, a gaze direction measurement unit 13, a facial expression estimation unit 14, a storage unit 15, a change amount calculation unit 16, an input unit 17, and a diagnosis support information generation unit 18. And an output unit 19. The diagnosis support information providing apparatus 1 may be realized by a single computer apparatus or may be realized by a plurality of computer apparatuses connected via a network. When realized by a plurality of computer devices, which computer device realizes which functional unit is arbitrary, and one functional unit may be realized by a plurality of computer devices.

  The image acquisition unit 11 acquires time-series image data. The image data is data including at least a face image of the subject 2. Time-series information indicating the order of photographing in time series is added to the image data. The time-series information may be a shooting date and time, or a number representing the order, and when time-series image data is included in one file, the arrangement order of the image data may be the time-series information. The image acquisition unit 11 may have a function of converting the acquired image data into a data format suitable for image analysis in the face orientation measurement unit 12, the gaze direction measurement unit 13, and the facial expression estimation unit 14. In the present embodiment, the image acquisition unit 11 receives video data captured by the imaging device 3. The imaging device 3 is a home video camera, for example. The image acquisition unit 11 acquires the data of each frame obtained from the received video data as time-series image data. In this case, the time-series information of each image data can be obtained based on the shooting date / time added to the video data and the number of frames. Note that the image acquisition unit 11 may read video data or time-series image data from a recording medium such as a USB (Universal Serial Bus) memory or a CD-ROM, or may receive it from another computer apparatus. Further, the diagnosis support information providing apparatus 1 may have the function of the imaging apparatus 3.

  The face orientation measurement unit 12 analyzes the image data and measures the face orientation of the subject 2 in each image data. The gaze direction measuring unit 13 analyzes the image data and measures the gaze direction of the subject 2 in each image data. The facial expression estimation unit 14 analyzes the image data and estimates the facial expression of the subject 2 in each image data. Any existing image analysis technique can be applied to the face orientation measurement in the face orientation measurement unit 12, the gaze direction measurement in the gaze direction measurement unit 13, and the facial expression estimation in the facial expression estimation unit 14.

  The storage unit 15 is a storage device such as a magnetic hard disk device or a semiconductor storage device. The storage unit 15 stores the measurement result in the face direction measurement unit 12, the measurement result in the gaze direction measurement unit 13, and the estimation result in the facial expression estimation unit 14 in association with time series information. The change amount calculation unit 16 calculates time-series face direction change amounts based on the face direction measurement results stored in the storage unit 15. Further, the change amount calculation unit 16 calculates a time-series gaze change amount based on the measurement result of the gaze direction stored in the storage unit 15. The storage unit 15 stores the information of the calculation result in the change amount calculation unit 16 in association with the time series information.

  The input unit 17 receives a user operation. The input unit 17 is configured using an existing input device such as a keyboard, a pointing device (such as a mouse or a tablet), a button, or a touch panel. The input unit 17 is operated by the user when inputting a user instruction to the diagnosis support information providing apparatus 1. The input unit 17 may be an interface for connecting the input device to the diagnosis support information providing device 1. In this case, the input unit 17 inputs an input signal generated in response to a user input in the input device to the diagnosis support information providing device 1. The input unit 17 receives input of information indicating the analysis target period when the user specifies the analysis target period. The analysis target period may be specified by, for example, the first time series information and the last time series information of the analysis target period. Further, the analysis target period may be specified by the name of the file or video data obtained from the time-series image data. A plurality of periods may be input as the analysis target period.

  The diagnosis support information generation unit 18 reads information on the time-series face direction change amount and the line-of-sight change amount calculated by the change amount calculation unit 16 from the storage unit 15, and each of the face direction and the line of sight does not change during the analysis target period. Calculate the percentage. Here, “the face direction or line of sight does not change” may include a case where the change is scarce. Further, the diagnosis support information generation unit 18 calculates statistical information related to the facial expression in the analysis target period based on the facial expression estimation result stored in the storage unit 15. The diagnosis support information generation unit 18 generates, as diagnosis support information, the ratio that the face orientation did not change and the ratio that the line of sight did not change in the analysis target period, and statistical information about each facial expression in the analysis target period.

  The output unit 19 outputs data such as diagnosis support information generated by the diagnosis support information generation unit 18. The output is, for example, display on a display, writing to a recording medium, or transmission to another computer device connected via a network. The output unit 19 outputs data to the user of the diagnosis support information providing apparatus 1 via an output device (not shown) connected to the diagnosis support information providing apparatus 1. The output device may be configured using, for example, a device that outputs images and characters on a screen. For example, the output device can be configured using a CRT (Cathode Ray Tube), a liquid crystal display, an organic EL (Electro-Luminescent) display, or the like. Further, the output device may be configured using a device that prints (prints) images and characters on a sheet. For example, the output device can be configured using an ink jet printer, a laser printer, or the like. The output device may be a device that writes data to a recording medium. The output unit 19 may transmit data to another computer device via a communication device provided in the diagnosis support information providing device 1.

  FIG. 2 is a diagram illustrating examples of face orientation measurement information, gaze direction measurement information, and facial expression estimation information stored in the storage unit 15. The face orientation measurement information indicates a measurement result of the face orientation measured by the face orientation measuring unit 12. The gaze direction measurement information indicates a measurement result of the gaze direction measured by the gaze direction measurement unit 13. The expression estimation information indicates the estimation result of the expression estimated by the expression estimation unit 14. In the same figure, face direction measurement information, gaze direction measurement information, and facial expression estimation information of the same time-series information are shown in the same row, and the lower row indicates a later order. That is, in the same figure, the arrangement order is used as time series information for the face orientation measurement information, the gaze direction measurement information, and the facial expression estimation information.

  The face orientation measurement information and the line-of-sight measurement information shown in the figure indicate the directions at respective angles in the vertical direction and the horizontal direction. For example, in the left-right direction, the angle when facing the front is set to 0, the right angle of the subject 2 is expressed as a negative number, and the left direction angle is expressed as a positive number. You can easily grasp the direction of the direction and the size of the direction. In addition, regarding the vertical direction, the angle when facing the front is 0, the upward angle can be expressed as a negative number, and the downward angle as a positive number.

  The facial expression estimation result may be any facial expression such as “joy”, “surprise”, “anger”, “sadness”, “true face”, etc. A score obtained by quantifying the likelihood of being performed may be used. When the score of each facial expression is obtained, the facial expression with the highest score may be used as the facial expression estimation result. In the facial expression estimation information shown in the figure, a score for each facial expression is set.

  FIG. 3 shows an example of the output result of diagnosis support information. The diagnosis support information shown in the figure includes the ratio of no change in face direction in the vertical direction and the horizontal direction in the analysis target period, the ratio of no change in line of sight in the vertical direction and the horizontal direction, and the appearance ratio of each facial expression. When a plurality of analysis target periods and groups to which these target periods belong are input by the input unit 17, the diagnosis support information generation unit 18 generates diagnosis support information for each analysis target period and performs diagnosis between groups. The result of performing the T test of the support information may be further output.

  FIG. 4 is a flowchart of analysis processing executed by the diagnosis support information providing apparatus 1. First, the image acquisition unit 11 acquires time-series image data based on the video data received from the imaging device 3 (step S110). The face orientation measurement unit 12 analyzes each image data acquired by the image acquisition unit 11 in step S110 and measures the face orientation of the subject 2 (step S120). In the present embodiment, the face orientation measurement unit 12 calculates the left-right angle and the vertical angle at which the face is facing as the face orientation measurement result. This is because it is assumed that the frequency of moving the face up and down or left and right is high and the frequency of moving the face in the oblique direction is low. The face orientation measurement unit 12 writes the face orientation measurement information in which the time series information and the face orientation measurement result using the image data specified by the time series information are set in the storage unit 15.

  The line-of-sight direction measurement unit 13 analyzes each image data acquired by the image acquisition unit 11 in step S110 and measures the line-of-sight direction of the subject 2 (step S130). In the present embodiment, the line-of-sight direction measurement unit 13 calculates the left-right angle and the vertical angle of the line-of-sight as the line-of-sight measurement result. The gaze direction measurement unit 13 writes the gaze direction measurement information in which the time series information and the gaze direction measurement result using the image data specified by the time series information are set in the storage unit 15.

  The facial expression estimation unit 14 analyzes each image data acquired by the image acquisition unit 11 in step S110, and estimates the facial expression of the subject 2 (step S140). The facial expression estimation unit 14 writes the facial expression estimation information in which the time series information and the facial expression estimation result using the image data specified by the time series information are set in the storage unit 15.

  The change amount calculation unit 16 calculates the change amount of each face orientation measurement result from the immediately previous face orientation measurement result when arranged in time series (step S150). For example, it is assumed that the storage unit 15 stores N face orientation measurement information. The change amount calculation unit 16 sets the face orientation measurement result set in the face orientation measurement information indicating that the time series information is nth (n is an integer not less than 2 and not more than N), and the time series information is (n−1) th. And the difference from the face orientation measurement result set in the face orientation measurement information indicating. The change amount calculation unit 16 sets the calculated difference as the n-th face direction change amount in time series information. The face direction change amount is calculated for each of the vertical direction and the horizontal direction. The change amount calculation unit 16 writes the face direction change amount information in which the time series information and the face direction change amount are set in the storage unit 15.

  The change amount calculation unit 16 calculates a change amount of each gaze direction measurement result from the gaze direction measurement result immediately before being arranged in time series (step S160). For example, it is assumed that the storage unit 15 stores N line-of-sight direction measurement information. The change amount calculation unit 16 includes the gaze direction measurement result set in the gaze direction measurement information indicating that the time series information is nth (n is an integer of 2 or more and N or less), and the time series information is (n−1) th. The difference from the gaze direction measurement result set in the gaze direction measurement information indicating is calculated. The change amount calculation unit 16 sets the calculated difference as the gaze change amount for the nth time-series information. The line-of-sight change amount is calculated for each of the vertical direction and the horizontal direction. The change amount calculation unit 16 writes the line-of-sight change amount information in which the time series information and the line-of-sight change amount are set in the storage unit 15.

  The input unit 17 receives input of information indicating the analysis target period from the user (step S170). The diagnosis support information generation unit 18 generates diagnosis support information based on the face direction change amount information, the line-of-sight change amount information, and the facial expression estimation information in which time-series information within the analysis target period is set (step S180). Specifically, the following processing is performed.

  The diagnosis support information generation unit 18 reads face direction change amount information in which time series information included in the analysis target period is set from the storage unit 15. The diagnosis support information generation unit 18 includes, in the read face direction change amount information, the face direction change amount information whose vertical direction face direction change amount is equal to or less than a predetermined threshold, and the left and right direction face direction change amount is predetermined. The face direction change amount information that is equal to or smaller than the threshold is specified. The diagnosis support information generation unit 18 divides the number of face direction change amount information whose face direction change amount in the vertical direction is equal to or less than the threshold by the number of face direction change amount information read from the storage unit 15, and calculates the calculation result. The ratio is the ratio of no change in face direction in the vertical direction in the analysis target period. Further, the diagnosis support information generation unit 18 divides the number of face direction change amount information whose face direction change amount in the left-right direction is equal to or less than the threshold by the number of face direction change amount information read from the storage unit 15, and calculates the same. The result is defined as the ratio of the face direction change in the left-right direction in the analysis target period. The threshold value is “0”, for example, but the threshold value can be arbitrarily set, and the threshold value may be different between the vertical direction and the horizontal direction.

  Subsequently, the diagnosis support information generation unit 18 reads the line-of-sight change amount information in which the time series information included in the analysis target period is set from the storage unit 15. The diagnosis support information generation unit 18 includes the gaze change amount information in which the gaze change amount in the vertical direction is equal to or less than a predetermined threshold, and the gaze change amount in the left-right direction is equal to or less than the predetermined threshold among the read gaze change amount information. Identify gaze change information. The diagnosis support information generation unit 18 divides the number of line-of-sight change information whose vertical line-of-sight change is equal to or less than a predetermined threshold by the number of line-of-sight change information read from the storage unit 15 and analyzes the calculation result It is the ratio of no change in the line of sight in the vertical direction during the target period. Further, the diagnosis support information generation unit 18 divides the number of line-of-sight change amount information in which the line-of-sight change amount in the left-right direction is equal to or less than a predetermined threshold by the number of line-of-sight change amount information read out from the storage unit 15. Is the rate of no change in the line of sight in the horizontal direction in the analysis target period. The threshold value is “0”, for example, but the threshold value can be arbitrarily set, and the threshold value may be different between the vertical direction and the horizontal direction.

  Further, the diagnosis support information generation unit 18 reads the facial expression estimation information in which the time series information included in the analysis target period is set from the storage unit 15. The diagnosis support information generation unit 18 calculates statistical information related to the facial expression based on the read facial expression estimation information. For example, the ratio of facial expression estimation information in which each facial expression appears in the read facial expression estimation information is calculated. When the estimation result of each facial expression is represented by a score, the facial expression may be regarded as appearing when the score is higher than a predetermined threshold. The statistical information may be a total score value for each facial expression.

  The diagnosis support information generation unit 18 includes a ratio of no change in face orientation in the vertical direction, a ratio of no change in face orientation in the horizontal direction, a ratio of no change in gaze direction in the vertical direction, a ratio of no change in gaze direction in the horizontal direction, Diagnosis support information in which the appearance ratio of each facial expression is set is generated. The output unit 19 outputs the diagnosis support information generated by the diagnosis support information generation unit 18 (step S190).

  If face direction change calculation (step S150) is performed after face orientation measurement (step S120) and gaze change calculation (step S160) is performed after gaze direction measurement (step S130), diagnosis support is performed. The information providing apparatus 1 may change the order of the processes in steps S120 to S160 in the above embodiment, or may perform some processes in parallel.

  In the above description, the diagnosis support information providing apparatus 1 generates face direction change amount information, line-of-sight change amount information, and facial expression estimation information for all time-series image data acquired by the image acquisition unit 11. Then, the diagnosis support information generation unit 18 generates diagnosis support information using the face direction change amount information, the line-of-sight change amount information, and the facial expression estimation information of the analysis target period input by the input unit 17. Thereby, for example, video data for several days can be input to the diagnosis support information providing apparatus 1 and diagnosis support information for a period arbitrarily designated by the user such as every day or every time zone can be quickly provided. In addition, you may perform the process of step S170 which inputs an analysis object period before step S120. In this case, the diagnosis support information providing apparatus 1 uses the time-series image data included in the analysis target period among the time-series image data acquired by the image acquisition unit 11 to perform steps S120 to S160 and steps S180 to S180. The process of step S190 is performed. Further, when the time-series image data acquired in step S110 is set as one analysis target period, the process in step S170 may be omitted.

Subsequently, a test result using the diagnosis support information providing apparatus 1 of the present embodiment will be described.
Here, the test results for three subjects with dementia are shown. The three subjects are a female subject A in her 70s, a female subject B in her 90s, and a female subject C in her 80s. Subject A has a degree of care of 4 years after diagnosis of dementia. Subject B has a degree of care of 3 years after diagnosis of dementia. Subject C has a care level of 6 years after diagnosis of Alzheimer-type dementia. Each subject ingested a food memion containing 6 grams of medium chain fatty acid oil (Nisshin Oilio Group Co., Ltd.), which has been studied for its effects on dementia, three times a day for two months. Hereinafter, the one month before the start of the intake of the memory-on is referred to as “before intake”, and the one month after the end of the intake of the memory-on is referred to as “after the intake”. For each subject, daily caregivers were evaluated for peripheral symptoms peculiar to dementia (hereinafter referred to as BPSD) and facial expressions before ingestion, 1 month after ingestion, 2 months after ingestion, and after ingestion.

  The effects of medium-chain fatty acids on Alzheimer's dementia are described in, for example, the following literature: “Samuel T Henderson, Janet L Vogel, Linda J Barr, Fiona Garvin, Julie J Jones, and Lauren C Costantini,“ Study of The ketogenic agent AC-1202 in mild to moderate Alzheimer's disease: a randomized, double-blind, placebo-controlled, multicenter trial, "Nutrition & Metabolism 2009, 6: 31".

  FIG. 5 is a diagram showing evaluation results by daily caregivers regarding BPSD and facial expressions in each period of each subject. 5A shows the evaluation result of the subject A, FIG. 5B shows the evaluation result of the subject B, and FIG. 5C shows the evaluation result of the subject C. The Abe BPSD score was used for the evaluation of BPSD. The Abe BPSD score is a maximum of 44 points, and the higher the score, the more “BPSD worse”. Regarding facial expressions, the increase or decrease of each item was evaluated in five stages based on “smiling time”, “richness of facial expression”, and “no expression time” before ingestion.

  It was confirmed that the subject A showed improvement in BPSD and a change in facial expression in a period continued from a certain point in time, and the change continued. A period before the time when the change occurs is referred to as period 1, and a period after the period is referred to as period 2. Period 1 is a period from before ingestion to 1 month after the start of ingestion of memory-on, and period 2 is a period from 2 months after ingestion of memory-on to after ingestion. On the other hand, for subject B and subject C, the Abe BPSD score was not improved from period 1 to period 2, and there was almost no change in facial expression.

  FIG. 6 is a diagram illustrating a test result using the diagnosis support information providing apparatus 1. The figure shows the results of analysis using video data for 5 days (Day 1 to 5) in period 1 and video data for 5 days (Day 1 to 5) in period 2 for each subject A to C. Yes. The video data is a video of about 30 minutes of snacks and talk time in a time zone that includes 30 minutes of recreation, 90 minutes of snacks and talk time performed every Tuesday and Thursday from 16:00 to 16:00. As the video data, a video obtained by photographing the subject with a home video camera was used, and the diagnosis support information providing apparatus 1 obtained time series image data at intervals of 0.05 seconds from the video data and analyzed. 6A shows the analysis result of the subject A, FIG. 6B shows the analysis result of the subject B, and FIG. 6C shows the analysis result of the subject C. In the figure, the result of the T test is also shown.

  As shown in FIG. 6 (a), with respect to the subject A who improved the facial expression, the ratio of the face direction change in the up and down direction and the left and right direction in the period 2 and the line of sight change in the up and down direction and the left and right direction were not changed. The rate has decreased significantly, and movements in both face direction and line of sight have increased. On the other hand, as shown in FIGS. 6B and 6C, for subjects B and C who did not improve their facial expressions, the face orientation changes in the vertical and horizontal directions in periods 1 and 2 respectively. There was no significant difference in the ratio of none and the ratio of no change in the line of sight in the vertical and horizontal directions.

  From FIG. 5, there is a correlation between the BPSD evaluation result and the expressionless evaluation result. 5 and 6, it can be seen that there is a correlation between the evaluation result of the expressionlessness, the small change in the face direction, and the small change in the line of sight. These indicate that the information indicating the small change in face orientation and the small change in line of sight output from the diagnosis support information providing apparatus 1 can be used as information useful for diagnosis related to dementia.

  Each threshold value used by the diagnosis support information providing apparatus 1 can be arbitrarily set by the user, but each threshold value may be determined by learning. For example, the diagnosis support information providing apparatus 1 uses the time-series image data of each of a plurality of subjects as learning data, and performs the above-described analysis processing while changing each threshold value within a predetermined range. The diagnosis support information providing apparatus 1 is set with a threshold value when diagnosis support information that matches well with the BPSD and facial expression evaluation results of each subject is obtained.

Below, the modification of the diagnostic assistance information provision apparatus 1 is shown.
The change amount calculation unit 16 may use a value obtained by averaging the face direction measurement result and the line-of-sight measurement result for each predetermined number for calculation of the face direction change amount and the line-of-sight change amount. For example, when time-series image data at intervals of 0.05 seconds is used, a face orientation measurement result and a line-of-sight measurement result every 0.05 seconds are obtained. For example, the face direction measuring unit 12 and the line-of-sight direction measuring unit 13 set 125 consecutive units as one section, and give time series information to each section. The face orientation measurement unit 12 writes the face orientation measurement information in which the time series information and the average value of the face orientation measurement results in the section to which the time series information is assigned are set in the storage unit 15. Similarly, the gaze direction measurement unit 13 writes the gaze direction measurement information in which the time series information and the average value of the gaze measurement results in the section to which the time series information is set are set in the storage unit 15. Thereby, the change amount calculation unit 16 calculates the face direction change amount and the line-of-sight change amount using the time-series face direction measurement result information and the line-of-sight direction measurement information at intervals of 6.25 seconds as in the above-described process. Since patients with dementia do not often see tremors, the processing load of the diagnostic support information providing apparatus 1 can be reduced while appropriately maintaining the information accuracy by appropriately determining the interval.

  In addition, the diagnosis support information providing apparatus 1 may further output the result of determining the possibility of dementia as diagnosis support information as follows. First, as a learning stage, the diagnosis support information providing apparatus 1 uses the time-series image data of each subject who is not dementia and each subject with dementia as learning data, and performs the above-described analysis processing. The ratio of the face direction change in the up-down direction and the left-right direction of the subject, the ratio of the gaze change in the up-down direction and the left-right direction, and the ratio of each expression are calculated. The diagnosis support information providing apparatus 1 determines the ratio of the face direction change in the up and down direction and the left and right direction, the ratio of no change in the line of sight in the up and down direction and the left and right direction based on the calculation result for each subject and the presence or absence of dementia. And, the correlation between the ratio of each facial expression and the presence or absence of dementia is learned. By this learning, for example, the ratio of vertical and horizontal face direction changes for determining the possibility of dementia, the vertical and horizontal line-of-sight ratios, the ratio of each facial expression threshold Get. When the analysis process is performed on the subject, the diagnosis support information generation unit 18 uses the threshold values obtained in the learning stage and the ratio of the face direction change in the vertical and horizontal directions obtained for the subject to be analyzed. The presence / absence of dementia is determined based on the results of comparing the ratios of the vertical and horizontal directions without any change in the line of sight and the ratios of the facial expressions. The diagnosis support information generation unit 18 may determine the degree of possibility of dementia based on the magnitude of deviation from the threshold. It should be noted that facial expressions determined to have no correlation with the presence or absence of dementia are not used to determine the presence or absence of suspected dementia.

  In addition, the correlation may be, for example, a dementia suspicion using, as parameters, the ratio of no change in face orientation in the vertical direction and the horizontal direction, the ratio of no change in gaze direction in the vertical direction and the horizontal direction, and the ratio of each facial expression. May be a function (hereinafter referred to as “discriminant expression”) for calculating the degree of. In this case, in the learning stage, the diagnosis support information providing apparatus 1 is based on the result obtained by performing the above analysis process using the learning data of each subject and the presence or absence of the subject's dementia. The value of each coefficient used in the discriminant is determined. When the analysis process is performed on the subject, the diagnosis support information generation unit 18 determines the ratio of the face direction change in the vertical direction and the horizontal direction obtained for the subject to be analyzed, and the change in the line of sight in the vertical direction and the horizontal direction. Using the value obtained by substituting the ratio of each expression and the ratio of each facial expression into the learned discriminant, the presence / absence or degree of dementia is determined.

  According to the present embodiment, it is possible to provide information useful for diagnosis of a brain function disease such as dementia by easy measurement using an image or the like taken of a subject's daily life. The subject does not need to wear an instrument for measurement or the like, and does not need to perform any operation according to the instruction for measurement, so the burden on the subject can be suppressed. This means that it is not necessary for the measurer to attach a measurement instrument or the like to the subject, and it is not necessary to instruct the subject to perform the measurement operation. The burden can be reduced.

[Second Embodiment]
FIG. 7 is a configuration diagram of the diagnosis support information providing system 4 of the present embodiment. The diagnosis support information providing system 4 shown in the figure has a configuration in which the diagnosis support information providing apparatus 1 according to the first embodiment and one or more terminal devices 6 are connected via a network 5. The network 5 is a public network such as the Internet network, for example, but may be a private network, a virtual private network, or a combination thereof. The terminal device 6 is a computer terminal and transmits video data to the diagnosis support information providing device 1. The diagnosis support information providing device 1 receives video data from the terminal device 6 and performs the same processing as in the first embodiment. When the analysis target period is designated, the diagnosis support information providing apparatus 1 receives the analysis target period information from the terminal device 6 instead of the input by the input unit 17. The output unit 19 of the diagnosis support information providing device 1 returns the diagnosis support information to the terminal device 6 that is the transmission source of the video data.

  Note that the terminal device 6 may have a part of the functional units of the diagnosis support information providing device 1. Which functional unit of the diagnosis support information providing device 1 is provided in the terminal device 6 can be arbitrarily determined. For example, the terminal device 6 includes an image acquisition unit 11, a face orientation measurement unit 12, a gaze direction measurement unit 13, and a facial expression estimation unit 14, and processing results of the face orientation measurement unit 12, the gaze direction measurement unit 13, and the facial expression estimation unit 14 are obtained. You may transmit to the diagnostic assistance information provision apparatus 1. FIG.

  The terminal device 6 may register the video data in the database server and notify the diagnosis support information providing device 1 of the address and video data name of the database server that registered the video data. The image acquisition unit 11 of the diagnosis support information providing apparatus 1 accesses the database server using the address and the video data name received from the terminal device 6 and reads the video data.

  According to the present embodiment, it is possible to provide diagnosis support information to a plurality of users by using one diagnosis support information providing apparatus 1.

[Third Embodiment]
In the present embodiment, comprehensive diagnosis support information is provided using the diagnosis support information output from the diagnosis support information providing apparatus 1 and other information.
FIG. 8 is a functional block diagram of the diagnosis support system 7 of the present embodiment. The diagnosis support system 7 includes the diagnosis support information providing apparatus 1 according to the first embodiment and an analysis unit 71. The diagnosis support system 7 may be realized by a single computer device or may be realized by a plurality of computer devices connected via a network. When realized by a plurality of computer devices, which computer device realizes which functional unit is arbitrary, and one functional unit may be realized by a plurality of computer devices.

  The analysis unit 71 generates and outputs diagnosis support information different from the diagnosis support information providing apparatus 1 based on the diagnosis support information output by the diagnosis support information providing apparatus 1 and other information. The analysis unit 71 can also use arbitrary feature points obtained from the image data used for the analysis processing by the diagnosis support information providing apparatus 1 for generating diagnosis support information. Note that the diagnosis support information providing system 4 of the second embodiment may include the diagnosis support system 7 of the present embodiment instead of the diagnosis support information providing apparatus 1. In this case, the diagnosis support system 7 receives information necessary for processing in the analysis unit 71 from the terminal device 6 in addition to the diagnosis support information output from the diagnosis support information providing apparatus 1.

  According to the present embodiment, it is possible to further provide information useful for diagnosis using the analysis result of the diagnosis support information providing apparatus 1.

  In the embodiment described above, dementia has been described as an example of a brain function disease. However, the diagnosis support information providing apparatus 1 performs the same process as described above even when a person suffering from another brain function disease is a target. Can provide information useful for diagnosis.

  The diagnosis support information providing apparatus 1 and the diagnosis support system 7 described above have a computer system therein. The operation process of the diagnosis support information providing apparatus 1 and the diagnosis support system 7 is stored in a computer-readable recording medium in the form of a program, and the computer system reads and executes this program, whereby the above processing is performed. Is done. The computer system here includes a CPU, various memories, an OS, and hardware such as peripheral devices.

Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory in a computer system serving as a server or a client in that case, and a program that holds a program for a certain period of time are also included. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 1 ... Diagnosis support information provision apparatus 2 ... Subject 3 ... Imaging device 4 ... Diagnosis support information provision system 5 ... Network 6 ... Terminal apparatus 7 ... Diagnosis support system 11 ... Image acquisition part 12 ... Face direction measurement part 13 ... Gaze direction Measurement unit 14 ... facial expression estimation unit 15 ... storage unit 16 ... change amount calculation unit 17 ... input unit 18 ... diagnosis support information generation unit 19 ... output unit 71 ... analysis unit

Claims (5)

A face orientation measurement unit for measuring the face orientation of the subject in each of the time-series image data;
Based on the face orientation measured by the face orientation measurement unit, a change amount calculation unit that calculates a time-series face direction change amount;
A diagnosis support information generating unit that calculates a ratio that the face direction change amount is equal to or less than a predetermined value in a predetermined period based on the face direction change amount calculated by the change amount calculation unit, and generates diagnosis support information indicating a calculation result When,
An output unit that outputs the diagnosis support information calculated by the diagnosis support information generation unit;
A diagnostic support information providing apparatus comprising: Further comprising a gaze direction measuring unit that measures the gaze direction of the subject in each of the time-series image data,
The change amount calculation unit calculates a time-series gaze change amount based on the gaze direction measured by the gaze direction measurement unit,
The diagnosis support information generation unit calculates a rate at which the line-of-sight change amount is equal to or less than a predetermined value in the predetermined period based on the time-series line-of-sight change amount calculated by the change amount calculation unit, and further calculates a calculation result Set in diagnosis support information,
The diagnosis support information providing apparatus according to claim 1. The diagnosis support information generation unit may be configured based on the ratio at which the face direction change amount is equal to or less than a predetermined value, or the ratio and the line-of-sight change amount at which the face direction change amount is equal to or less than a predetermined value. Based on the ratio, determine the possibility or degree of brain function disease,
The output unit outputs a determination result by the diagnosis support information generation unit.
The diagnosis support information providing apparatus according to claim 2. Further comprising a facial expression estimation unit that estimates the facial expression of the subject in each of the time-series image data,
The diagnosis support information generation unit further determines the possibility or degree of a brain function disease using the facial expression estimated by the facial expression estimation unit for the predetermined period.
The diagnosis support information provision apparatus according to any one of claims 1 to 3. A face orientation measuring step for measuring the face orientation of the subject in each of the time-series image data;
Based on the face orientation measured in the face orientation measuring step, a change amount calculating step for calculating a time-series face orientation change amount;
Diagnostic support information generation for calculating a ratio that the face direction change amount is a predetermined value or less in a predetermined period based on the face direction change amount calculated in the change amount calculating step, and generating diagnosis support information indicating the calculation result Steps,
An output step for outputting the diagnosis support information calculated in the diagnosis support information generation step;
A method for providing diagnosis support information.

Images