JP2018029764A - Diagnosis support apparatus, diagnosis support method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a diagnosis support apparatus capable of supporting the diagnosis of the risk of attention deficit-hyperkinetic trouble highly accurately.SOLUTION: A diagnosis support apparatus comprises: an image data acquisition unit for acquiring the image data of the eyes of a subject; a fixation point detection unit for detecting the position data of the fixation point of the subject on the basis of the image data; a moving data acquisition unit for acquiring the moving data of the eyes of the subject on the basis of the image data; and an output control unit for outputting the moving data.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a diagnosis support apparatus, a diagnosis support method, and a computer program.

  Persons with developmental disabilities have the characteristic of not seeing the other person's eyes. In particular, a person with attention deficit hyperactivity disorder (ADHD) has a feature that it is difficult to concentrate on one thing and the gaze point is easy to move. Therefore, a technique for detecting a subject's gazing point and diagnosing a subject's risk of developmental disability has been proposed. In addition, a technique for diagnosing a subject's risk of attention deficit / hyperactivity disorder by saccade movement of the eye has been proposed.

JP 2011-206542 A JP 2002-360518 A

  In the medical field, diagnosis and evaluation of a subject's risk of attention deficit / hyperactivity disorder is performed, and a response to attention deficit / hyperactivity disorder is determined based on the diagnosis and evaluation results. Therefore, there is a demand for a technology that can support diagnosis of the risk of attention deficit / hyperactivity disorder with high accuracy.

  An aspect of the present invention is to provide a diagnosis support apparatus, a diagnosis support method, and a computer program that can support diagnosis of a risk of attention deficit / hyperactivity disorder with high accuracy.

  According to the first aspect of the present invention, an image data acquisition unit that acquires image data of a subject's eye, a gazing point detection unit that detects position data of the gazing point of the subject based on the image data, A diagnosis support apparatus is provided that includes a movement data acquisition unit that acquires movement data of the subject's eyes based on the image data, and an output control unit that outputs the movement data.

  According to the second aspect of the present invention, acquiring image data of the eye of the subject, detecting position data of the gazing point of the subject based on the image data, and based on the image data A diagnostic support method is provided that includes obtaining movement data of the subject's eyes and outputting the movement data.

  According to the third aspect of the present invention, the computer acquires image data of the eye of the subject, detects position data of the subject's point of gaze based on the image data, and the image data Based on the above, there is provided a computer program for executing the movement data of the eye of the subject and outputting the movement data.

  ADVANTAGE OF THE INVENTION According to the aspect of this invention, the diagnostic assistance apparatus, diagnostic assistance method, and computer program which can support the diagnosis of the risk of attention deficit / hyperactivity disorder with high precision are provided.

FIG. 1 is a perspective view schematically showing an example of a visual line detection device according to the present embodiment. FIG. 2 is a diagram schematically illustrating a positional relationship among the display device, the stereo camera device, the light source, and the eye of the subject according to the present embodiment. FIG. 3 is a diagram illustrating an example of a hardware configuration of the visual line detection device according to the present embodiment. FIG. 4 is a functional block diagram illustrating an example of a visual line detection device according to the present embodiment. FIG. 5 is a schematic diagram for explaining the calculation method of the position data of the corneal curvature center according to the present embodiment. FIG. 6 is a schematic diagram for explaining the calculation method of the position data of the corneal curvature center according to the present embodiment. FIG. 7 is a flowchart illustrating an example of a gaze detection method according to the present embodiment. FIG. 8 is a schematic diagram for explaining an example of calibration processing according to the present embodiment. FIG. 9 is a flowchart showing an example of the calibration process according to the present embodiment. FIG. 10 is a schematic diagram for explaining an example of a gazing point detection process according to the present embodiment. FIG. 11 is a flowchart illustrating an example of a gazing point detection process according to the present embodiment. FIG. 12 is a diagram illustrating an example of a diagnostic image displayed on the display device by the display control unit according to the present embodiment. FIG. 13 is a diagram schematically showing movement data of the eyes of a subject with a standard development. FIG. 14 is a diagram schematically illustrating eye movement data of a subject with ADHD. FIG. 15 is a schematic diagram for explaining an example of a method for acquiring eye movement data of a subject according to the present embodiment. FIG. 16 is a diagram schematically showing the relationship between the moving average point and the moving distance in the global coordinate system. FIG. 17 is a flowchart illustrating an example of the diagnosis support method according to the present embodiment.

  Hereinafter, embodiments according to the present invention will be described with reference to the drawings, but the present invention is not limited thereto. The components of the embodiments described below can be combined as appropriate. Some components may not be used.

  In the following description, a three-dimensional global coordinate system is set and the positional relationship of each part is demonstrated. The direction parallel to the X axis of the predetermined surface is the X axis direction, the direction parallel to the Y axis of the predetermined surface orthogonal to the X axis is the Y axis direction, and is parallel to the Z axis orthogonal to each of the X axis and the Y axis. Let the direction be the Z-axis direction. The predetermined plane includes an XY plane.

[Outline of eye gaze detection device]
FIG. 1 is a perspective view schematically showing an example of a visual line detection device 100 according to the present embodiment. In the present embodiment, the line-of-sight detection device 100 is used in a diagnosis support device that supports diagnosis of developmental disabilities.

  As shown in FIG. 1, the line-of-sight detection device 100 includes a display device 101, a stereo camera device 102, and a light source 103.

  The display device 101 includes a flat panel display such as a liquid crystal display (LCD) or an organic EL display (OELD).

  In the present embodiment, the display screen 101S of the display device 101 is substantially parallel to the XY plane. The X-axis direction is the left-right direction of the display screen 101S, the Y-axis direction is the up-down direction of the display screen 101S, and the Z-axis direction is the depth direction orthogonal to the display screen 101S.

  The stereo camera device 102 captures a subject and acquires image data of the subject. The stereo camera device 102 includes a first camera 102A and a second camera 102B arranged at different positions. The stereo camera device 102 is disposed below the display screen 101S of the display device 101. The first camera 102A and the second camera 102B are arranged in the X-axis direction. The first camera 102A is arranged in the −X direction with respect to the second camera 102B. Each of the first camera 102A and the second camera 102B includes an infrared camera, and includes, for example, an optical system capable of transmitting near-infrared light having a wavelength of 850 [nm] and an imaging element capable of receiving near-infrared light.

  The light source 103 emits detection light. The light source 103 includes a first light source 103A and a second light source 103B arranged at different positions. The light source 103 is disposed below the display screen 101S of the display device 101. The first light source 103A and the second light source 103B are arranged in the X-axis direction. The first light source 103A is arranged in the −X direction with respect to the first camera 102A. The second light source 103B is arranged in the + X direction with respect to the second camera 102B. Each of the first light source 103A and the second light source 103B includes an LED (Light Emitting Diode) light source, and can emit, for example, near infrared light having a wavelength of 850 [nm]. Note that the first light source 103A and the second light source 103B may be disposed between the first camera 102A and the second camera 102B.

  FIG. 2 is a diagram schematically showing a positional relationship among the display device 101, the stereo camera device 102, the light source 103, and the eye 111 of the subject according to the present embodiment.

  The light source 103 emits infrared light as detection light to illuminate the eye 111 of the subject. The stereo camera device 102 captures the eye 111 with the second camera 102B when the detection light emitted from the first light source 103A is applied to the eye 111, and the detection light emitted from the second light source 103B is applied to the eye 111. The eye 111 is imaged by the first camera 102A when irradiated.

  A frame synchronization signal is output from at least one of the first camera 102A and the second camera 102B. The first light source 103A and the second light source 103B emit detection light based on the frame synchronization signal. The first camera 102A acquires image data of the eye 111 when the detection light emitted from the second light source 103B is irradiated to the eye 111. The second camera 102B acquires image data of the eye 111 when the detection light emitted from the first light source 103A is irradiated on the eye 111.

  When the eye 111 is irradiated with detection light, a part of the detection light is reflected by the pupil 112. The light reflected by the pupil 112 enters the stereo camera device 102. When the eye 111 is irradiated with detection light, a cornea reflection image 113 is formed on the eye 111. The cornea reflection image 113 is a reflection image of the light source 103 on the cornea surface. The light from the cornea reflection image 113 enters the stereo camera device 102.

  By appropriately setting the relative positions of the first camera 102A and the second camera 102B and the first light source 103A and the second light source 103B, the intensity of light incident on the stereo camera device 102 from the pupil 112 is reduced, and the cornea The intensity of light incident on the stereo camera device 102 from the reflected image 113 is increased. That is, the image of the pupil 112 acquired by the stereo camera device 102 has low brightness, and the image of the cornea reflection image 113 has high brightness. The stereo camera device 102 can detect the position of the pupil 112 and the position of the cornea reflection image 113 based on the luminance of the acquired image.

[Hardware configuration]
FIG. 3 is a diagram illustrating an example of a hardware configuration of the visual line detection device 100 according to the present embodiment. As shown in FIG. 3, the line-of-sight detection device 100 includes a display device 101, a stereo camera device 102, a light source 103, a computer system 20, an input / output interface device 30, a drive circuit 40, an output device 50, An input device 60 and an audio output device 70 are provided. The computer system 20 includes an arithmetic processing device 20A and a storage device 20B. A computer program 20C is stored in the storage device 20B.

  The computer system 20, the drive circuit 40, the output device 50, the input device 60, and the audio output device 70 perform data communication via the input / output interface device 30.

  The arithmetic processing unit 20A includes a microprocessor such as a CPU (Central Processing Unit). The storage device 20B includes a nonvolatile memory such as a ROM (Read Only Memory) or a volatile memory such as a RAM (Random Access Memory). The arithmetic processing device 20A performs arithmetic processing according to the computer program 20C stored in the storage device 20B.

  The drive circuit 40 generates a drive signal and outputs it to the display device 101, the stereo camera device 102, and the light source 103. In addition, the drive circuit 40 supplies the image data of the eye 111 acquired by the stereo camera device 102 to the computer system 20 via the input / output interface device 30.

  The output device 50 includes a display device such as a flat panel display. The output device 50 may include a printing device. The input device 60 generates input data when operated. The input device 60 includes a keyboard or mouse for a computer system. The input device 60 may include a touch sensor provided on the display screen of the output device 50 that is a display device. The audio output device 70 includes a speaker and outputs, for example, audio for prompting the subject to pay attention.

  In the present embodiment, the display device 101 and the computer system 20 are separate devices. Note that the display device 101 and the computer system 20 may be integrated. For example, when the line-of-sight detection device 100 includes a tablet personal computer, the computer system 20, the input / output interface device 30, the drive circuit 40, and the display device 101 may be mounted on the tablet personal computer.

  FIG. 4 is a functional block diagram illustrating an example of the line-of-sight detection device 100 according to the present embodiment. As shown in FIG. 4, the input / output interface device 30 includes an input / output unit 302. The drive circuit 40 generates a drive signal for driving the display device 101 and outputs the drive signal to the display device 101, and generates a drive signal for driving the first camera 102A to generate the first camera. A first camera input / output unit 404A that outputs to the second camera 102B, a second camera input / output unit 404B that generates a drive signal for driving the second camera 102B and outputs it to the second camera 102B, the first light source 103A and the second light source 103A. A light source drive unit 406 that generates a drive signal for driving the two light sources 103B and outputs the drive signals to the first light source 103A and the second light source 103B. The first camera input / output unit 404 </ b> A supplies the image data of the eye 111 acquired by the first camera 102 </ b> A to the computer system 20 via the input / output unit 302. The second camera input / output unit 404B supplies the image data of the eye 111 acquired by the second camera 102B to the computer system 20 via the input / output unit 302.

  The computer system 20 controls the line-of-sight detection device 100. The computer system 20 includes a display control unit 202, a light source control unit 204, an image data acquisition unit 206, an input data acquisition unit 208, a position detection unit 210, a curvature center calculation unit 212, and a gaze point detection unit 214. , A movement data acquisition unit 216, an area setting unit 218, a determination unit 220, a calculation unit 222, a storage unit 224, an evaluation unit 226, and an output control unit 228. The functions of the computer system 20 are exhibited by the arithmetic processing unit 20A, the storage unit 20B, and the computer program 20C stored in the storage unit 20B.

  The display control unit 202 causes the display device 101 to display diagnostic image data indicating a diagnostic image for supporting the diagnosis of developmental disability.

  The light source control unit 204 controls the operation state of the first light source 103A and the second light source 103B by controlling the light source driving unit 406. The light source control unit 204 controls the first light source 103A and the second light source 103B so that the first light source 103A and the second light source 103B emit detection light at different timings.

  The image data acquisition unit 206 acquires the image data of the eye 111 of the subject acquired by the stereo camera device 102 including the first camera 102A and the second camera 102B from the stereo camera device 102 via the input / output unit 302. The image data acquisition unit 206 acquires image data of the eye 111 irradiated with infrared light, which is detection light from the light source 103, from the stereo camera device 102.

  The input data acquisition unit 208 acquires input data generated by operating the input device 60 from the input device 60 via the input / output unit 302.

  The position detection unit 210 detects the position data of the pupil center based on the image data of the eye 111 acquired by the image data acquisition unit 206. The position detection unit 210 detects position data of the corneal reflection center based on the image data of the eye 111 acquired by the image data acquisition unit 206. The pupil center is the center of the pupil 112. The cornea reflection center is the center of the cornea reflection image 113. The position detection unit 210 detects the position data of the pupil center and the position data of the corneal reflection center for each of the left and right eyes 111 of the subject.

  The curvature center calculation unit 212 calculates position data of the corneal curvature center of the eye 111 based on the image data of the eye 111 acquired by the image data acquisition unit 206.

  The gaze point detection unit 214 detects position data of the gaze point of the subject based on the image data of the eye 111 acquired by the image data acquisition unit 206. In the present embodiment, the position data of the gazing point refers to the position data of the intersection point between the subject's line-of-sight vector defined by the three-dimensional global coordinate system and the display screen 101S of the display device 101. The gaze point detection unit 214 detects the eye gaze vectors of the left and right eyes 111 of the subject based on the position data of the pupil center and the position data of the corneal curvature center acquired from the image data of the eye 111. After the gaze vector is detected, the gaze point detection unit 214 detects position data of the gaze point indicating the intersection of the gaze vector and the display screen 101S.

  The movement data acquisition unit 216 acquires eye movement data of the subject based on the image data of the eye 111 acquired by the image data acquisition unit 206. The eye movement data includes at least one of movement data of the pupil 112 and movement data of the cornea reflection image 113 that is a reflection image of the light source 103 on the cornea surface.

  The area setting unit 218 sets a specific area on the display screen 101 </ b> S of the display device 101.

  The determination unit 220 determines whether or not the gazing point exists in the specific area based on the position data of the gazing point.

  Based on the determination data of the determination unit 220, the calculation unit 222 calculates time data indicating the presence time during which the subject's gaze point is present in the specific area of the display screen 101S.

  The storage unit 224 stores data for supporting diagnosis of developmental disabilities. In the present embodiment, the storage unit 224 stores threshold data indicating a threshold for eye movement data.

  The evaluation unit 226 outputs subject evaluation data based on the threshold data stored in the storage unit 224 and the eye movement data acquired by the movement data acquisition unit 216. Further, the evaluation unit 226 outputs the evaluation data of the subject based on the time data calculated by the calculation unit 222 and the eye movement data acquired by the movement data acquisition unit 216.

  The output control unit 228 outputs data to at least one of the display device 101, the output device 50, and the audio output device 70. In the present embodiment, the output control unit 228 outputs the eye movement data acquired by the movement data acquisition unit 216 and causes the display device 101 or the output device 50 to display the eye movement data. In addition, the output control unit 228 outputs position data of the gazing point of the left and right eyes 111 of the subject and causes the display device 101 or the output device 50 to display the position data. Further, the output control unit 228 outputs the evaluation data output from the evaluation unit 226 and causes the display device 101 or the output device 50 to display the evaluation data.

[Principle of eye detection]
Next, the principle of eye gaze detection according to this embodiment will be described. In the following description, an outline of processing of the curvature center calculation unit 212 will be mainly described. The curvature center calculation unit 212 calculates position data of the corneal curvature center of the eye 111 based on the image data of the eye 111.

  5 and 6 are schematic diagrams for explaining a method of calculating position data of the corneal curvature center 110 according to the present embodiment. FIG. 5 shows an example in which the eye 111 is illuminated by one light source 103C. FIG. 6 shows an example in which the eye 111 is illuminated by the first light source 103A and the second light source 103B.

  First, the example shown in FIG. 5 will be described. The light source 103C is disposed between the first camera 102A and the second camera 102B. Pupil center 112 </ b> C is the center of pupil 112. The cornea reflection center 113 </ b> C is the center of the cornea reflection image 113. In FIG. 5, the pupil center 112C indicates the pupil center when the eye 111 is illuminated by one light source 103C. The corneal reflection center 113C indicates the corneal reflection center when the eye 111 is illuminated by one light source 103C.

  The corneal reflection center 113 </ b> C exists on a straight line connecting the light source 103 </ b> C and the corneal curvature center 110. The corneal reflection center 113C is positioned at the midpoint between the corneal surface and the corneal curvature center 110. The corneal curvature radius 109 is the distance between the corneal surface and the corneal curvature center 110.

  The position data of the corneal reflection center 113C is detected by the stereo camera device 102. The corneal curvature center 110 exists on a straight line connecting the light source 103C and the corneal reflection center 113C. The curvature center calculation unit 212 calculates, as position data of the corneal curvature center 110, position data that has a predetermined distance from the corneal reflection center 113C on the straight line. The predetermined value is a value determined in advance based on a general radius of curvature of the cornea and the like, and is stored in the storage unit 224.

  Next, the example shown in FIG. 6 will be described. In the present embodiment, the first camera 102A and the second light source 103B, and the second camera 102B and the first light source 103A are symmetrical with respect to a straight line passing through an intermediate position between the first camera 102A and the second camera 102B. It is arranged at the position. It can be considered that the virtual light source 103V exists at an intermediate position between the first camera 102A and the second camera 102B.

  A corneal reflection center 121 indicates a corneal reflection center in an image obtained by photographing the eye 111 with the second camera 102B. A corneal reflection center 122 indicates a corneal reflection center in an image obtained by photographing the eye 111 with the first camera 102A. A corneal reflection center 124 indicates a corneal reflection center corresponding to the virtual light source 103V.

  The position data of the cornea reflection center 124 is calculated based on the position data of the cornea reflection center 121 and the position data of the cornea reflection center 122 acquired by the stereo camera device 102. The stereo camera device 102 detects position data of the corneal reflection center 121 and position data of the corneal reflection center 122 in the local coordinate system defined by the stereo camera device 102. For the stereo camera device 102, camera calibration is performed in advance by the stereo calibration method, and conversion parameters for converting the three-dimensional local coordinate system of the stereo camera device 102 into a three-dimensional global coordinate system are calculated. The conversion parameters are stored in the storage unit 224.

  The curvature center calculation unit 212 converts the position data of the corneal reflection center 121 and the position data of the corneal reflection center 122 acquired by the stereo camera device 102 into position data in the global coordinate system using the conversion parameters. The curvature center calculator 212 calculates position data of the corneal reflection center 124 in the global coordinate system based on the position data of the corneal reflection center 121 and the position data of the corneal reflection center 122 defined in the global coordinate system.

  The corneal curvature center 110 exists on a straight line 123 connecting the virtual light source 103 </ b> V and the corneal reflection center 124. The curvature center calculation unit 212 calculates position data on the straight line 123 where the distance from the corneal reflection center 124 is a predetermined value as position data of the corneal curvature center 110. The predetermined value is a value determined in advance based on a general radius of curvature of the cornea and the like, and is stored in the storage unit 224.

  As described with reference to FIG. 6, even when there are two light sources, the corneal curvature center 110 is calculated by a method similar to the method when there is one light source.

  The corneal curvature radius 109 is the distance between the corneal surface and the corneal curvature center 110. Accordingly, the corneal curvature radius 109 is calculated by calculating the position data of the corneal surface and the position data of the corneal curvature center 110.

  Thus, in the present embodiment, position data of the corneal curvature center 110, position data of the corneal reflection center 124, and position data of the pupil center 112C in the global coordinate system are calculated.

  The gaze point detection unit 214 can detect the gaze vector of the subject based on the position data of the pupil center 112C and the position data of the corneal curvature center 110. In addition, the gazing point detection unit 214 can detect gazing point position data indicating the intersection of the line-of-sight vector and the display screen 101S.

[Gaze detection method]
Next, an example of a line-of-sight detection method according to this embodiment will be described. FIG. 7 is a flowchart illustrating an example of a gaze detection method according to the present embodiment. In the present embodiment, calibration processing (step S100) including position data calculation processing of the corneal curvature center 110 and distance data calculation processing between the pupil center 112C and the corneal curvature center 110, and gaze point detection processing (step S200). ) Is implemented.

(Calibration process)
The calibration process (step S100) will be described. FIG. 8 is a schematic diagram for explaining an example of calibration processing according to the present embodiment. The calibration process includes calculating position data of the corneal curvature center 110 and calculating a distance 126 between the pupil center 112C and the corneal curvature center 110.

  A target position 130 for allowing the subject to gaze is set. The target position 130 is defined in the global coordinate system. In the present embodiment, the target position 130 is set to the center position of the display screen 101S of the display device 101, for example. The target position 130 may be set to the end position of the display screen 101S.

  The display control unit 202 displays the target image at the set target position 130. This makes it easier for the subject to gaze at the target position 130.

  A straight line 131 is a straight line connecting the virtual light source 103V and the corneal reflection center 113C. A straight line 132 is a straight line connecting the target position 130 and the pupil center 112C. The corneal curvature center 110 is an intersection of the straight line 131 and the straight line 132. The curvature center calculation unit 212 is based on the position data of the virtual light source 103V, the position data of the target position 130, the position data of the pupil center 112C, and the position data of the corneal reflection center 113C, and the position data of the corneal curvature center 110. Can be calculated.

  FIG. 9 is a flowchart showing an example of the calibration process (step S100) according to the present embodiment. The output control unit 228 displays the target image on the display screen 101S of the display device 101 (step S101). The subject can watch the target position 130 by watching the target image.

  Next, the light source control unit 204 controls the light source driving unit 406 to emit detection light from one of the first light source 103A and the second light source 103B (step S102). The stereo camera apparatus 102 images the subject's eyes with the camera having the longer distance from the light source that has emitted the detection light among the first camera 102A and the second camera 102B (step S103).

  Next, the light source control unit 204 controls the light source driving unit 406 to emit detection light from the other light source of the first light source 103A and the second light source 103B (step S104). The stereo camera apparatus 102 photographs the subject's eyes with the camera having the longer distance from the light source that emitted the detection light among the first camera 102A and the second camera 102B (step S105).

  The pupil 112 is detected by the stereo camera device 102 as a dark portion, and the cornea reflection image 113 is detected by the stereo camera device 102 as a bright portion. That is, the image of the pupil 112 acquired by the stereo camera device 102 has low brightness, and the image of the cornea reflection image 113 has high brightness. The position detection unit 210 can detect the position data of the pupil 112 and the position data of the cornea reflection image 113 based on the luminance of the acquired image. Further, the position detection unit 210 calculates the position data of the pupil center 112 </ b> C based on the image data of the pupil 112. Further, the position detection unit 210 calculates position data of the corneal reflection center 113C based on the image data of the corneal reflection image 113 (step S106).

  The position data detected by the stereo camera device 102 is position data defined by a three-dimensional local coordinate system. The position detection unit 210 uses the conversion parameters stored in the storage unit 224 to perform coordinate conversion on the position data of the pupil center 112C and the position data of the cornea reflection center 113C detected by the stereo camera device 102, and perform global conversion. The position data of the pupil center 112C and the position data of the cornea reflection center 113C defined by the coordinate system are calculated (step S107).

  The curvature center calculator 212 calculates a straight line 131 that connects the corneal reflection center 113C defined by the global coordinate system and the virtual light source 103V (step S108).

  Next, the curvature center calculation unit 212 calculates a straight line 132 connecting the target position 130 defined on the display screen 101S of the display device 101 and the pupil center 112C (step S109). The curvature center calculation unit 212 obtains an intersection between the straight line 131 calculated in step S108 and the straight line 132 calculated in step S109, and sets this intersection as the corneal curvature center 110 (step S110).

  The curvature center calculation unit 212 calculates the distance 126 between the pupil center 112C and the corneal curvature center 110, and stores it in the storage unit 224 (step S111). The stored distance is used to calculate the corneal curvature center 110 in the point-of-gaze detection in step S200.

(Gaze point detection process)
Next, the gazing point detection process (step S200) will be described. The gazing point detection process is performed after the calibration process. The gaze point detection unit 214 calculates the gaze vector of the subject and the position data of the gaze point based on the image data of the eye 111.

  FIG. 10 is a schematic diagram for explaining an example of a gazing point detection process according to the present embodiment. The gazing point detection process corrects the position of the corneal curvature center 110 using the distance 126 between the pupil center 112C and the corneal curvature center 110 obtained in the calibration process (step S100), and the corrected corneal curvature center. Including calculating a point of gaze using 110 position data.

  In FIG. 10, a gazing point 165 indicates a gazing point obtained from the corneal curvature center calculated using a general curvature radius value. The gazing point 166 indicates a gazing point obtained from the corneal curvature center calculated using the distance 126 obtained in the calibration process.

  The pupil center 112C indicates the pupil center calculated in the calibration process, and the corneal reflection center 113C indicates the corneal reflection center calculated in the calibration process.

  A straight line 173 is a straight line connecting the virtual light source 103V and the corneal reflection center 113C. The corneal curvature center 110 is the position of the corneal curvature center calculated from a general curvature radius value.

  The distance 126 is a distance between the pupil center 112C and the corneal curvature center 110 calculated by the calibration process.

  The corneal curvature center 110H indicates the position of the corrected corneal curvature center obtained by correcting the corneal curvature center 110 using the distance 126.

  The corneal curvature center 110H is obtained from the fact that the corneal curvature center 110 exists on the straight line 173 and the distance between the pupil center 112C and the corneal curvature center 110 is the distance 126. Thus, the line of sight 177 calculated when a general radius of curvature value is used is corrected to the line of sight 178. Further, the gazing point on the display screen 101S of the display device 101 is corrected from the gazing point 165 to the gazing point 166.

  FIG. 11 is a flowchart showing an example of a gazing point detection process (step S200) according to the present embodiment. Note that the processing from step S201 to step S207 shown in FIG. 11 is the same as the processing from step S102 to step S108 shown in FIG.

  The curvature center calculation unit 212 calculates a position on the straight line 173 calculated in step S207 and the distance from the pupil center 112C equal to the distance 126 obtained by the calibration process as the corneal curvature center 110H (step S208).

  The gazing point detection unit 214 calculates a line-of-sight vector connecting the pupil center 112C and the corneal curvature center 110H (step S209). The line-of-sight vector indicates the line-of-sight direction that the subject is looking at. The gaze point detection unit 214 calculates the position data of the intersection between the line-of-sight vector and the display screen 101S of the display device 101 (step S210). The position data of the intersection of the line-of-sight vector and the display screen 101S of the display device 101 is the position data of the subject's point of interest on the display screen 101S defined by the global coordinate system.

  The gaze point detection unit 214 converts gaze point position data defined in the global coordinate system into position data on the display screen 101S of the display device 101 defined in the two-dimensional coordinate system (step S211). Thereby, the position data of the gazing point on the display screen 101S of the display device 101 that the subject looks at is calculated.

[Diagnosis support method]
Next, a diagnosis support method for developmental disabilities according to the present embodiment will be described. In the present embodiment, the line-of-sight detection device 100 is used in a diagnosis support device that supports diagnosis of a developmental disorder risk of a subject. The line-of-sight detection apparatus 100 according to the present embodiment is used in a diagnosis support apparatus that supports diagnosis of a risk of attention deficit / hyperactivity disorder particularly for a subject. In the following description, the line-of-sight detection device 100 is appropriately referred to as a diagnosis support device 100. In the following description, attention deficit / hyperactivity disorder is appropriately referred to as ADHD.

(Diagnostic image)
In the diagnosis support for the risk of developmental disability, the display control unit 202 of the diagnosis support apparatus 100 displays a specific diagnosis image on the display apparatus 101. In order to evaluate the risk of the developmental disorder of the subject, the subject is instructed to watch the diagnostic image displayed on the display device 101.

  FIG. 12 is a diagram illustrating an example of a diagnostic image displayed on the display device 101 by the display control unit 202 according to the present embodiment. As shown in FIG. 12, the diagnostic image includes a region A1, a region A2, a region A3, and a region A4. Region A1 is a region including the center of the diagnostic image. The area A2 and the area A3 are areas that do not include the center of the diagnostic image, and are arranged around the area A1. In the present embodiment, a plurality of areas A2 are provided around the area A1. A plurality of areas A3 are provided around the area A1. On the display screen 101S of the display device 101, the area A1, the plurality of areas A2, and the plurality of areas A3 are separated from each other. The area A4 is a blank area other than the area A1, the area A2, and the area A3. The region A2 is arranged above, below, leftward, and rightward of the region A1. The region A3 is disposed above, below, left, and right of the region A1.

  In the example shown in FIG. 12, the area A2 is arranged at 11 places and the area A3 is arranged at 3 places around the area A1. Note that the numbers of the regions A2 and A3 and the positions with respect to the region A1 are not limited to the example in FIG.

  A specific image including a person is displayed in the central area A1. In each of the plurality of regions A2 and the plurality of regions A3, an image that is less conspicuous than the image displayed in the region A1 is displayed. In other words, images that are less likely to be noticed than the image displayed in the region A1 are displayed in the regions A2 and A3.

  In the present embodiment, a natural image moving image including a person or a large person image is displayed in the area A1. In the area A2 and the area A3, a moving image that moves slowly or a still image having a lower contrast than the image displayed in the area A1 is displayed.

  In the present embodiment, an illustration that is a still image or an animation that is a moving image is displayed in the area A2. In the area A3, an image including a geometric pattern is displayed. The image including the geometric pattern may be a still image or a moving image.

  In the diagnosis support process for the risk of developmental disabilities, the subject is instructed to watch the image displayed in the area A1 among the diagnostic images displayed on the display device 101. For example, the output control unit 228 causes the sound output device 70 to output a sound such as “please look at the center image”.

  When the subject has a typical development, the subject can gaze at the image displayed in the area A1 for a predetermined time according to the instruction.

  When the subject is ADHD, it is difficult for the subject to gaze at the image displayed in the area A1. ADHD subjects tend to see many images displayed in at least one of the area A2 and the area A3 around the area A1. In addition, when the subject has Autistic Spectrum Disorder (ASD), the subject tends to prefer the geometric pattern, and thus the subject tends to see many images of the region A3.

  That is, when an instruction is given to watch the image displayed in the area A1, the subject with regular development can continue to watch the image displayed in the area A1. A subject with ADHD tends to see an image displayed in at least one of the area A2 and the area A3 around the area A1. ASD subjects tend to see images containing the geometric pattern displayed in area A3. As described above, a specific diagnostic image is displayed on the display screen 101S of the display device 101, and the region viewed by the subject is detected, so that the typical development, ADHD, and ASD can be distinguished.

(Principle of diagnosis support for ADHD risk)
Persons with developmental disabilities, especially those with ADHD, are difficult to stay still and tend to shake their head or entire body frequently. On the other hand, a person with typical development does not shake the head or the whole body greatly. That is, the tendency of the movement state of the head or the whole body differs between the person with typical development and the person with ADHD. In the present embodiment, the diagnosis support apparatus 100 supports the diagnosis of ADHD risk of a subject by using the feature that an ADHD person frequently shakes the head or the entire body.

  In the present embodiment, the diagnosis support apparatus 100 obtains movement data of the subject's eyes and supports diagnosis of ADHD risk of the subject. When the subject shakes the head or the entire body, the position of the subject's eyes varies with the shaking of the head or the entire body. That is, if the subject shakes the head or the entire body and the position of the subject's head changes, the position of the subject's eyes also changes. Therefore, the diagnosis support apparatus 100 obtains the movement data of the head indicating the variation state of the head position of the subject by obtaining the movement data of the eye indicating the variation state of the eye position of the subject, and the subject Can provide diagnostic support and assessment of ADHD risk.

  The eye movement data of the subject includes at least one of the movement distance of the subject's eye at the specified time, the moving speed of the eye of the subject at the specified time, and the number of inflection points of the movement vector of the subject's eye at the specified time. . The inflection point of the eye movement vector refers to a point where the direction of the movement vector substantially changes by 90 [°] or more.

  The movement data acquisition unit 216 acquires movement distance data indicating the movement distance of the subject's eyes at the specified time, and derives the movement distance of the subject's head at the specified time. In addition, the movement data acquisition unit 216 acquires movement speed data indicating the movement speed of the subject's eyes at the specified time, and derives the movement speed of the subject's head at the specified time. In addition, the movement data acquisition unit 216 acquires inflection point data indicating the number of inflection points of the movement vector of the subject's eye at the specified time, and the inflection point of the movement vector of the subject's head at the specified time. Deriving a number.

  As described above, in the gazing point detection process, the subject is irradiated with infrared light, and the subject's eye 111 irradiated with the infrared light is photographed by the stereo camera device 102. The image data of the eye 111 of the subject irradiated with infrared light is acquired by the image data acquisition unit 206. Based on the image data of the eye 111 acquired by the image data acquisition unit 206, the position detection unit 210 detects position data of the pupil center 112C indicating the center of the pupil 112 in the global coordinate system. Further, the position detection unit 210 detects position data of the corneal reflection center 113C indicating the center of the corneal reflection image 113 in the global coordinate system based on the image data of the eye 111 acquired by the image data acquisition unit 206.

  The movement data acquisition unit 216 acquires the eye position data of the subject including at least one of the position data of the pupil 112 and the position data of the cornea reflection image 113 detected by the position detection unit 210, and moves the eye of the subject. Data can be acquired. The movement data of the subject's eye includes a variation state of the position of the subject's eye at a specified time. The movement data acquisition unit 216 obtains eye movement data of the subject in the global coordinate system based on at least one of the position data of the pupil 112 and the position data of the corneal reflection image 113 in the global coordinate system detected by the position detection unit 210. Can be acquired. The movement data acquisition unit 216 acquires movement data of the subject's eyes and derives movement data of the subject's head. A subject's risk of ADHD is assessed based on the subject's eye movement data or head movement data.

  The gaze point detection unit 214 calculates the subject's gaze vector based on the relative position between the subject's pupil center 112C and the corneal reflection center 113C in the global coordinate system, and the subject's gaze vector and the display screen 101S of the display device 101. The point of gaze that is the intersection of is detected.

  That is, in the present embodiment, based on the image data of the eye 111 acquired by the image data acquisition unit 206, the eye movement data of the subject defined by the global coordinate system is acquired. Further, based on the image data of the eye 111 acquired by the image data acquisition unit 206, the position data of the subject's gazing point on the display screen 101S of the display device 101 is acquired.

  FIG. 13 is a diagram schematically showing movement data of the eyes of a subject with a standard development. FIG. 13 is a diagram schematically illustrating the movement trajectory of the subject's eye in the three-dimensional global coordinate system. FIG. 13 shows the movement trajectory of the subject's eye during the specified time. In FIG. 13, the subject's eyes are shown moving in a two-dimensional plane, but actually, the subject's eyes move in a three-dimensional space.

  The eye movement data of the subject includes at least one of movement data of the pupil 112 and movement data of the cornea reflection image 113 in the global coordinate system. As described in step S107 of FIG. 9 and the like, the position detection unit 210 includes position data of the pupil center 112C indicating the center of the pupil 112 in the global coordinate system and position data of the cornea reflection center 113C indicating the center of the cornea reflection image 113. Can be detected. In the following description, for simplification of description, it is assumed that the movement data of the subject's eyes is movement data of the pupil 112 or movement data of the pupil center 112C. As described above, the movement data of the subject's eyes may be movement data of the corneal reflection image 113 of the subject or movement data of the corneal reflection center 113C.

  In this embodiment, the movement data of the subject's eyes includes the distance of movement of the subject's eyes at the specified time, the movement speed of the subject's eyes at the specified time, and the number of inflection points of the movement vector of the subject's eyes at the specified time. Including at least one of The movement data acquisition unit 216 acquires a plurality of position data of the subject's pupil 112 at a predetermined sampling period at a predetermined specified time, and based on the acquired plurality of position data of the pupil 112, Get movement data.

  In FIG. 13, a point S21 indicates the position of the pupil 112 at the start time of the specified time. Point E21 indicates the position of pupil 112 at the end of the specified time. Point A 21 is the position of the inflection point of the movement vector of pupil 112. A detection point indicated by a dot “●” in FIG. 13 indicates the position of the pupil 112 acquired at a specified sampling period. The movement trajectory of the pupil 112 is indicated by a line connecting a plurality of detection points. The inflection point of the movement vector of the pupil 112 is a point where the direction of the movement vector of the pupil 112 substantially changes by 90 [°] or more.

  The longer the movement trajectory of the pupil 112 indicated by the line between the points S21 and E21, the longer the movement distance of the pupil 112 at the specified time, and the shorter the movement trajectory of the pupil 112 indicated by the line, the shorter the movement trajectory of the pupil 112 at the specified time. Indicates that the moving distance is short. Also, the larger the detection point interval, the higher the moving speed of the pupil 112, and the smaller the detection point interval, the lower the moving speed of the pupil 112.

  A subject with typical development does not shake the head or the entire body. As shown in FIG. 13, in a subject with a fixed development, the moving distance of the pupil 112 at a specified time is short and the moving speed is low. In addition, the number of inflection points of the movement vector of the pupil 112 at the specified time is small.

  FIG. 14 is a diagram schematically illustrating eye movement data of a subject with ADHD. FIG. 14 is a diagram schematically illustrating the movement trajectory of the subject's eye in the three-dimensional global coordinate system. FIG. 14 shows the movement trajectory of the subject's eye during the specified time. The specified time for the eye movement trajectory of the routinely developed subject described with reference to FIG. 13 is equal to the specified time for the eye movement trajectory of the ADHD subject described with reference to FIG.

  In FIG. 14, a point S22 indicates the position of the pupil 112 at the start time of the specified time. Point E22 indicates the position of pupil 112 at the end of the specified time. The point A22 (A22a, A22b, A22c, A22d, A22e) is the position of the inflection point of the movement vector of the pupil 112. A detection point indicated by a dot “●” in FIG. 14 indicates the position of the pupil 112 acquired at a specified sampling period. The movement trajectory of the pupil 112 is indicated by a line connecting a plurality of detection points.

  The longer the movement trajectory of the pupil 112 indicated by the line between the points S22 and E22, the longer the movement distance of the pupil 112 at the specified time, and the shorter the movement trajectory of the pupil 112 indicated by the line, the shorter the movement trajectory of the pupil 112 at the specified time. Indicates that the moving distance is short. Also, the larger the detection point interval, the higher the moving speed of the pupil 112, and the smaller the detection point interval, the lower the moving speed of the pupil 112.

  ADHD subjects tend to rock their head or entire body frequently. As shown in FIG. 14, in an ADHD subject, the moving distance of the pupil 112 at a specified time is long and the moving speed is high. In addition, the number of inflection points of the movement vector of the pupil 112 during the specified time is large.

  As described above, the subject with normal development and the subject with ADHD have different moving distance, moving speed, and number of inflection points of the moving vector in the same specified time. At the same specified time, the moving distance of the pupil 112 of the ADHD subject is longer than the moving distance of the pupil 112 of the normal development subject. At the same specified time, the moving speed of the pupil 112 of the ADHD subject is higher than the moving speed of the pupil 112 of the normal development subject. At the same specified time, the number of inflection points of the movement vector of the pupil 112 of the subject with ADHD is larger than the number of inflection points of the movement vector of the pupil 112 of the subject with regular development.

  FIG. 15 is a schematic diagram for explaining an example of a method for acquiring eye movement data of a subject according to the present embodiment. In FIG. 15, the subject's eyes are shown moving in a two-dimensional plane, but in actuality, the subject's eyes move in a three-dimensional space.

  The position data of the pupil 112 is detected at a predetermined sampling period. In the present embodiment, the movement data acquisition unit 216 calculates a moving average for the position of the pupil 112 detected at a specified sampling period.

  In FIG. 15, the detection point indicated by the dot “●” indicates the position of the pupil 112 in the global coordinate system detected by the position detection unit 210 at a specified sampling period. A moving average point indicated by “Δ” indicates a moving average point calculated based on 10 detection points in the moving data acquisition unit 216.

  In the present embodiment, the movement trajectory of the pupil 112 of the subject at the specified time includes a moving average trajectory connecting a plurality of moving average points.

  FIG. 16 is a diagram schematically showing the relationship between the moving average point and the moving distance in the global coordinate system. The movement data acquisition unit 216 derives the movement distance of the pupil 112 at a specified time based on a moving average locus defined by a line connecting a plurality of moving average points. The moving distance of the pupil 112 is a moving distance in a three-dimensional global coordinate system. The movement data acquisition unit 216 calculates the coordinates (xn, yn, zn) of a plurality of moving average points Pn in the global coordinate system. The sum of the distances between the plurality of moving average points Pn is the moving distance of the pupil 112 during the specified time. FIG. 16 shows, as an example, coordinates (x1, y1, z1) of the moving average point P1, coordinates (x2, y2, z2) of the moving average point P2, and coordinates (x3, y3, z3) of the moving average point P3. Show. The moving distance of the pupil 112 is the sum of the distance d12 between the moving average point P1 and the moving average point P2 and the distance d23 between the moving average point P2 and the moving average point P3.

  In addition, the movement data acquisition unit 216 has an inflection point of the moving speed of the pupil 112 at a specified time and an inflection point of the movement vector of the pupil 112 at a specified time based on a moving average locus defined by a line connecting a plurality of moving average points. Deriving the number of

  The moving average of the coordinates of a plurality of detection points detected by the position detection unit 210 is calculated, and the coordinates of the moving average point are calculated, so that the accuracy of the moving data of the pupil 112 is improved. The position detection unit 210 calculates a detection point based on the image data of the pupil 112 occupying a small range among the image data acquired by the stereo camera device 102. Due to the influence of noise or the like included in the image data of the pupil 112, there is a high possibility that the position data of the detection point includes an error. According to the present embodiment, the movement data acquisition unit 216 calculates a movement average point from the detection points detected by the position detection unit 210, and acquires movement data of the pupil 112 based on the calculated movement average point. Therefore, movement data of the pupil 112 in which the error is suppressed is acquired.

  Note that the movement data acquisition unit 216 may acquire movement data including the movement locus of the pupil 112 by fitting a plurality of detection points based on the least square method.

  The movement data acquisition unit 216 acquires position data of the left and right eyes of the subject, and based on the position data of the left and right eyes, the movement distance, movement speed, and inflection point of the movement vector of the subject's eyes The eye movement data of the subject including at least one of the numbers may be acquired. The movement data acquisition unit 216 acquires the movement data of the subject's left eye and the movement data of the subject's right eye, respectively, and the movement data of the subject's left eye and the movement data of the subject's right eye, The eye movement data can be acquired with high accuracy.

(Diagnosis support processing)
Next, an example of the diagnosis support method according to the present embodiment will be described. FIG. 17 is a flowchart illustrating an example of the diagnosis support method according to the present embodiment. Note that the calibration process described with reference to FIG. 9 is performed before the diagnosis support process shown in FIG.

  The display control unit 202 causes the display device 101 to display the diagnostic image described with reference to FIG.

  The subject is instructed to watch the diagnostic image displayed on the display device 101. In the present embodiment, the subject is instructed to watch the image of the central area A1 of the display screen 101S of the display device 101. The output control unit 228 causes the voice output device 70 to output a sound such as “please look at the center image” in order to cause the subject to gaze at the image of the area A1 (step S301).

  In the present embodiment, the display control unit 202 displays a moving image in the area A1. The display control unit 202 starts playing a moving image (step S302).

  The display control unit 202 resets a timer that manages the playback time of the moving image (step S303). In the present embodiment, the playback time of the moving image is set to a specified time for the eye movement data of the subject.

  The area setting unit 218 sets an area A1 in which a moving image is displayed as a first specific area on the display screen 101S of the display device 101. Further, the area setting unit 218 sets an area A2 where an illustration or animation is displayed on the display screen 101S of the display device 101 as a second specific area. In addition, the area setting unit 218 sets an area A3 in which an image including a geometric pattern is displayed as a third specific area on the display screen 101S of the display device 101.

  The evaluation unit 226 includes a counter CA1 for calculating the existence time in which the subject's gaze point is in the region A1 that is the first specific area, and the regions A2 and 3 that have the subject's gaze point in the second specific area. The counter CA23 for calculating the existence time existing in at least one of the areas A3, which is the specific area, is reset (step S304).

  The gaze point detection unit 214 detects the gaze point of the subject on the display screen 101S (step S305). The detection of the gazing point is performed according to the procedure of step S200 described with reference to FIG.

  The gazing point detection unit 214 detects the position of the gazing point of the subject on the display screen 101S at a specified sampling period. In the present embodiment, the gazing point detection unit 214 detects the position of the gazing point of the subject every time a frame synchronization signal is output from the stereo camera device 102. In other words, the gazing point detection unit 214 detects the gazing point for each frame of the stereo camera device 102. The existence time in which the gazing point is present in the specific area is calculated based on the relative position between the gazing point detected for each frame and the specific area.

  The gaze point detection unit 214 determines whether or not the gaze point detection has failed (step S306). For example, the gaze point may fail to be detected when the subject blinks.

  In Step S306, when it is determined that the detection of the gazing point has failed (Step S306: Yes), the process of Step S315 is performed.

  When it is determined in step S306 that the detection of the gazing point is successful (step S306: No), the movement data acquisition unit 216 determines the position data of the right-eye pupil center 112C and the left-eye pupil center in the global coordinate system. The position data of 112C is acquired (step S307).

  The storage unit 226 stores the position data of the right-eye pupil center 112C and the position data of the left-eye pupil center 112C in the global coordinate system acquired by the movement data acquisition unit 216 (step S308).

  Next, the evaluation unit 226 detects a specific area where the gazing point exists among the plurality of specific areas set by the area setting unit 218 based on the position data of the gazing point (step S309).

  Based on the position data of the gazing point, the determination unit 220 determines whether or not the gazing point exists in the area A1 that is the first specific area (step S310).

  In step S310, when it is determined that the gazing point exists in the area A1 (step S310: Yes), the evaluation unit 226 increments the counter CA1 (step S311).

  When it is determined in step S310 that the gazing point does not exist in the area A1 (step S310: No), the determination unit 220 determines, based on the position data of the gazing point, the area A2 in which the gazing point is the second specific area. (Step S312).

  In Step S312, when it is determined that the gazing point is present in the area A2 (Step S312: Yes), the evaluation unit 226 increments the counter CA23 (Step S314).

  When it is determined in step S312 that the gazing point does not exist in the area A2 (step S312: No), the determination unit 220 determines, based on the position data of the gazing point, the area A3 in which the gazing point is the third specific area. (Step S313).

  In step S313, when it is determined that the gazing point exists in the area A3 (step S313: Yes), the evaluation unit 226 increments the counter CA23 (step S314).

  If it is determined in step S313 that the gazing point does not exist in the area A3 (step S313: No), the process of step S315 is performed. In step S313, determining that the gazing point does not exist in the area A3 means that there is no gazing point in the areas A1, A2, and A3, which are the specific areas set in the area setting unit 218. .

  Next, the display control unit 202 determines whether or not the playback time of the moving image in the area A1 has passed the specified time based on the timer (step S315).

  If it is determined in step S315 that the playback time of the moving image has not passed the specified time (step S315: No), the process returns to step S305, and gazing point detection is continued.

  If it is determined in step S315 that the playback time of the moving image has passed the specified time (step S315: Yes), the display control unit 202 stops the playback of the moving image in the area A1 (step S316).

  In the present embodiment, the prescribed time when evaluating the movement data of the subject's pupil center 112C is the time from the start of the playback of the movie in the area A1 to the stop of the playback of the movie. The point S21 described with reference to FIG. 13 and the point S22 described with reference to FIG. 14 include the position of the pupil center 112C of the subject detected when the reproduction of the moving image is started. The point E21 described with reference to FIG. 13 and the point E22 described with reference to FIG. 14 include the position of the pupil center 112C of the subject detected when the reproduction of the moving image is stopped.

  Next, the movement data acquisition unit 216 moves the moving distance of the pupil center 112C of the right eye of the subject at the specified time, the moving speed of the pupil center 112C of the right eye of the subject at the specified time, and the pupil of the right eye of the subject at the specified time. Movement data including the movement vector of the center 112C is acquired. In addition, the movement data acquisition unit 216 moves the moving distance of the pupil center 112C of the subject's left eye at the specified time, the moving speed of the pupil center 112C of the subject's left eye at the specified time, and the pupil center of the left eye of the subject at the specified time. Movement data including the movement vector of 112C is acquired (step S317).

  The movement vector of the pupil center 112C includes the movement direction of the pupil center 112C. By calculating the movement vector of the pupil center 112C, the movement data acquisition unit 216 can derive the number of inflection points of the movement vector of the pupil center 112C.

  Further, the movement data acquisition unit 216 calculates an average value of the movement data of the right-eye pupil center 112C and the movement data of the left-eye pupil center 112C (step S318).

  The average value of the movement data of the right eye pupil center 112C and the movement data of the left eye pupil center 112C is the average value of the movement distance of the right eye pupil center 112C and the left eye pupil center 112C, right The average value of the moving speed of the pupil center 112C of the eye and the moving speed of the pupil center 112C of the left eye, the number of inflection points of the moving vector of the pupil center 112C of the right eye, and the moving vector of the pupil center 112C of the left eye Including the average value with the number of inflection points.

  For example, even when the subject closes the eyelid of either the left eye or the right eye and only movement data of the pupil center 112C of either the left or right eye of the subject is acquired, the pupil center of the right eye By calculating the average value of the movement data of 112C and the movement data of the pupil center 112C of the left eye, a decrease in the accuracy of the acquired eye movement data is suppressed.

  Next, an evaluation operation is performed. The evaluation calculation includes a pupil movement evaluation calculation, a gaze point evaluation calculation, and a comprehensive evaluation calculation. The pupil movement evaluation calculation is a process of calculating evaluation data regarding the movement state of the subject's head based on the movement data of the subject's pupil center 112C. The gazing point evaluation calculation is a process of calculating evaluation data on the movement state of the subject's gazing point based on the position data of the gazing point of the subject. The comprehensive evaluation calculation is a process of calculating evaluation data regarding the risk of ADHD of the subject based on the evaluation data calculated by the pupil movement evaluation calculation and the evaluation data calculated by the gaze point evaluation calculation.

  The calculation unit 222 performs pupil movement evaluation calculation (step S319). In the present embodiment, the calculation unit 222 performs the calculation of the evaluation function of equation (1).

Ans1 = (K1 × dis) + (K2 × sp) + (K3 × n) (1)
However,
Ans1: Evaluation value,
dis: distance traveled,
sp: moving speed,
n: number of inflection points,
K1: weighting factor for travel distance,
K2: weighting factor for moving speed,
K3: weighting factor for the number of inflection points,
It is.

  In the present embodiment, the moving speed sp is an average moving speed at a specified time. The weighting factors K1, K2, and K3 may be set as appropriate based on, for example, a large number of data collected from a plurality of subjects in the past, or may be set as appropriate according to the state of the subjects.

  The evaluation value Ans1 indicates evaluation data regarding the moving state of the subject's head. A larger evaluation value Ans1 means that the subject's head moves more greatly, moves faster, or vibrates. Since the evaluation value Ans1 indicates the evaluation value of the variation state of the subject's head, the larger the evaluation value Ans1, the higher the risk of ADHD for the subject.

  Next, the calculating part 222 performs a gaze point evaluation calculation (step S320). In the present embodiment, the calculation unit 222 calculates a ratio between the count value of the counter CA1 and the count value of the counter CA23 as an evaluation value. The count value of the counter CA1 indicates time data indicating the presence time in which the gazing point is present in the area A1 which is the first specific area. The count value of the counter CA23 indicates time data indicating the existence time in which the gazing point exists in at least one of the area A2 that is the second specific area and the area A3 that is the third specific area. The calculation unit 222 increments the counter CA1 based on the determination data of the determination unit 220 in step S310, and calculates time data indicating the presence time in which the gazing point is present in the area A1 that is the first specific area. The calculation unit 222 increments the counter CA23 based on the determination data of the determination unit 220 in step S312 and step S313, and the region A2 in which the gazing point is the second specific area and the region A3 that is the third specific area. Time data indicating the existence time existing in at least one of them is calculated.

  In the present embodiment, the evaluation unit 226 performs the calculation of the evaluation function of equation (2).

  Ans2 = CA23 / CA1 (2)

  The evaluation value Ans2 indicates evaluation data regarding the movement state of the subject's gaze point. As the evaluation value Ans2 is larger, that is, as the ratio of the counter CA23 is higher, it means that the subject is not gazing at the designated area A1. Since the evaluation value Ans2 indicates the evaluation value of the variation state of the subject's gaze point, the larger the evaluation value Ans2, the higher the risk of the subject's ADHD.

  Next, the evaluation unit 226 performs a comprehensive evaluation calculation (step S321). The evaluation unit 226 evaluates the evaluation value Ans1 for the movement data of the subject's pupil center 112C calculated in step S319 and the evaluation value for the time data indicating the existence time in which the gazing point is present in the specific area calculated in step S320. Based on Ans2, comprehensive evaluation calculation is performed and the evaluation data of the subject is output.

  In the present embodiment, the evaluation unit 226 performs the calculation of the evaluation function of Expression (3).

Ans3 = (K4 × Ans1) + (K5 × Ans2) (3)
However,
K4: a weighting factor for the variation state of the head,
K5: weighting factor for the changing state of the gazing point
It is.

  The weighting factors K4 and K5 may be set as appropriate based on, for example, a large number of data collected from a plurality of subjects in the past, or may be set as appropriate according to the state of the subjects.

  Next, the output control unit 228 outputs the evaluation value Ans3, which is the evaluation data of the subject calculated by the evaluation unit 226, to the display device 101 or the output device 50 (step S322). Further, the output control unit 228 outputs the movement data of the pupil center 112 </ b> C of the subject acquired by the movement data acquisition unit 216 to the display device 101 or the output device 50. Further, the output control unit 228 outputs the evaluation value Ans1 and the evaluation value Ans2 that are evaluation data of the subject to the display device 101 or the output device 50.

  The output control unit 228 causes the display device 101 or the output device 50 to display at least one of the evaluation value Ans1, the evaluation value Ans2, and the evaluation value Ans3, which are evaluation data of the subject. Further, the output control unit 228 causes the display device 101 or the output device 50 to display movement data of the subject's pupil center 112C. The output control unit 228 causes the display device 101 or the output device 50 to display the image data of the subject's eye movement trajectory described with reference to, for example, FIG. Further, the output control unit 228 causes the display device 101 or the output device 50 to display the position data of the subject's gaze point.

  In the present embodiment, the evaluation unit 226 calculates evaluation data indicating whether or not the subject has a high risk of ADHD based on the evaluation value Ans1. The storage unit 224 stores threshold value data indicating a threshold value for the evaluation value Ans1. The threshold value for the evaluation value Ans1 is defined in advance based on, for example, a large number of data collected from a plurality of subjects in the past. The evaluation unit 226 can output the evaluation data of the subject based on the threshold data stored in the storage unit 224 and the evaluation value Ans1 calculated by the calculation unit 222. When the evaluation value Ans1 is less than the threshold value stored in the storage unit 224, the evaluation unit 226 can output first evaluation data indicating that the subject has a low risk of ADHD. The evaluation unit 226 can output second evaluation data indicating that the subject has a high risk of ADHD when the evaluation value Ans1 is equal to or greater than the threshold stored in the storage unit 224. One of the first evaluation data and the second evaluation data output from the evaluation unit 226 is displayed on the display device 101 or the output device 50.

  As shown in the equation (1), in this embodiment, the moving distance dis of the subject's pupil center 112C at the specified time is weighted by the weighting coefficient K1, and the moving speed sp of the subject's pupil center 112C at the specified time is weighted. Weighted by the coefficient K2, the number n of inflection points of the movement vector of the subject's pupil center 112C at the specified time is weighted by the weighting coefficient K3. The evaluation unit 226 indicates that the risk of ADHD of the subject is low when the evaluation value Ans1 that is the sum of the weighted moving distance dis, the moving speed sp, and the number n of inflection points of the moving vector is less than the threshold value. The first evaluation data is output, and when the evaluation value Ans1 is equal to or greater than the threshold value, the second evaluation data indicating that the subject has a high risk of ADHD can be output.

  In the present embodiment, the evaluation unit 226 calculates evaluation data indicating whether or not the subject has a high risk of ADHD based on the evaluation value Ans2. The storage unit 224 stores threshold data indicating a threshold for the evaluation value Ans2. The threshold value for the evaluation value Ans2 is defined in advance based on, for example, a large number of data collected from a plurality of subjects in the past. The evaluation unit 226 can output the evaluation data of the subject based on the threshold data stored in the storage unit 224 and the evaluation value Ans2 calculated by the calculation unit 222. When the evaluation value Ans2 is less than the threshold value stored in the storage unit 224, the evaluation unit 226 can output first evaluation data indicating that the subject has a low risk of ADHD. The evaluation unit 226 can output second evaluation data indicating that the subject has a high risk of ADHD when the evaluation value Ans2 is equal to or greater than the threshold stored in the storage unit 224. One of the first evaluation data and the second evaluation data output from the evaluation unit 226 is displayed on the display device 101 or the output device 50.

  In the present embodiment, the evaluation unit 226 calculates evaluation data indicating whether or not the subject has a high risk of ADHD based on the evaluation value Ans3. The storage unit 224 stores threshold data indicating a threshold for the evaluation value Ans3. The threshold value for the evaluation value Ans3 is defined in advance based on a large number of data collected from a plurality of subjects in the past, for example. The evaluation unit 226 can output the evaluation data of the subject based on the threshold data stored in the storage unit 224 and the evaluation value Ans3 calculated by the calculation unit 222. When the evaluation value Ans3 is less than the threshold value stored in the storage unit 224, the evaluation unit 226 can output first evaluation data indicating that the subject has a low risk of ADHD. The evaluation unit 226 can output second evaluation data indicating that the subject has a high risk of ADHD when the evaluation value Ans3 is equal to or greater than the threshold value stored in the storage unit 224. One of the first evaluation data and the second evaluation data output from the evaluation unit 226 is displayed on the display device 101 or the output device 50.

  Note that the evaluation value Ans1 calculated based on the equation (1) includes a moving distance dis weighted by the weighting factor K1, a moving speed sp weighted by the weighting factor K2, and an inflection weighted by the weighting factor K3. It is the sum of the number of points n. At least one of the moving distance dis, the moving speed sp, and the number of inflection points n may not be weighted. The evaluation unit 226 may output the subject's evaluation data based only on the moving distance dis, may output the subject's evaluation data based only on the moving speed sp, or the number of inflection points. The evaluation data of the subject may be output based only on n.

  The evaluation unit 226 can calculate evaluation data indicating whether or not the subject has a high risk of ADHD based on the movement data of the pupil center 112C of the subject acquired by the movement data acquisition unit 216. The storage unit 224 stores threshold data indicating a threshold for movement data of the subject's pupil center 112C. The threshold for the movement data is defined in advance based on a large number of data collected from a plurality of subjects, for example. The evaluation unit 226 can output the evaluation data of the subject based on the threshold data stored in the storage unit 224 and the movement data of the pupil center 112C of the subject acquired by the movement data acquisition unit 216.

  In the present embodiment, the storage unit 224 stores a threshold value for the moving distance of the subject's pupil center 112C during a specified time. The evaluation unit 226 indicates that the subject's ADHD risk is low when the movement distance of the subject's pupil center 112C acquired by the movement data acquisition unit 216 is less than the threshold stored in the storage unit 224. Evaluation data can be output. The evaluation unit 226 indicates that the subject's ADHD risk is high when the movement distance of the subject's pupil center 112C acquired by the movement data acquisition unit 216 is equal to or greater than a threshold stored in the storage unit 224. Evaluation data can be output. One of the first evaluation data and the second evaluation data output from the evaluation unit 226 is displayed on the display device 101 or the output device 50.

  In the present embodiment, the storage unit 224 stores a threshold value for the moving speed of the subject's pupil center 112C during the specified time. The evaluation unit 226 indicates that the subject's ADHD risk is low when the movement speed of the subject's pupil center 112C acquired by the movement data acquisition unit 216 is less than the threshold stored in the storage unit 224. Evaluation data can be output. The evaluation unit 226 indicates that the subject's ADHD risk is high when the moving speed of the subject's pupil center 112 </ b> C acquired by the movement data acquisition unit 216 is equal to or higher than the threshold stored in the storage unit 224. Evaluation data can be output. One of the first evaluation data and the second evaluation data output from the evaluation unit 226 is displayed on the display device 101 or the output device 50.

  Further, in the present embodiment, the storage unit 224 stores a threshold value regarding the number of inflection points of the movement vector of the subject's pupil center 112C at a specified time. When the number of inflection points of the movement vector of the subject's pupil center 112C acquired by the movement data acquisition unit 216 is less than the threshold stored in the storage unit 224, the evaluation unit 226 determines that the risk of ADHD of the subject is It is possible to output first evaluation data indicating low. When the number of inflection points of the movement vector of the subject's pupil center 112C acquired by the movement data acquisition unit 216 is equal to or greater than the threshold stored in the storage unit 224, the evaluation unit 226 determines that the risk of ADHD of the subject is The second evaluation data indicating high can be output. One of the first evaluation data and the second evaluation data output from the evaluation unit 226 is displayed on the display device 101 or the output device 50.

[Action and effect]
As described above, according to the present embodiment, the movement data acquisition unit 216 acquires the movement data of the eye of the subject defined in the global coordinate system. The movement data of the eye of the subject indicates the movement state of the head of the subject. ADHD subjects have the characteristic of frequently shaking their head or entire body. In the present embodiment, the movement data acquisition unit 216 acquires the movement data of the subject's eyes, and the acquired movement data of the eyes of the subject is output from the output control unit 228. Therefore, the diagnosis support apparatus 100 can acquire the movement state of the subject's head and evaluate the risk of ADHD of the subject. Therefore, the diagnosis support apparatus 100 can support the diagnosis of ADHD risk of the subject with high accuracy.

  In the present embodiment, the gaze point of the subject is detected based on the corneal reflection method. In the corneal reflection method, the subject is irradiated with infrared light emitted from the light source 103, the subject's eyes irradiated with the infrared light are photographed with a camera, and the corneal reflection image 113, which is a reflected image of the light source on the corneal surface, is applied. In this method, the position of the pupil 112 is detected to detect the subject's gaze and gaze point. In the present embodiment, the eye movement data includes at least one of movement data of the pupil 112 and movement data of the cornea reflection image 113. Therefore, based on the image data of the subject's eye 111 acquired by the image data acquisition unit 206, the diagnosis support apparatus 100 performs the detection of the position data of the subject's gaze point and the acquisition of the movement data of the subject's eye in parallel. Can be implemented.

  In the present embodiment, the eye movement data of the subject includes the eye movement distance dis at a specified time, the eye movement speed sp at a specified time, and the number n of inflection points of the eye movement vector at the specified time. Including at least one. The tendency of the movement distance “dis”, the movement speed “sp”, and the number of inflection points “n” is remarkably different between those with typical development and those with ADHD. Therefore, by obtaining at least one of the movement distance dis, the movement speed sp, and the number of inflection points n as the movement data of the subject's eyes, the diagnosis support apparatus 100 can increase the diagnosis of ADHD risk of the subject. Can help with accuracy.

  In the present embodiment, threshold data indicating a threshold for the movement data of the subject's eyes is defined. Therefore, the evaluation unit 226 uses the first evaluation data indicating that the subject's ADHD risk is low and the second evaluation indicating that the subject's ADHD risk is high based on the threshold data and the subject's eye movement data. At least one of the data can be output.

  In the present embodiment, the area A1, the area A2, and the area A3 are set as specific areas by the area setting unit 218 on the display screen 101S of the display device 101. In the diagnosis support process, the determination unit 220 determines whether or not the subject's gazing point exists in the specific area. Based on the determination data of the determination unit 220, the calculation unit 222 has first time data indicating the time of presence in the area A1 in which the gazing point is the first specific area, and the gazing point is the second specific area. Second time data indicating the existence time existing in at least one of the region A2 and the region A3 which is the third specific area is calculated. In the present embodiment, the count value of the counter CA1 is calculated as the first time data, and the count value of the counter CA23 is calculated as the second time data. As shown in the equation (2), the evaluation unit 226 uses the first time data indicated by the count value of the counter CA1 and the second time data indicated by the count value of the counter CA23 to pay attention to the subject. An evaluation value Ans2 indicating the movement state is calculated. The evaluation unit 226 receives at least one of the first evaluation data indicating that the subject's ADHD risk is low and the second evaluation data indicating that the subject's ADHD risk is high based on the movement state of the subject's gaze point. Can be output.

  In this embodiment, as shown in the equation (3), the evaluation unit 226 calculates both the evaluation value Ans1 for the eye movement data of the subject and the evaluation value Ans2 for the time data of the gaze point of the subject. It is possible to calculate an evaluation value Ans3, which is comprehensive evaluation data regarding the ADHD risk of the subject. Therefore, the diagnosis support apparatus 100 can evaluate the risk of ADHD of the subject with high accuracy.

  In the above-described embodiment, the count value of the counter CA1 indicating the existence time when the gazing point is present in the area A1 and the counter CA23 indicating the existence time when the gazing point is present in at least one of the area A2 and the area A3 are calculated. A numerical value is calculated, and the movement state of the gaze point of the subject is acquired based on the count value of the counter CA1 and the count value of the counter CA23. The count value of the counter CA1 indicating the presence time in which the gazing point is present in the area A1 and the count value of the counter CA 234 indicating the presence time in which the gazing point is present in at least one of the area A2, the area A3, and the area A4 are calculated. Then, based on the count value of the counter CA1 and the count value of the counter CA234, the movement state of the subject's gaze point may be acquired.

  Further, in the above-described embodiment, the count value of the counter CA1 indicating the existence time when the gazing point exists in the area A1, the count value of the counter CA2 indicating the existence time when the gazing point exists in the area A2, and the gazing point is the area. The counter CA3 indicating the presence time existing in A3 may be calculated, and the movement state of the subject's point of interest may be acquired based on the count value of the counter CA1 and the count value of the counter CA3. As described above, the geometric pattern that the ASD subject tends to see is displayed in the area A3. The evaluation unit 226 can evaluate the ASD risk of the subject with high accuracy based on the ratio between the count value of the counter CA3 and the count value of the counter CA1.

  In the above-described embodiment, a moving image of a natural image including a person or a large human image is displayed in the area A1 of the diagnostic image. In the area A1, an animated movie including a person may be displayed.

  20 Computer System, 20A Arithmetic Processing Device, 20B Storage Device, 20C Computer Program, 30 Input / Output Interface Device, 40 Drive Circuit, 50 Output Device, 60 Input Device, 70 Audio Output Device, 100 Diagnosis Support Device (Gaze Detection Device), 101 display device, 101S display screen, 102 stereo camera device, 102A first camera, 102B second camera, 103 light source, 103A first light source, 103B second light source, 103C light source, 103V virtual light source, 109 cornea radius of curvature, 110 cornea Center of curvature, 111 eyes, 112 pupil, 112C pupil center, 113 cornea reflection image, 113C cornea reflection center, 121 cornea reflection center, 122 cornea reflection center, 123 straight line, 124 cornea reflection center, 126 distance, 130 target position, 1 5 Gaze point, 166 Gaze point, 202 Display control unit, 204 Light source control unit, 206 Image data acquisition unit, 208 Input data acquisition unit, 210 Position detection unit, 212 Curvature center calculation unit, 214 Gaze point detection unit, 216 Movement data Acquisition unit, 218 area setting unit, 220 determination unit, 222 calculation unit, 224 storage unit, 226 evaluation unit, 228 output control unit, 302 input / output unit, 402 display device drive unit, 404A first camera input / output unit, 404B first 2 camera input / output unit, 406 light source driving unit.

Claims (8)

An image data acquisition unit for acquiring image data of the eyes of the subject;
Based on the image data, a gaze point detection unit that detects position data of the gaze point of the subject,
Based on the image data, a movement data acquisition unit that acquires movement data of the subject's eyes;
An output control unit for outputting the movement data;
A diagnosis support apparatus comprising: The movement data includes at least one of the movement distance of the eye at a specified time, the moving speed of the eye at a specified time, and the number of inflection points of the eye movement vector at a specified time.
The diagnosis support apparatus according to claim 1. A storage unit for storing threshold data indicating a threshold for the movement data;
Based on the threshold data and the movement data, an evaluation unit that outputs the evaluation data of the subject,
The diagnosis support apparatus according to claim 1 or 2, further comprising: The movement data includes at least one of the movement distance of the eye at a specified time, the movement speed of the eye at a specified time, and the number of inflection points of the eye movement vector at a specified time,
The threshold includes a threshold for at least one of the moving distance, the moving speed, and the number of inflection points of the moving vector,
The evaluation unit outputs first evaluation data when at least one of the moving distance, the moving speed, and the number of inflection points of the moving vector is less than a threshold value, and outputs second evaluation data when the number is greater than or equal to the threshold value. ,
The diagnosis support apparatus according to claim 3. The evaluation unit weights each of the moving distance, the moving speed, and the number of inflection points of the moving vector, and weights the moving distance, the moving speed, and the number of inflection points of the moving vector. First evaluation data is output when the sum is less than the threshold, and second evaluation data is output when the sum is equal to or greater than the threshold.
The diagnosis support apparatus according to claim 3. A display control unit for displaying diagnostic image data on a display device;
An area setting unit for setting a specific area on the display screen of the display device;
A determination unit that determines whether the gazing point exists in the specific area based on the position data of the gazing point;
A calculation unit that calculates time data indicating an existing time in which the gazing point is present in the specific area based on determination data of the determination unit;
The evaluation unit outputs the evaluation data of the subject based on the time data and the movement data.
The diagnosis support apparatus according to any one of claims 1 to 5. Obtaining image data of the subject's eyes;
Detecting position data of the gaze point of the subject based on the image data;
Obtaining movement data of the subject's eyes based on the image data;
Outputting the movement data;
A diagnostic support method including: On the computer,
Obtaining image data of the subject's eyes;
Detecting position data of the gaze point of the subject based on the image data;
Obtaining movement data of the subject's eyes based on the image data;
Outputting the movement data;
A computer program that executes