JP6245093B2 - Diagnosis support apparatus and diagnosis support method - Google Patents

Description

  The present invention relates to a diagnosis support apparatus and a diagnosis support method.

  Recently, people with developmental disabilities are said to be on the rise. It is known that developmental disabilities are found early and start treatment to reduce symptoms and increase the effect of adapting to society. In Japan, we are aiming for early detection through interviews at the age of 1 and a half. However, there are problems such as shortage of psychiatrists and time-consuming interviews, and the effect is not sufficient. Therefore, there is a need for an objective and efficient diagnosis support device for developmental disabilities.

  For early detection of developmental disabilities, it is ideal to be able to diagnose at the age of 1 and a half years. One characteristic of children with developmental disabilities (especially autism) is that they do not look at the eyes of their opponents. It is also known that children with developmental disabilities prefer geometrical images over human images.

  In addition, a method has been proposed for diagnosing developmental disabilities by applying a method of detecting a gazing point by photographing a human face with a camera and calculating a corneal reflection and a pupil position.

JP 2005-185431 A JP 2008-125619 A JP 2005-198743 A JP 2011-206542 A

Pierce K et al., “Preference for Geometric Patterns Early in Life as a Risk Factor for Autism.”, Arch Gen Psychiatry. 2011 Jan; 68 (1): 101-109.

  Although a method for supporting diagnosis of a subject with developmental disabilities is known, a more accurate detection method has been demanded.

  The present invention has been made in view of the above, and an object thereof is to provide a diagnosis support apparatus and a diagnosis support method capable of improving the accuracy of diagnosis.

  In order to solve the above-described problems and achieve the object, the present invention provides a display unit, an imaging unit that images a subject, and a line of sight that detects the gaze direction of the subject from a captured image captured by the imaging unit. A detection unit; a viewpoint detection unit that detects the viewpoint of the subject in the display area of the display unit based on the line-of-sight direction; an image of the object; and an image of a person indicating the direction of the object. An output control unit that displays a diagnostic image on the display unit, and an evaluation unit that calculates an evaluation value of the subject based on the viewpoint detected by the viewpoint detection unit when the diagnostic image is displayed. It is characterized by that.

  The diagnosis support apparatus and diagnosis support method according to the present invention have an effect of improving the accuracy of diagnosis.

FIG. 1 is a diagram illustrating an example of an arrangement of a display unit, a stereo camera, and a light source used in the first embodiment. FIG. 2 is a diagram illustrating an outline of functions of the diagnosis support apparatus according to the first embodiment. FIG. 3 is a block diagram illustrating an example of detailed functions of the respective units illustrated in FIG. FIG. 4 is a diagram illustrating an example of eye and distance detection when two cameras are used. FIG. 5A is an explanatory diagram of an example of a diagnostic image. FIG. 5B is an explanatory diagram of an example of the diagnostic image. FIG. 6 is a flowchart illustrating an example of the diagnosis support process. FIG. 7A is an explanatory diagram of another example of the diagnostic image. FIG. 7B is an explanatory diagram of another example of the diagnostic image. FIG. 8 is a flowchart illustrating another example of the diagnosis support process. FIG. 9 is a flowchart illustrating another example of the diagnosis support process. FIG. 10 is a flowchart illustrating an example of a diagnosis support process when a plurality of diagnosis images are used. FIG. 11 is a flowchart illustrating an example of a diagnosis support process when a plurality of diagnosis images are used. FIG. 12A is an explanatory diagram of another example of a diagnostic image. FIG. 12B is an explanatory diagram of another example of the diagnostic image. FIG. 12C is an explanatory diagram of another example of the diagnostic image. FIG. 12D is an explanatory diagram of another example of the diagnostic image. FIG. 13 is a diagram illustrating an example of the arrangement of the display unit, the stereo camera, the infrared light source, and the subject according to the second embodiment. FIG. 14 is a diagram illustrating an example of the arrangement of the display unit, the stereo camera, the infrared light source, and the subject according to the second embodiment. FIG. 15 is a diagram illustrating an outline of functions of the diagnosis support apparatus. FIG. 16 is a block diagram illustrating an example of detailed functions of each unit illustrated in FIG. 15. FIG. 17 is a diagram illustrating an outline of processing executed by the diagnosis support apparatus according to the second embodiment. FIG. 18 is an explanatory diagram showing the difference between the method using two light sources and the second embodiment using one light source. FIG. 19 is a diagram for explaining calculation processing for calculating the distance between the pupil center position and the corneal curvature center position. FIG. 20 is a flowchart illustrating an example of calculation processing according to the second embodiment. FIG. 21 is a diagram showing a method of calculating the position of the corneal curvature center using the distance obtained in advance. FIG. 22 is a flowchart illustrating an example of a line-of-sight detection process according to the second embodiment. FIG. 23 is a diagram for explaining the calculation process of the modification. FIG. 24 is a flowchart illustrating an example of a modification calculation process.

  Hereinafter, embodiments of a diagnosis support apparatus and a diagnosis support method according to the present invention will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment.

(First embodiment)
In the diagnosis support for infants with developmental disabilities, a plurality of tests are performed and comprehensively determined in order to improve the detection accuracy of information related to developmental disabilities. Diagnosis tests include tests that use the feature that children with developmental disabilities do not see the eyes of people, and tests that use the feature that children with developmental disabilities prefer geometrical images over human images.

  For example, infants with developmental disabilities do not see the object pointed by the guardian, do not look at the object that the guardian looks at, and do not point and request what they want There is. Therefore, the diagnosis support apparatus according to the present embodiment displays a person and an image in which the person indicates one of a plurality of gaze objects. In addition, the diagnosis support apparatus according to the present embodiment measures the dwell time of a gazing point for each region such as a region including an object pointed by a person with a finger and a region including an object other than the object, and develops Calculate the evaluation value of disability. This realizes highly accurate assessment support for developmental disability risk.

  FIG. 1 is a diagram illustrating an example of an arrangement of a display unit, a stereo camera, and a light source used in the first embodiment. As shown in FIG. 1, in this embodiment, a set of stereo cameras 102 is arranged below the display screen 101. The stereo camera 102 is an imaging unit that can perform stereo shooting with infrared rays, and includes a right camera 202 and a left camera 204.

  Infrared LED (Light Emitting Diode) light sources 203 and 205 are arranged in the circumferential direction immediately before the lenses of the right camera 202 and the left camera 204, respectively. The infrared LED light sources 203 and 205 include an inner peripheral LED and an outer peripheral LED having different wavelengths for emitting light. The pupils of the subject are detected by the infrared LED light sources 203 and 205. As a pupil detection method, for example, the method described in Patent Document 2 can be applied.

  When detecting the line of sight, the space is expressed by coordinates to identify the position. In this embodiment, the center position of the display screen 101 is the origin, the top and bottom are the Y coordinate (up is +), the side is the X coordinate (right is +), and the depth is the Z coordinate (front is +). .

  FIG. 2 is a diagram illustrating an outline of functions of the diagnosis support apparatus 100. FIG. 2 shows a part of the configuration shown in FIG. 1 and a configuration used for driving the configuration. As shown in FIG. 2, the diagnosis support apparatus 100 includes a right camera 202, a left camera 204, infrared LED light sources 203 and 205, a speaker 105, a drive / IF (interface) unit 208, and a control unit 300. A storage unit 150 and a display unit 210. In FIG. 2, the display screen 101 shows the positional relationship between the right camera 202 and the left camera 204 in an easy-to-understand manner, but the display screen 101 is a screen displayed on the display unit 210. The drive unit and the IF unit may be integrated or separate.

  The speaker 105 functions as an audio output unit that outputs an audio or the like for alerting the subject during calibration.

  The drive / IF unit 208 drives each unit included in the stereo camera 102. The drive / IF unit 208 serves as an interface between each unit included in the stereo camera 102 and the control unit 300.

  The storage unit 150 stores various information such as a control program, a measurement result, and a diagnosis support result. The storage unit 150 stores, for example, an image to be displayed on the display unit 210. The display unit 210 displays various information such as a target image for diagnosis.

  FIG. 3 is a block diagram illustrating an example of detailed functions of the respective units illustrated in FIG. As shown in FIG. 3, a display unit 210 and a drive / IF unit 208 are connected to the control unit 300. The drive / IF unit 208 includes camera IFs 314 and 315, an LED drive control unit 316, and a speaker drive unit 322.

  A right camera 202 and a left camera 204 are connected to the drive / IF unit 208 via camera IFs 314 and 315, respectively. The driving / IF unit 208 drives these cameras to image the subject.

  The infrared LED light source 203 includes a wavelength 1-LED 303 and a wavelength 2-LED 304. The infrared LED light source 205 includes a wavelength 1-LED 305 and a wavelength 2-LED 306.

  Wavelength 1-LEDs 303 and 305 emit infrared light having wavelength 1. Wavelength 2-LEDs 304 and 306 irradiate wavelength 2 infrared rays.

  The wavelength 1 and the wavelength 2 are, for example, a wavelength of less than 900 nm and a wavelength of 900 nm or more, respectively. When the reflected light reflected by the pupil is irradiated with infrared light having a wavelength of less than 900 nm, a bright pupil image is obtained as compared with the case where reflected light reflected by the pupil is irradiated with infrared light having a wavelength of 900 nm or more. It is because it is obtained. In addition, about the wavelength of the infrared rays to irradiate, it is not restricted to the above, The result which imaged the reflected light reflected by the pupil by irradiating the infrared rays of wavelength 1, and the reflection reflected by the pupil after irradiating the infrared rays of wavelength 2 What is necessary is just to be able to make a difference with the result of imaging light.

  The speaker driving unit 322 drives the speaker 105. The diagnosis support apparatus 100 may include an interface (printer IF) for connecting to a printer as a printing unit. In addition, the printer may be provided inside the diagnosis support apparatus 100.

  The control unit 300 controls the entire diagnosis support apparatus 100. The control unit 300 includes a line-of-sight detection unit 351, a viewpoint detection unit 352, an output control unit 353, and an evaluation unit 354.

  The gaze detection unit 351 detects the gaze (gaze direction) of the subject from the captured image captured by the imaging unit (stereo camera 102). The process for detecting the line of sight includes a process for detecting the eye position of the subject. The viewpoint detection unit 352 detects the viewpoint of the subject using the detected gaze direction. For example, the viewpoint detection unit 352 detects a viewpoint (gaze point) that is a point that the subject gazes out of the target image displayed on the display screen 101. As a gaze detection method by the gaze detection unit 351 and a viewpoint detection method by the viewpoint detection unit 352, any conventionally used method can be applied. Below, similarly to patent document 3, it demonstrates as an example the case where a test subject's gaze direction and a gaze point are detected using a stereo camera.

  In this case, the gaze detection unit 351 first detects the gaze direction of the subject from the image captured by the stereo camera 102. The gaze detection unit 351 detects the gaze direction of the subject using, for example, the methods described in Patent Documents 1 and 2. Specifically, the line-of-sight detection unit 351 obtains the difference between the image captured by irradiating the infrared light having the wavelength 1 and the image captured by irradiating the infrared light having the wavelength 2, and the image in which the pupil image is clarified. Generate. The line-of-sight detection unit 351 uses the two images generated as described above from the images captured by the left and right cameras (the right camera 202 and the left camera 204) and uses the stereo vision technique to determine the position of the subject's pupil ( Eye position) is calculated. The line-of-sight detection unit 351 calculates the position of the subject's corneal reflection using images taken by the left and right cameras. Then, the gaze detection unit 351 calculates a gaze vector representing the gaze direction of the subject from the position of the pupil of the subject and the corneal reflection position.

  In addition, the detection method of the eye position and the line of sight of the subject is not limited to this. For example, the eye position and line of sight of the subject may be detected by analyzing an image captured using visible light instead of infrared light.

  The viewpoint detection unit 352 detects, for example, the intersection of the line-of-sight vector represented in the coordinate system as shown in FIG. 1 and the XY plane as the gaze point of the subject. When the gaze direction of both eyes is obtained, the gaze point may be measured by obtaining the intersection of the left and right gazes of the subject.

  FIG. 4 is a diagram illustrating an example of eye and distance detection when two cameras (the right camera 202 and the left camera 204) are used. For the two cameras, a camera calibration theory based on a stereo calibration method is applied in advance to obtain camera parameters. As the stereo calibration method, any conventionally used method such as a method using Tsai's camera calibration theory can be applied. Using the eye position detected from the image captured by the right camera 202, the eye position detected from the image captured by the left camera 204, and the camera parameters, the three-dimensional coordinates of the eye in the world coordinate system are obtained. It is done. Thereby, the distance between the eyes and the stereo camera 102 and the pupil coordinates can be estimated. The pupil coordinate is a coordinate value representing the position of the subject's eye (pupil) on the XY plane. The pupil coordinates can be, for example, coordinate values obtained by projecting the eye position expressed in the world coordinate system onto the XY plane. Usually, the pupil coordinates of the left and right eyes are obtained. A diagnostic image is displayed on the display screen 101. As will be described later, the diagnostic image includes, for example, an image of a target object, an image of a person pointing to the target object, and an image of an indicator such as a finger used for pointing. The pointing object is not limited to the finger, and any object such as a rod-shaped object can be applied as long as it can point to another object.

  Returning to FIG. 3, the output control unit 353 controls the output of various information to the display unit 210, the speaker 105, and the like. For example, the output control unit 353 controls the output to the display unit 210 such as the diagnostic image and the evaluation result by the evaluation unit 354. The output control unit 353 causes the display unit 210 to display a diagnostic image including an image of the object and an image of a person pointing to the object.

  The evaluation unit 354 calculates an evaluation value as an index related to the degree of developmental disability based on the diagnostic image and the gazing point detected by the viewpoint detection unit 352. For example, the evaluation unit 354 calculates an evaluation value based on the position of the gaze point of the subject when a diagnostic image as illustrated in FIGS. A specific example of the evaluation value calculation method will be described later. The evaluation unit 354 may calculate the evaluation value based on the diagnostic image and the gazing point, and the calculation method is not limited to the embodiment.

  5A and 5B are explanatory diagrams illustrating an example of a diagnostic image. FIG. 5A and FIG. 5B are examples of images representing a situation where a person points to an object. Note that the image may be either a still image or a moving image (video), but a moving image is desirable because it is easier to watch. As shown in FIGS. 5A and 5B, the person points to the object to be watched while gazing at the object.

  An area including the object is denoted as T, an area including an indicator (such as a hand) is denoted as F, and an area including the face is denoted as E. Let C be the other area of the screen. On the screen, for example, the position is expressed by coordinates where the upper left is (0, 0) and the lower right is (Xmax, Ymax). That is, the X coordinate is positive on the right and the Y coordinate is positive on the bottom.

  A diagnosis support process in the case of using a diagnostic image including one object, an indicator, and one face as shown in FIGS. 5A and 5B will be described. FIG. 6 is a flowchart showing an example of the diagnosis support process in this case.

First, the output control unit 353 starts video reproduction (step S101). The output control unit 353 resets a timer that measures a time slightly shorter than the video playback time (step S102). The evaluation unit 354 resets (initializes) the following counters used for calculating the evaluation value (step S103).
A counter CE that counts up when the region E including the face is watched
-Counter CF that counts up when an area F including an indicator (such as a hand) is watched
-Counter CT that counts up when the region T including the object is watched
-Counter CC that counts up when an area C other than the above area is watched

  The gazing point measurement is performed, for example, for each frame of a stereo camera that captures images synchronously. That is, the gaze point is measured at predetermined time intervals. Further, the count value of each counter corresponds to the gaze time (gaze point stop time).

  Next, the viewpoint detection unit 352 performs gaze point detection (step S104). The viewpoint detection unit 352 determines whether or not gazing point detection has failed (step S105). Gaze point detection fails when an image of the pupil and corneal reflection cannot be obtained due to blinking or the like. In addition, when the gazing point does not exist in the display screen 101 (when the subject looks at something other than the display screen 101), it is also determined as a failure.

  If gazing point detection fails (step S105: Yes), the process proceeds to step S117 in order not to affect each counter. When the gazing point detection is successful (step S105: No), the viewpoint detection unit 352 detects in which region the subject's gazing point exists from the coordinates of the gazing point (step S106). Next, the viewpoint detection unit 352 determines whether or not a gazing point exists in the region E (step S107). When the gazing point exists in the region E (step S107: Yes), the viewpoint detection unit 352 counts up the counter CE (step S108). When the gazing point does not exist in the area E (step S107: No), the viewpoint detection unit 352 determines whether or not the gazing point exists in the area F (step S109). When the gazing point exists in the area F (step S109: Yes), the viewpoint detection unit 352 counts up the counter CF (step S110). When the gazing point does not exist in the region F (step S109: No), the viewpoint detection unit 352 determines whether the gazing point exists in the region T (step S111). When the gazing point exists in the region T (step S111: Yes), the viewpoint detection unit 352 counts up the counter CT (step S112). When the gazing point does not exist in the region T (step S111: No), that is, when the gazing point does not exist in any of the regions E, F, and T, the viewpoint detection unit 352 counts up the counter CC ( Step S115).

  As a result, the count value of the counter of the gaze area increases, and the gaze stop time can be measured.

  Next, the output control unit 353 determines whether the timer is completed (step S117). If the timer has not been completed (step S117: No), the process returns to step S104 and is repeated. When the timer is completed (step S117: Yes), the output control unit 353 stops the reproduction of the video (step S118).

  Next, the evaluation unit 354 performs evaluation calculation related to diagnosis support for the developmental disability risk based on the stop time of the gazing point in each region (step S119). Next, the evaluation unit 354 outputs an evaluation value and ends (step S120).

  Many methods can be applied to the evaluation calculation in step S119. The simplest method is a method of calculating the gazing point stopping time itself (count value of the counter CT) in the region T including the object as an evaluation value. Subjects with typical development tend to gaze at objects in the indicated direction. Therefore, the lower the value of the stop time, the higher the possibility of developmental disability.

In addition, subjects with typical development tend to see a person's face and tend to look beyond the line of sight when pointing. Therefore, the accuracy is further improved by taking into account the stop time of the gazing point in the region including the face (region E). For example, an evaluation value ANS calculated by the following equation (1) may be used. Kt and Ke are weighting coefficients. Kt> Ke.
ANS = (Kt × CT) + (Ke × CE) (1)

Furthermore, since a subject with a typical development has a high tendency to see an indicator (such as a pointing hand), the accuracy can be further improved by taking into account the stop time of the gazing point in the region F. For example, an evaluation value ANS calculated by the following equation (2) may be used. Kf is a weighting coefficient.
ANS = (Kt × CT) + (Kf × CF) (2)

Furthermore, the evaluation value ANS calculated by the following equation (3) may be used by combining the above equations (1) and (2).
ANS = (Kt × CT) + (Ke × CE) + (Kf × CF) (3)

A subject with a high risk of developmental disability has a high probability of gazing outside the predetermined area C. For this reason, if the stop time of the gazing point in the region C is taken into consideration, the accuracy is further improved. For example, an evaluation value ANS calculated by the following equation (4) may be used. Kc is a weighting coefficient.
ANS =
(Kt × CT) + (Ke × CE) + (Kf × CF) − (Kc × CC) (4)

  The diagnostic images in FIGS. 5A and 5B are examples, and the diagnostic images used for the diagnosis are not limited to these. For example, a diagnostic image including a plurality of objects may be used.

  7A and 7B are explanatory diagrams illustrating other examples of the diagnostic image. FIG. 7A and FIG. 7B are examples of diagnostic images that further include other objects similar to the indicated object. In FIGS. 7A and 7B, two areas including other objects are referred to as an area A and an area B, respectively. A subject with a developmental disorder has a higher probability of gazing at the region A or the region B than the region T. Therefore, the detection accuracy is improved by using the diagnostic images as shown in FIGS.

  A diagnosis support process in the case of using a diagnostic image including a plurality of objects, an indicator, and a face as shown in FIGS. FIG. 8 is a flowchart showing an example of the diagnosis support process in this case.

  6 is different from FIG. 6 in that step S213, step S214, and step S216 are added. Further, the difference is that the counter CAB is further reset (initialized) in step S203. Further, the evaluation calculation in step S219 is different from that in step S119. The other steps are the same as those in FIG.

  Steps S 213, S 214, and S 216 are processes related to the area A and the area B.

  If the gazing point does not exist in the region T (step S211: No), the viewpoint detection unit 352 determines whether or not the gazing point exists in the region A (step S213). When the gazing point exists in the area A (step S213: Yes), the viewpoint detection unit 352 counts up the counter CAB (step S216). When the gazing point does not exist in the area A (step S213: No), the viewpoint detection unit 352 determines whether or not the gazing point exists in the area B (step S214). When the gazing point exists in the area B (step S214: Yes), the viewpoint detection unit 352 counts up the counter CAB (step S216). When the gazing point does not exist in the region B (step S214: No), the viewpoint detection unit 352 counts up the counter CC (step S215).

Thus, the counter CAB is used in the example of FIG. The counter CAB is a counter that counts up when the area A or the area B is watched. The evaluation calculation in step S219 is different from step S119 in that the counter CAB is used as follows. That is, the evaluation unit 354 may use an evaluation value ANS calculated by the following equation (5).
ANS = (Kt × CT) + (Ke × CE) − (Kab × CAB) (5)

Furthermore, since a subject with a typical development has a high tendency to see an indicator (such as a pointing hand), the accuracy can be further improved by taking into account the stop time of the gazing point in the region F. For example, an evaluation value ANS calculated by the following equation (6) may be used.
ANS = (Kt × CT) + (Kf × CF) − (Kab × CAB) (6)

Furthermore, the evaluation value ANS calculated by the following equation (7) may be used by combining the above equations (5) and (6).
ANS =
(Kt × CT) + (Ke × CE) + (Kf × CF) − (Kab × CAB) (7)

A subject with a high risk of developmental disability has a high probability of gazing outside the predetermined area C. For this reason, if the stop time of the gazing point in the region C is taken into consideration, the accuracy is further improved. For example, an evaluation value ANS calculated by the following equation (8) may be used.
ANS = (Kt × CT) + (Ke × CE) + (Kf × CF)
− (Kc × CC) − (Kab × CAB) (8)

  Another example of the diagnosis support process when using a diagnostic image including a plurality of objects will be described. FIG. 9 is a flowchart showing another example of the diagnosis support process in this case. In this example, a video is started in a state where the person does not point to the target, and a diagnostic image that points to the target T is used while the person is gazing at the target T after a lapse of a predetermined time from the start. That is, after the subject is alerted to the movement of the hand or finger, a diagnosis support process is realized that takes advantage of the tendency to see the direction indicated by the hand or finger. Such a configuration further improves the accuracy of diagnosis.

  9 is different from FIG. 8 in that step S302-1 is added. In step S302-1, the viewpoint detection unit 352 performs a process of waiting for the elapse of a predetermined time using a timer or the like (step S302-1). The predetermined time is a time until the person completes the operation of pointing the object. Accordingly, it is possible to measure the gaze point of the subject after completing the pointing operation. In FIG. 9, the viewpoint detection unit 352 uses a timer or the like to perform a process of waiting for the elapse of a predetermined time. Instead, the evaluation unit outputs from the viewpoint detection unit after the elapse of the predetermined time. An evaluation value may be calculated based on the obtained value.

  So far, an example using one diagnostic image has been described, but a plurality of diagnostic images may be used. 10 and 11 are flowcharts illustrating an example of a diagnosis support process when a plurality of diagnosis images are used. FIG. 10 and FIG. 11 are examples in the case where, for example, two images as shown in FIGS. 7-1 and 7-2 are continuously displayed and comprehensively evaluated. Two videos are connected, for example. The same procedure can be applied when three or more videos are displayed continuously.

  10 is different from FIG. 8 in that step S403 is added. In step S403, the viewpoint detection unit 352 sets each area (area E, F, T, A, B) for the first video (video 1). Since two different videos are displayed, area setting corresponding to each video is performed in this step. 11 is different from FIG. 8 in that step S422 is added. In step S422, the viewpoint detection unit 352 sets each area (area E, F, T, A, B) for the second video (video 2). For example, the viewpoint detection unit 352 sets a region for each video with reference to coordinate values that are predetermined for each video and stored in the storage unit 150 or the like.

  In the example of FIGS. 10 and 11, the gazing point stop time in each region is continuously measured for two videos in which the positions of the object and the person are different from each other. By this method, it is possible to eliminate the influence of preference and tendency when the subject observes the video. For example, as shown in FIGS. 7A and 7B, if two diagnostic images in which the positions of the object and the person are symmetrical are used, even if the subject is easy to see only on the left side or the subject easy to see only on the right side, By eliminating the influence of this tendency, more accurate diagnosis can be realized. Also, the image of the object is preferably an image that is not a geometric pattern image that the child with developmental disabilities tends to view. For example, a character image, a natural image (animal, plant, natural landscape, etc.), a vehicle, etc., which can be distinguished from a human image are preferable.

As described above, according to the present embodiment, for example, the following effects can be obtained.
(1) It becomes possible to measure the reaction to the infant's pointing.
(2) Since the reaction after displaying and pointing a plurality of objects can be measured, the accuracy is improved.
(3) Since the characteristics of gazing at the fingertips and face of a person can be taken into account, the accuracy is improved.

(Modification 1)
In the above example, a diagnostic image including an image of a person pointing to an object and an image of an indicator such as a finger used for pointing is used. The diagnostic image is not limited to the above example. As described above, an infant with a developmental disability has a feature that the guardian does not look at an object to be watched with his face facing. In this modification, in consideration of this feature, a diagnostic image is used in which a person turns his / her face to an object and gazes at the object.

  12A to 12D are explanatory diagrams illustrating other examples of diagnostic images. FIG. 12A and FIG. 12B are examples of images showing how a person turns his / her face in the direction of the gaze target and looks at the gaze target. The person in the diagnostic image turns his / her face in the direction while gazing at the object.

  In FIGS. 12A and 12B, it is assumed that a region including the object is T and a region including the face is E. Let C be the other area of the screen. On the screen, for example, the position is expressed by coordinates where the upper left is (0, 0) and the lower right is (Xmax, Ymax). That is, the X coordinate is positive on the right and the Y coordinate is positive on the bottom.

Diagnosis support processing in the case of using diagnostic images as shown in FIGS. 12A and 12B can be realized by the same flowchart as in FIG. Since the diagnostic image of this modification does not include the region F including the indicator, for example, the following processing relating to the region F in the flowchart of FIG. 6 can be omitted.
-Initialization of the counter CF in step S103-Step S109
Step S110

Also in this modification, various evaluation calculation methods can be applied as follows.
A method for calculating the gazing point stop time itself (count value of the counter CT) in the region T including the object as an evaluation value. A method using the evaluation value ANS calculated by the above equation (1). The above equation (4). A method using an evaluation value ANS calculated by the following equation (9) similar to ANS = (Kt × CT) + (Ke × CE) − (Kc × CC) (9)

  12-3 and 12-4 are explanatory diagrams illustrating other examples of the diagnostic image. FIGS. 12-3 and 12-4 are examples of diagnostic images that further include other objects similar to the object. In FIG. 12-3 and FIG. 12-4, the two areas including other objects are referred to as an area A and an area B, respectively. A subject with a developmental disorder has a higher probability of gazing at the region A or the region B than the region T. Therefore, the accuracy of detection is improved by using diagnostic images as shown in FIGS. 12-3 and 12-4.

Diagnosis support processing in the case of using diagnostic images as shown in FIGS. 12-3 and 12-4 can be realized by, for example, the same flowchart as in FIG. Since the diagnostic image of this modification does not include the region F including the indicator, for example, the following processing related to the region F in the flowchart of FIG. 8 can be omitted.
-Initialization of the counter CF in step S203-Step S209
Step S210

When using FIG. 12-3 and FIG. 12-4, the following evaluation calculation method may be applied.
A method using the evaluation value ANS calculated by the above equation (5) A method using the evaluation value ANS calculated by the following equation (10) similar to the above equation (8) ANS = (Kt × CT) + ( Ke x CE)
− (Kc × CC) − (Kab × CAB) (10)

  Similarly to FIG. 9, a diagnostic image in which a person gazes at an object after a predetermined time may be used. That is, the image is started in a state where the person does not turn his face to the object and does not gaze, and after a predetermined time has elapsed from the start, the person turns his face to the object T and uses the diagnostic image to gaze at the object T Also good. In the first modification, the orientation of the person's face has been described. However, the face orientation is not limited to the face orientation. In other words, after the subject is alerted to the movement of the person's face and eyes, a diagnosis support process is realized that takes advantage of the tendency to see the direction indicated by the face and eyes. Such a configuration further improves the accuracy of diagnosis. Further, it may be realized by both the face direction and the eye direction of the person, or may be realized by all the directions of the pointing object such as a finger used for pointing to the object, the face direction, and the eye direction. .

  Similar to FIG. 10 described above, a method may be applied in which a plurality of videos are continuously displayed and comprehensively evaluated. For example, two images as shown in FIGS. 12-1 and 12-2, or FIGS. 12-3 and 12-4 may be continuously displayed for comprehensive evaluation. The two videos are connected, for example, and both are configured with a predetermined time.

As described above, according to the present modification, in addition to the effects of the first embodiment, for example, the following effects can be obtained.
(1) Since the pointing is not included, the movement of the line of sight following the movement is suppressed, and the specificity of the measurement is increased.

(Second Embodiment)
In the second embodiment, a line-of-sight detection device and a line-of-sight detection method capable of further simplifying the apparatus configuration are realized as compared with the first embodiment.

  Hereinafter, the line-of-sight detection device and the line-of-sight detection method of the second embodiment will be described in detail with reference to the drawings. In addition, this invention is not limited by this embodiment. In the following, an example in which the line-of-sight detection apparatus is used as a diagnosis support apparatus that supports diagnosis of developmental disabilities using the line-of-sight detection result will be described. Applicable devices are not limited to diagnosis support devices.

  The line-of-sight detection apparatus (diagnosis support apparatus) of the present embodiment detects the line of sight using an illumination unit installed at one place. In addition, the line-of-sight detection device (diagnosis support device) of the present embodiment calculates the corneal curvature center position with high accuracy by using a result obtained by gazing at one point on the subject before the line-of-sight detection.

  In addition, an illumination part is an element which can irradiate light to a test subject's eyeball including a light source. The light source is an element that generates light, such as an LED (Light Emitting Diode). A light source may be comprised from one LED, and may be comprised by combining several LED and arrange | positioning in one place. Hereinafter, the “light source” may be used as a term representing the illumination unit in this way.

  13 and 14 are diagrams illustrating an example of the arrangement of the display unit, the stereo camera, the infrared light source, and the subject according to the second embodiment. In addition, about the structure similar to 1st Embodiment, the same code | symbol may be attached | subjected and description may be abbreviate | omitted.

  As illustrated in FIG. 13, the diagnosis support apparatus according to the second embodiment includes a display unit 210, a stereo camera 2102, and an LED light source 2103. The stereo camera 2102 is disposed below the display unit 210. The LED light source 2103 is arranged at the center position of two cameras included in the stereo camera 2102. The LED light source 2103 is a light source that irradiates near infrared rays having a wavelength of 850 nm, for example. FIG. 13 shows an example in which an LED light source 2103 (illumination unit) is configured by nine LEDs. Note that the stereo camera 2102 uses a lens that can transmit near-infrared light having a wavelength of 850 nm.

  As illustrated in FIG. 14, the stereo camera 2102 includes a right camera 2202 and a left camera 2203. The LED light source 2103 irradiates near-infrared light toward the eyeball 111 of the subject. In the image acquired by the stereo camera 2102, the pupil 112 is reflected and darkened with low brightness, and the corneal reflection 113 generated as a virtual image in the eyeball 111 is reflected and brightened with high brightness. Accordingly, the positions of the pupil 112 and the corneal reflection 113 on the image can be acquired by each of the two cameras (the right camera 2202 and the left camera 2203).

  Further, the three-dimensional world coordinate values of the positions of the pupil 112 and the corneal reflection 113 are calculated from the positions of the pupil 112 and the corneal reflection 113 obtained by the two cameras. In the present embodiment, as the three-dimensional world coordinates, with the center position of the display screen 101 as the origin, the top and bottom are the Y coordinate (up is +), the side is the X coordinate (right is +), and the depth is the Z coordinate (front is +).

  FIG. 15 is a diagram illustrating an outline of functions of the diagnosis support apparatus 2100 according to the second embodiment. FIG. 15 shows a part of the configuration shown in FIGS. 13 and 14 and a configuration used for driving the configuration. As shown in FIG. 15, the diagnosis support apparatus 2100 includes a right camera 2202, a left camera 2203, an LED light source 2103, a speaker 105, a drive / IF (interface) unit 208, a control unit 2300, and a storage unit 150. And a display unit 210. In FIG. 15, the display screen 101 shows the positional relationship between the right camera 2202 and the left camera 2203 in an easy-to-understand manner, but the display screen 101 is a screen displayed on the display unit 210. The drive unit and the IF unit may be integrated or separate.

  The speaker 105 functions as an audio output unit that outputs an audio or the like for alerting the subject during calibration.

  The drive / IF unit 208 drives each unit included in the stereo camera 2102. The drive / IF unit 208 serves as an interface between each unit included in the stereo camera 2102 and the control unit 2300.

  The control unit 2300 is a communication I / F that communicates with a control device such as a CPU (Central Processing Unit) and a storage device such as a ROM (Read Only Memory) and a RAM (Random Access Memory) by connecting to a network. And a computer equipped with a bus for connecting each unit.

  The storage unit 150 stores various information such as a control program, a measurement result, and a diagnosis support result. The storage unit 150 stores, for example, an image to be displayed on the display unit 210. The display unit 210 displays various information such as a target image for diagnosis.

  FIG. 16 is a block diagram illustrating an example of detailed functions of each unit illustrated in FIG. 15. As shown in FIG. 16, a display unit 210 and a drive / IF unit 208 are connected to the control unit 2300. The drive / IF unit 208 includes camera IFs 314 and 315, an LED drive control unit 316, and a speaker drive unit 322.

  A right camera 2202 and a left camera 2203 are connected to the drive / IF unit 208 via camera IFs 314 and 315, respectively. The driving / IF unit 208 drives these cameras to image the subject.

  The speaker driving unit 322 drives the speaker 105. The diagnosis support apparatus 2100 may include an interface (printer IF) for connecting to a printer as a printing unit. Further, the printer may be provided inside the diagnosis support apparatus 2100.

  The control unit 2300 controls the entire diagnosis support apparatus 2100. The control unit 2300 includes a first calculation unit 2351, a second calculation unit 2352, a third calculation unit 2353, a gaze detection unit 2354, a viewpoint detection unit 2355, an output control unit 2356, and an evaluation unit 2357. I have. Note that the line-of-sight detection device may include at least the first calculation unit 2351, the second calculation unit 2352, the third calculation unit 2353, and the line-of-sight detection unit 2354.

  Each element included in the control unit 2300 (the first calculation unit 2351, the second calculation unit 2352, the third calculation unit 2353, the line-of-sight detection unit 2354, the viewpoint detection unit 2355, the output control unit 2356, and the evaluation unit 2357) It may be realized by software (program), may be realized by a hardware circuit, or may be realized by using software and a hardware circuit in combination.

  When implemented by a program, the program is a file in an installable or executable format, such as a CD-ROM (Compact Disk Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk Recordable), a DVD ( Digital Versatile Disk) is recorded on a computer-readable recording medium and provided as a computer program product. The program may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. The program may be provided or distributed via a network such as the Internet. The program may be provided by being incorporated in advance in a ROM or the like.

  The first calculation unit 2351 calculates the position of the pupil center (first position) indicating the center of the pupil from the eyeball image captured by the stereo camera 2102. The second calculator 2352 calculates the position of the corneal reflection center (second position) indicating the center of corneal reflection from the captured image of the eyeball. The first calculation unit 2351 and the second calculation unit 2352 correspond to a position detection unit that detects a first position indicating the center of the pupil and a second position indicating the center of corneal reflection.

  The third calculator 2353 calculates the corneal curvature center (fourth position) from the straight line (first straight line) connecting the LED light source 2103 and the corneal reflection center. For example, the third calculation unit 2353 calculates a position on the straight line where the distance from the corneal reflection center is a predetermined value as the corneal curvature center. As the predetermined value, a value determined in advance from a general radius of curvature of the cornea or the like can be used.

  Since there may be individual differences in the radius of curvature of the cornea, there is a possibility that an error will increase if the center of corneal curvature is calculated using a predetermined value. Therefore, the third calculation unit 2353 may calculate the corneal curvature center in consideration of individual differences. In this case, the third calculation unit 2353 first uses a pupil center and a corneal reflection center that are calculated when the subject is gazes at the target position (third position), and a straight line that connects the pupil center and the target position (the first position). 2) and the first straight line connecting the corneal reflection center and the LED light source 2103 is calculated. Then, the third calculation unit 2353 calculates a distance (first distance) between the pupil center and the calculated intersection, and stores it in the storage unit 150, for example.

  The target position may be a position that is determined in advance and can calculate a three-dimensional world coordinate value. For example, the center position of the display screen 101 (the origin of the three-dimensional world coordinates) can be set as the target position. In this case, for example, the output control unit 2356 displays an image (target image) or the like that causes the subject to gaze at the target position (center) on the display screen 101. Thereby, a test subject can be made to gaze at a target position.

  The target image may be any image as long as it can attract the attention of the subject. For example, an image in which a display mode such as luminance or color changes, an image in which the display mode is different from other regions, or the like can be used as the target image.

  The target position is not limited to the center of the display screen 101, and may be an arbitrary position. If the center of the display screen 101 is set as the target position, the distance from an arbitrary end of the display screen 101 is minimized. For this reason, it becomes possible to make the measurement error at the time of gaze detection smaller, for example.

  The processing up to the calculation of the distance is executed in advance, for example, before the actual gaze detection is started. At the time of actual line-of-sight detection, the third calculation unit 2353 calculates, on the straight line connecting the LED light source 2103 and the corneal reflection center, a position where the distance from the pupil center is a previously calculated distance as the corneal curvature center. . The third calculator 2353 calculates the center of corneal curvature from the position of the LED light source 2103, a predetermined position (third position) indicating the target image on the display unit, the position of the pupil center, and the position of the corneal reflection center ( This corresponds to a calculation unit that calculates (fourth position).

  The line-of-sight detection unit 2354 detects the line of sight of the subject from the pupil center and the corneal curvature center. For example, the gaze detection unit 2354 detects the direction from the corneal curvature center to the pupil center as the gaze direction of the subject.

  The viewpoint detection unit 2355 detects the viewpoint of the subject using the detected gaze direction. The viewpoint detection unit 2355 detects, for example, a viewpoint (gaze point) that is a point on the display screen 101 where the subject gazes. The viewpoint detection unit 2355 detects, for example, the intersection of the line-of-sight vector represented in the three-dimensional world coordinate system as shown in FIG. 14 and the XY plane as the gaze point of the subject.

  The output control unit 2356 controls output of various information to the display unit 210, the speaker 105, and the like. For example, the output control unit 2356 outputs the target image at the target position on the display unit 210. Further, the output control unit 2356 controls the output to the display unit 210 such as the diagnostic image and the evaluation result by the evaluation unit 2357.

  The diagnostic image may be an image corresponding to the evaluation process based on the line-of-sight (viewpoint) detection result. For example, in the case of diagnosing a developmental disorder, a diagnostic image including an image (such as a geometric pattern image) preferred by a subject with a developmental disorder and other images (such as a person image) may be used.

  The evaluation unit 2357 performs an evaluation process based on the diagnostic image and the gazing point detected by the viewpoint detection unit 2355. For example, in the case of diagnosing a developmental disorder, the evaluation unit 2357 analyzes the diagnostic image and the gazing point, and evaluates whether or not the image preferred by the subject with the developmental disorder has been gazed.

  The output control unit 2356 may display the same diagnostic image as that of the first embodiment, and the evaluation unit 2357 may perform the same evaluation process as that of the evaluation unit 354 of the first embodiment. In other words, the gaze detection process (gaze detection unit 351) of the first embodiment is replaced with the gaze detection process (first calculation unit 2351, second calculation unit 2352, third calculation unit 2353, gaze detection of the second embodiment. Part 2354). Thereby, in addition to the effect of 1st Embodiment, the effect (simplification of an apparatus structure etc.) of 2nd Embodiment can be achieved.

  FIG. 17 is a diagram illustrating an outline of processing executed by the diagnosis support apparatus 2100 of this embodiment. The elements described in FIGS. 13 to 16 are denoted by the same reference numerals and description thereof is omitted.

  The pupil center 407 and the corneal reflection center 408 represent the center of the pupil and the center of the corneal reflection point detected when the LED light source 2103 is turned on, respectively. The corneal curvature radius 409 represents the distance from the corneal surface to the corneal curvature center 410.

  FIG. 18 is an explanatory diagram showing a difference between a method using two light sources (illumination units) (hereinafter referred to as method A) and the present embodiment using one light source (illumination unit). The elements described in FIGS. 13 to 16 are denoted by the same reference numerals and description thereof is omitted.

  Method A uses two LED light sources 511 and 512 instead of the LED light source 2103. In the method A, a straight line 515 connecting the cornea reflection center 513 and the LED light source 511 when the LED light source 511 is irradiated, and a straight line 516 connecting the cornea reflection center 514 and the LED light source 512 when the LED light source 512 is irradiated. An intersection is calculated. This intersection is the corneal curvature center 505.

  In contrast, in the present embodiment, a straight line 523 connecting the corneal reflection center 522 and the LED light source 2103 when the LED light source 2103 is irradiated is considered. A straight line 523 passes through the corneal curvature center 505. It is also known that the radius of curvature of the cornea is almost constant with little influence from individual differences. Thus, the corneal curvature center when the LED light source 2103 is irradiated exists on the straight line 523 and can be calculated by using a general curvature radius value.

  However, if the viewpoint is calculated using the position of the center of corneal curvature calculated using a general radius of curvature, the viewpoint position may deviate from the original position due to individual differences in the eyeballs, and accurate viewpoint position detection cannot be performed. There is.

  FIG. 19 is a diagram for explaining calculation processing for calculating the corneal curvature center position and the distance between the pupil center position and the corneal curvature center position before performing viewpoint detection (line-of-sight detection). The elements described in FIGS. 13 to 16 are denoted by the same reference numerals and description thereof is omitted. The connection between the left and right cameras (the right camera 2202 and the left camera 2203) and the control unit 2300 is not shown and is omitted.

  The target position 605 is a position for displaying a target image or the like at one point on the display unit 210 and causing the subject to stare. In this embodiment, the center position of the display screen 101 is set. A straight line 613 is a straight line connecting the LED light source 2103 and the corneal reflection center 612. A straight line 614 is a straight line connecting the target position 605 (gaze point) that the subject looks at and the pupil center 611. A corneal curvature center 615 is an intersection of the straight line 613 and the straight line 614. The third calculation unit 2353 calculates and stores the distance 616 between the pupil center 611 and the corneal curvature center 615.

  FIG. 20 is a flowchart illustrating an example of calculation processing according to the present embodiment.

  First, the output control unit 2356 reproduces the target image at one point on the display screen 101 (step S501), and causes the subject to gaze at the one point. Next, the control unit 2300 turns on the LED light source 2103 toward the eyes of the subject using the LED drive control unit 316 (step S502). The control unit 2300 images the eyes of the subject with the left and right cameras (the right camera 2202 and the left camera 2203) (step S503).

  The pupil part is detected as a dark part (dark pupil) by irradiation of the LED light source 2103. Further, a corneal reflection virtual image is generated as a reflection of LED irradiation, and a corneal reflection point (corneal reflection center) is detected as a bright portion. That is, the first calculation unit 2351 detects a pupil portion from the captured image, and calculates coordinates indicating the position of the pupil center. For example, the first calculation unit 2351 detects, as a pupil part, a region having a predetermined brightness or less including the darkest part in a certain region including the eyes, and a region having a predetermined brightness or more including the brightest part is reflected by the cornea. Detect as. In addition, the second calculation unit 2352 detects a corneal reflection portion from the captured image, and calculates coordinates indicating the position of the corneal reflection center. Note that the first calculation unit 2351 and the second calculation unit 2352 calculate coordinate values for each of the two images acquired by the left and right cameras (step S504).

  Note that the left and right cameras are pre-calibrated with a stereo calibration method to obtain three-dimensional world coordinates, and conversion parameters are calculated. As the stereo calibration method, any conventionally used method such as a method using Tsai's camera calibration theory can be applied.

  The first calculation unit 2351 and the second calculation unit 2352 use the conversion parameters to convert the coordinates of the left and right cameras into the three-dimensional world coordinates of the pupil center and the corneal reflection center (step S505). The third calculation unit 2353 obtains a straight line connecting the obtained world coordinates of the corneal reflection center and the world coordinates of the center position of the LED light source 2103 (step S506). Next, the third calculation unit 2353 calculates a straight line connecting the world coordinates of the center of the target image displayed at one point on the display screen 101 and the world coordinates of the pupil center (step S507). The 3rd calculation part 2353 calculates | requires the intersection of the straight line calculated by step S506, and the straight line calculated by step S507, and makes this intersection a cornea curvature center (step S508). The third calculation unit 2353 calculates the distance between the pupil center and the corneal curvature center at this time, and stores it in the storage unit 150 or the like (step S509). The stored distance is used to calculate the corneal curvature center at the time of subsequent detection of the viewpoint (line of sight).

  The distance between the pupil center and the corneal curvature center when looking at one point on the display unit 210 in the calculation process is kept constant within a range in which the viewpoint in the display unit 210 is detected. The distance between the center of the pupil and the center of corneal curvature may be obtained from the average of all the values calculated during playback of the target image, or from the average of several values of the values calculated during playback. You may ask for it.

  FIG. 21 is a diagram illustrating a method of calculating the corrected position of the corneal curvature center using the distance between the pupil center and the corneal curvature center that is obtained in advance when performing viewpoint detection. A gazing point 805 represents a gazing point obtained from a corneal curvature center calculated using a general curvature radius value. A gazing point 806 represents a gazing point obtained from a corneal curvature center calculated using a distance obtained in advance.

  The pupil center 811 and the corneal reflection center 812 indicate the position of the pupil center and the position of the corneal reflection center calculated at the time of viewpoint detection, respectively. A straight line 813 is a straight line connecting the LED light source 2103 and the corneal reflection center 812. The corneal curvature center 814 is the position of the corneal curvature center calculated from a general curvature radius value. The distance 815 is the distance between the pupil center and the corneal curvature center calculated by the prior calculation process. The corneal curvature center 816 is the position of the corneal curvature center calculated using the distance obtained in advance. The corneal curvature center 816 is obtained from the fact that the corneal curvature center exists on the straight line 813 and the distance between the pupil center and the corneal curvature center is the distance 815. Thus, the line of sight 817 calculated when a general radius of curvature value is used is corrected to the line of sight 818. Also, the gazing point on the display screen 101 is corrected from the gazing point 805 to the gazing point 806. The connection between the left and right cameras (the right camera 2202 and the left camera 2203) and the control unit 2300 is not shown and is omitted.

  FIG. 22 is a flowchart illustrating an example of a line-of-sight detection process according to the present embodiment. For example, the line-of-sight detection process of FIG. 22 can be executed as the process of detecting the line of sight in the diagnostic process using the diagnostic image. In the diagnostic process, in addition to the steps in FIG. 22, a process for displaying a diagnostic image, an evaluation process by the evaluation unit 2357 using the detection result of the gazing point, and the like are executed.

  Steps S601 to S605 are the same as steps S502 to S506 in FIG.

  The third calculation unit 2353 calculates, as the corneal curvature center, a position on the straight line calculated in step S605 and whose distance from the pupil center is equal to the distance obtained by the previous calculation process (step S606).

  The line-of-sight detection unit 2354 obtains a vector (line-of-sight vector) connecting the pupil center and the corneal curvature center (step S607). This vector indicates the line-of-sight direction viewed by the subject. The viewpoint detection unit 2355 calculates the three-dimensional world coordinate value of the intersection between the line-of-sight direction and the display screen 101 (step S608). This value is a coordinate value representing one point on the display unit 210 that the subject gazes in world coordinates. The viewpoint detection unit 2355 converts the obtained three-dimensional world coordinate value into a coordinate value (x, y) represented in the two-dimensional coordinate system of the display unit 210 (step S609). Thereby, the viewpoint (gaze point) on the display part 210 which a test subject looks at can be calculated.

(Modification 2)
The calculation process for calculating the distance between the pupil center position and the corneal curvature center position is not limited to the method described with reference to FIGS. Hereinafter, another example of the calculation process will be described with reference to FIGS.

  FIG. 23 is a diagram for explaining calculation processing of the present modification. Elements described in FIGS. 13 to 16 and FIG. 19 are denoted by the same reference numerals and description thereof is omitted.

  A line segment 1101 is a line segment (first line segment) connecting the target position 605 and the LED light source 103. A line segment 1102 is a line segment (second line segment) that is parallel to the line segment 1101 and connects the pupil center 611 and the straight line 613. In this modification, the distance 616 between the pupil center 611 and the corneal curvature center 615 is calculated and stored using the line segment 1101 and the line segment 1102 as follows.

  FIG. 24 is a flowchart illustrating an example of calculation processing according to the present modification.

  Steps S701 to S707 are the same as steps S501 to S507 in FIG.

  The third calculation unit 2353 calculates and calculates a line segment (the line segment 1101 in FIG. 11) connecting the center of the target image displayed at one point on the screen of the display unit 101 and the center of the LED light source 103. The length of the line segment (L1101) is calculated (step S708).

  The third calculation unit 2353 calculates a line segment (line segment 1102 in FIG. 23) that passes through the pupil center 611 and is parallel to the line segment calculated in step S708, and the calculated line segment length (L1102). Is calculated (step S709).

  The third calculation unit 2353 includes a triangle having the corneal curvature center 615 as a vertex and the line segment calculated in step S708 as a lower side, and a triangle having the corneal curvature center 615 as a vertex and the line segment calculated in step S709 as a lower side. Are similar to each other, a distance 616 between the pupil center 611 and the corneal curvature center 615 is calculated (step S710). For example, the third calculation unit 2353 causes the ratio of the length of the line segment 1102 to the length of the line segment 1101 and the ratio of the distance 616 to the distance between the target position 605 and the corneal curvature center 615 to be equal. The distance 616 is calculated.

The distance 616 can be calculated by the following equation (11). L614 is the distance from the target position 605 to the pupil center 611.
Distance 616 = (L614 × L1102) / (L1101−L1102) (11)

  The third calculation unit 2353 stores the calculated distance 616 in the storage unit 150 (step S711). The stored distance is used to calculate the corneal curvature center at the time of subsequent detection of the viewpoint (line of sight).

As described above, according to the present embodiment, for example, the following effects can be obtained.
(1) It is not necessary to arrange light sources (illuminating units) at two places, and it becomes possible to perform line-of-sight detection with the light sources arranged at one place.
(2) Since the number of light sources is one, the apparatus can be made compact and the cost can be reduced.

100, 2100 Diagnosis support device 101 Display screen 102, 2102 Stereo camera 105 Speaker 150 Storage unit 202, 2202 Right camera 203 Infrared LED light source 204, 2203 Left camera 205 Infrared LED light source 208 Drive / IF unit 210 Display unit 300, 2300 Control unit 351, 2354 Gaze detection unit 352, 2355 View point detection unit 353, 2356 Output control unit 354, 2357 Evaluation unit 2351 First calculation unit 2352 Second calculation unit 2353 Third calculation unit

Claims (10)

A display unit;
An imaging unit for imaging a subject;
From a captured image captured by the imaging unit, a gaze detection unit that detects the gaze direction of the subject,
A viewpoint detection unit that detects the viewpoint of the subject in the display area of the display unit based on the line-of-sight direction;
An output control unit that causes the display unit to display a diagnostic image including an image of the object and an image of a person indicating the direction of the object;
An evaluation unit that calculates an evaluation value of the subject based on the viewpoint detected by the viewpoint detection unit when the diagnostic image is displayed;
A diagnostic support apparatus comprising: The image of the person is an image of a person indicating the direction of the object with a finger;
The diagnosis support apparatus according to claim 1. The image of the person is an image of a person indicating the direction of the object by face or eye orientation;
The diagnosis support apparatus according to claim 1, wherein The evaluation unit is to calculate the evaluation value based on the between first time dwell representing the time of the viewpoint in the region including the image of the object is detected,
The diagnosis support apparatus according to any one of claims 1 to 3. The evaluation unit may further include a second stop time indicating a time during which the viewpoint is detected in an area including the face of the person image and an area including an image of an indicator indicating the direction of the object. Calculating the evaluation value based on at least one of the third stop time representing the detected time ;
The diagnosis support apparatus according to claim 4 . The evaluation unit further calculates the evaluation value based on a fourth stop time representing a time during which the viewpoint is detected in a region other than a region including the object image and a region including the person image. ,
The diagnosis support apparatus according to claim 5 . The output control unit displays a first diagnostic image on the display unit, and then a second diagnosis in which the position of the image of the object and the position of the image of the person are different from the first diagnostic image. An image is displayed on the display unit,
The evaluation unit is detected by the viewpoint detection unit when the first diagnostic image is displayed and the viewpoint detected by the viewpoint detection unit and when the second diagnostic image is displayed. Calculating the evaluation value based on the viewpoint;
The diagnosis support apparatus according to any one of claims 1 to 6, wherein: The output control unit causes the display unit to display a diagnostic image on which an image indicating the direction of the object is displayed after a predetermined time has elapsed since the display was started,
The evaluation unit calculates the evaluation value of the subject based on the viewpoint detected by the viewpoint detection unit after the predetermined time has elapsed since the display of the diagnostic image was started;
The diagnosis support apparatus according to claim 1, wherein: An illumination unit including a light source that emits light;
A plurality of imaging units;
A position detection unit that calculates a first position indicating the center of the pupil and a second position indicating the center of corneal reflection from the image of the eyeball of the subject imaged by the imaging unit and irradiated with light; ,
A calculation unit that calculates a fourth position indicating the center of curvature of the cornea based on the position of the light source, the third position on the display unit, the first position, and the second position;
The line-of-sight detection unit detects the line of sight of the subject based on the first position and the fourth position;
The diagnosis support apparatus according to claim 1, wherein : A line-of-sight detection step for detecting a line-of-sight direction of the subject from a captured image captured by an imaging unit that images the subject,
A viewpoint detection step of detecting the viewpoint of the subject in the display area of the display unit based on the line-of-sight direction;
An output control step of causing the display unit to display a diagnostic image including an image of the object and an image of a person indicating the direction of the object;
An evaluation step of calculating an evaluation value of the subject based on the viewpoint detected by the viewpoint detection step when the diagnostic image is displayed;
A diagnostic support method including: